CN110019440A - The processing method and processing device of data - Google Patents
The processing method and processing device of data Download PDFInfo
- Publication number
- CN110019440A CN110019440A CN201710764963.2A CN201710764963A CN110019440A CN 110019440 A CN110019440 A CN 110019440A CN 201710764963 A CN201710764963 A CN 201710764963A CN 110019440 A CN110019440 A CN 110019440A
- Authority
- CN
- China
- Prior art keywords
- data
- configurable interface
- module
- interface file
- name
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
Abstract
The invention discloses a kind of processing method and processing devices of data, are related to technical field of information processing, and main purpose is to solve the problems, such as traditional data warehouse poor expandability and maintainable reduction.Technical solution of the present invention includes: acquisition data processing request, includes the data name of pending data in the data processing request；The first configurable interface file is called, pending data corresponding with the data name is obtained from the data memory module；The second configurable interface file is called, the corresponding processing method of corresponding with the data name pending data is obtained from the data processing module；The pending data is handled according to the processing method of acquisition；The third configurable interface file is called, corresponding with data name data application pattern is determined from the data application module, and Apply Styles displaying treated pending data based on the data.
Description
Technical field
The present invention relates to technical field of information processing, more particularly to a kind of processing method and processing device of data.
Background technique
Traditional data warehouse be issued transaction is carried out centered on single data resource, and the data of data warehouse and
Information is extracted from different data sources, is had the characteristics that analysis processing, is DSS (Decision Support
System, DSS) and on-line analytical processing (On-Line Analysis Processing, OLAP) structural data environment,
The support of all types data can be provided.With using the time to increase, data warehouse needs support more data processings and more
Complicated user query, so data warehouse is while keeping original system ability, it is also necessary to which there is addition new function component
Ability.But since the mechanism degree of coupling of traditional data warehouse system interior is high, DSS and data mining
Functional unit is tightly combined, and causes the scalability of data warehouse and maintainability poor, is not able to satisfy addition new function component
Demand.
In order to improve the expansibility and maintainability in traditional data warehouse, following two side is mainly taken in the prior art
Method: first method is extended to each layer in system using Stratified Strategy, in every layer of increase component, by by system
Component models improve the scalability of data warehouse；Second method is to be placed on component using high performance hardware
On different hardware, the scalability of data warehouse is improved by expansible software parallel and hardware concurrent.
During inventor states invention in realization, discovery in the prior art, according to above-mentioned first method, then can be led
Cause system single module is continuously increased, and the inner couplings of each layer are continuously improved, and reduces the maintainability of system；According to above-mentioned
Second method then needs to establish high performance platform and runs, high performance software parallel and hardware concurrent, and performance in system
The overall performance that weaker component can reduce data warehouse is horizontal, increases the exploitation and maintenance difficulties of data warehouse.Therefore, existing
There is technology cannot achieve, while ensureing data warehouse maintainability, improves the scalability in traditional data warehouse.
Summary of the invention
In view of this, a kind of processing method and processing device of data provided by the invention, main purpose is to solve traditional number
According to warehouse poor expandability and maintainable the problem of reducing.
To solve the above-mentioned problems, present invention generally provides following technical solutions:
In a first aspect, the present invention provides a kind of data warehouse, comprising:
Data management layer, metadata control centre is provided in the data management layer, and the metadata control centre is logical
It crosses multiple configurable interface files and multiple and different functional modules carries out data interaction；
Wherein, each configurable interface file and a functional module be there are mapping relations, it is the multiple can
Configuring interface document includes: the first configurable interface file, the second configurable interface file, third configurable interface file；Institute
Stating multiple and different functional modules includes: data memory module, data processing module, data application module；
The data memory module, for storing data；
The data processing module, for recording the corresponding processing method of the data；
The data application module, the Apply Styles for the data that record that treated.
Second aspect, the present invention provides a kind of processing methods of data, and the method is applied in data warehouse, described
Data warehouse includes data management layer, multiple configurable interface files and multiple and different functional modules；The data management layer
In be provided with metadata control centre, the multiple configurable interface file includes: the first configurable interface file, second can match
Set interface document, third configurable interface file；The plurality of different functional module includes: at data memory module, data
Manage module, data application module；The metadata control centre passes through in multiple configurable interface files and the data warehouse
Multiple and different functional modules carry out data interactions, one in each configurable interface file and the data warehouse
Functional module is there are mapping relations, and record has corresponding the included data of functional module in each configurable interface file
Data name；
Data processing request is obtained, includes the data name of pending data in the data processing request；
The first configurable interface file is called, is obtained from the data memory module corresponding with the data name
Pending data；
The second configurable interface file is called, is obtained from the data processing module corresponding with the data name
The corresponding processing method of the pending data；
The pending data is handled according to the processing method of acquisition；
The third configurable interface file is called, determination is corresponding with the data name from the data application module
Data application pattern, and Apply Styles show treated pending data based on the data.
Optionally, in the first configurable interface file by calling metadata control centre, from data memory module
Before obtaining pending data, the method also includes:
The data memory module, data processing module, the classification data group for including in data application module are obtained respectively
Part, the classification data component is for recording sorted data in different function module；
Data name in the classification data component is recorded in the first configurable interface file, second can configure
In interface document, third configurable interface file, wherein the data memory module corresponds to the first configurable interface text
Part, the data processing module correspond to the second configurable interface file, and the data application module corresponds to the third can
Configure interface document.
Optionally, the method also includes:
The classification data component is monitored in real time, determines the classification data component with the presence or absence of update；
If it exists, then configurable interface file is updated based on updated classification data component.
Optionally, the first configurable interface file based on metadata control centre, from data memory module obtain to
Handling data includes:
The data name of pending data is obtained, and determines and whether there is the number in the first configurable interface file
According to title；
If it is determined that in the presence of the corresponding pending data of the data name is then obtained from the data memory module.
The third aspect, the present invention also provides a kind of processing units of data, and described device is applied in data warehouse, described
Data warehouse includes data management layer, multiple configurable interface files and multiple and different functional modules；The data management layer
In be provided with metadata control centre, the multiple configurable interface file includes: the first configurable interface file, second can match
Set interface document, third configurable interface file；The plurality of different functional module includes: at data memory module, data
Manage module, data application module；The metadata control centre passes through in multiple configurable interface files and the data warehouse
Multiple and different functional modules carry out data interactions, one in each configurable interface file and the data warehouse
Functional module is there are mapping relations, and record has corresponding the included data of functional module in each configurable interface file
Data name；Include:
First acquisition unit includes pending data in the data processing request for obtaining data processing request
Data name；
First call unit, for calling the first configurable interface file；
Second acquisition unit is used for after first call unit calls the first configurable interface file, from
Pending data corresponding with the data name is obtained in the data memory module；
Second call unit, for calling the second configurable interface file；
Third acquiring unit is used for after second call unit calls the second configurable interface file, from
The corresponding processing method of corresponding with the data name pending data is obtained in the data processing module；
Processing unit, the processing method for being obtained according to the third acquiring unit to the pending data into
Row processing；
Third call unit, for calling the third configurable interface file；
First determination unit is used for after the third call unit calls the third configurable interface file, from
Data application pattern corresponding with the data name is determined in the data application module；
Display unit, for being applied after first determination unit determines data application pattern based on the data
Pattern shows treated pending data.
Optionally, described device further include:
4th acquiring unit, for obtaining pending data from the data memory module in the second acquisition unit
Before, the data memory module, data processing module, the classification data component for including in data application module are obtained respectively,
The classification data component is for recording sorted data in different function module；
Recording unit, the data name record in the classification data component for obtaining the 4th acquiring unit
In the first configurable interface file, the second configurable interface file, third configurable interface file, wherein the number
The first configurable interface file is corresponded to according to memory module, the data processing module corresponds to the second configurable interface text
Part, the data application module correspond to the third configurable interface file.
Optionally, described device further include:
Monitoring unit, for being monitored in real time to the classification data component；
Second determination unit, for determining described point after the monitoring unit is monitored the classification data component
Class data package is with the presence or absence of update；
Updating unit, for being based on updated classification data group when determining that the classification data component has update
Part is updated configurable interface file.
Optionally, the second acquisition unit includes:
First obtains module, for obtaining the data name of pending data；
Determining module, for determining institute after the title for the pending data that the first acquisition module obtains
It states in the first configurable interface file with the presence or absence of the data name；
Second obtains module, and user determines that there are the numbers in the first configurable interface file when the determining module
When according to title, the corresponding pending data of the data name is obtained from the data memory module.
Fourth aspect, the present invention also provides a kind of storage medium, which includes the program of storage, wherein in institute
State the place that equipment where controlling the storage medium when program operation executes data described in any one described in second aspect
Reason method.
5th aspect, the present invention also provides a kind of processor, the processor is for running program, wherein described program fortune
The processing method of data described in any one described in second aspect is executed when row.
By above-mentioned technical proposal, technical solution provided by the invention is at least had the advantage that
The processing method and processing device of data provided by the invention, after obtaining data processing request, by calling metadata
First configurable interface file of control centre obtains pending data corresponding with data name from data memory module,
By calling the second configurable interface file, the pending data corresponding with data name is obtained from data processing module
Corresponding processing method, by calling third configurable interface file, determination is corresponding with data name from data application module
Data application pattern, and show treated pending data, compared with prior art, the present invention is according in data warehouse
Data Layer specification adds corresponding data-interface to metadata control centre, is controlled by data-interface each in each data Layer
The extension and maintenance of component realize the data exchange of metadata control centre and data Layer, and then improve expanding for data warehouse
Malleability and ease for maintenance.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, the followings are specific embodiments of the present invention.
Detailed description of the invention
By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field
Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention
Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 shows the composition block diagram of the original data warehouse of one kind in the prior art；
Fig. 2 shows a kind of configuration diagrams of data warehouse provided in an embodiment of the present invention；
Fig. 3 shows a kind of flow chart of the processing method of data provided in an embodiment of the present invention；
Fig. 4 shows the flow chart of the processing method of second of data provided in an embodiment of the present invention；
Fig. 5 shows the embodiment of the present invention and provides a kind of composition block diagram of data warehouse；
Fig. 6 shows a kind of composition block diagram of the processing unit of data provided in an embodiment of the present invention；
Fig. 7 shows the processing unit composition block diagram of second of data provided in an embodiment of the present invention.
Specific embodiment
Exemplary embodiments of the present disclosure are described in more detail below with reference to accompanying drawings.Although showing the disclosure in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
It is fully disclosed to those skilled in the art.
In the prior art, in order to allow data warehouse to meet more user demands, support more data processings and
Query processing needs to add new functional module while the original function module in holding system, generallys use Stratified Strategy
Method either increase high-performance component method.As shown in Figure 1, the data warehouse packet original for one kind in the prior art
Data management layer and data Layer A, data Layer B, data Layer C are included, separately includes several components again in each data Layer.Work as user
When needing a new application function I, need to add new functional unit in the original data warehouse, Stratified Strategy
Method is, A layers of increase component a4 of data, in B layers of increase component b4 of data, in C layers of increase component c4 of data, it is each to realize
A data Layer needs the corresponding operating executed to the realization of the new function I, by the way that the component models of system are improved number
According to the scalability in warehouse；The method for increasing high-performance component is to add data D layers in systems, be will be carried out using function
Component d1, component d2, the component d3 of energy I are added in D layers of data, and D layers of data are placed in new hardware device and is run, and lead to
Expansible software parallel and hardware concurrent are crossed to improve the scalability of data warehouse.
But need to add more application functions when subsequent, for example, application function II, application function III and applying function
Can IV it is equal whens, if continuing the method using Stratified Strategy, data Layer A, data Layer B, data Layer C number of components can constantly increase
Add, the coupling inside each layer is continuously improved, and the maintainable of system is caused to reduce；If continuing using increase high-performance component
Method needs to establish the high platform of performance, the lower data Layer A of the performance of original data warehouse, data Layer B, data Layer C meeting
Inhibition is generated to entire data warehouse performance, in addition, being directed to multiple hardware development softwares, increases the exploitation of data warehouse
With the difficulty of maintenance.
To solve the above problems, the embodiment of the present invention is transformed by the structural system in traditional data warehouse, to solve
Traditional data warehouse poor expandability and maintainable the problem of reducing.As shown in Fig. 2, a kind of number provided in an embodiment of the present invention
According to the configuration diagram in warehouse, data warehouse described in the embodiment of the present invention, comprising:
Data management layer, metadata control centre is provided in the data management layer, and the metadata control centre is logical
It crosses multiple configurable interface files and multiple and different functional modules carries out data interaction；
Wherein, each configurable interface file and a functional module be there are mapping relations, it is the multiple can
Configuring interface document includes: the first configurable interface file, the second configurable interface file, third configurable interface file；Institute
Stating multiple and different functional modules includes: data memory module, data processing module, data application module；
The data memory module, for storing data；
The data processing module, for recording the corresponding processing method of the data；
The data application module, the Apply Styles for the data that record that treated.
It should be noted that being to include three modules (data memory modules, number in data warehouse in the embodiment of the present invention
According to processing module, data application module) for the explanation that carries out, still, it is understood that, this kind explanation is not intended to restriction
Metadata control centre can only interact with above three module, in practical applications, can increase by one or at least two
Functional module, meanwhile, increase third configurable interface file, the 4th configurable file ... established by configurable file it is newly-increased
Interactive relation between functional module and metadata control centre passes through data bins shown in Fig. 2 compared to existing technologies
Library structure carries out data storage, processing, and data warehouse is made to be more easily extensible, safeguard.
When carrying out data module extension, the extension of disparate modules can be carried out, data in same module can also be carried out
The extension of amount of storage.For extending disparate modules, while increasing a module, increasing in metadata control centre
Add a data-interface, the interface and newly-increased module carry out data interaction, and specific interaction content is recorded in new with data-interface
Increase corresponding configurable interface file (being assumed to be the 4th class configuration interface document).For extending same module, it is only necessary to
In configurable interface file to the data name for updating newly-increased data, there is modification for hardware store address, it can also be
Synchronized update in configurable interface file.It can thus be seen that data warehouse provided in an embodiment of the present invention is easy to extend, be easy to
Maintenance.
In above-described embodiment, the first configurable interface file, the second configurable interface file, third configurable interface are referred to
File, the 4th configurable interface file, described first, second, third ... exists just for the sake of distinguishing metadata control centre
Data type when interacting with data module, role is all the same in embodiments of the present invention for configurable interface file, that is, remembers
Record the data name for there are lower the included data of different function module, address data memory, data volume size, modification time etc..
The embodiment of the present invention provides a kind of processing method of data, and the method is applied to data warehouse as shown in Figure 2
In, as shown in figure 3, this method comprises:
101, data processing request is obtained, includes the data name of pending data in the data processing request.
The data processing request that data warehouse obtains, can be but not limited to asking for the inquiry data from front end
Ask, analyze the request of data or the request of more new data, received mode to can be real-time reception, as long as front end sends one
A data processing request immediately treats the data processing request；It is also possible to receive in batches, front end sends data processing and asks
It asks, timing or a quantitative batch receive the data processing request, the number of each received data processing request of batch
It measures unlimited.
For example, user can be drawn when carrying out data query, data processing, data update based on data warehouse by search
It holds up to data warehouse and sends data processing request, include the data name of pending data in the request of data, for example, user
When demand is the data calculation processing of 1+1, indicates that pending data is 1,1, execute add operation, above-mentioned example is merely to just
It is provided in understanding, to do more complicated data manipulation processing in practical application according to the actual demand of user.
102, the first configurable interface file is called, is obtained and the data name from the data memory module
Corresponding pending data.
Please continue to refer to Fig. 2, data warehouse described in the embodiment of the present invention, including it is data management layer, multiple configurable
Interface document and multiple and different functional modules；Be provided with metadata control centre in the data management layer, it is the multiple can
Configuring interface document includes: the first configurable interface file, the second configurable interface file, third configurable interface file；Institute
Stating multiple and different functional modules includes: data memory module, data processing module, data application module；The metadata control
Center processed carries out data interaction by multiple and different functional modules in multiple configurable interface files and the data warehouse,
There are mapping relations for a functional module in each configurable interface file and the data warehouse, each described to match
Set the data name that record in interface document there are corresponding the included data of functional module.
After receiving data processing request, by calling the first configurable interface file of metadata control centre, from
Pending data is obtained in data memory module.
The type of the metadata of the metadata control centre can be technology metadata, to develop and managing data warehouse
Data engineering teacher use, describe to data warehouse exploitation, manage and maintain relevant data, including data source information, number
According to conversion described, data warehouse model, data cleansing and update rule, data mapping and access authority etc.；It is also possible to business
Metadata is that management level and business analyst service, describes in data, including business terms, data warehouse from operational angle
There are what data, the position of data and availability of data etc., business personnel is helped to more fully understand which is counted in data warehouse
According to being available and how to use.In addition, the metadata control centre includes at least three configurable interface files, and
At least one configurable interface file that can be exchanged with data warehouse, at least one can be exchanged with data processing module
Configurable interface file, wherein the configurable interface file and the functional module unique association, the configurable interface
The content of file is including but not limited to the method name and parameter and return value inside packet name, the class name of interface existing for class, class
Deng, to the exchange of guide data, the mode of calling interface file can be it is implicit realize interface method, or explicit realize
Interface method.
103, the second configurable interface file is called, is obtained and the data name from the data processing module
The corresponding processing method of the corresponding pending data.
It can realize that interface method either explicitly realizes the second configurable interface described in calling interface method using implicit
File, the second interface file standardize data-processing interface operation, and how guide data interface is according to data processing
Various components performance in module obtains corresponding data, or exports corresponding data to first control centre.
The data processing method for including in the data processing module include but is not limited to following several one kind or
Two kinds and its more than:
(1) polymerization of data refers to that (polymerization based on multidimensional data is embodied in multidimensional to the simple aggregation based on particular demands
In data model), it can be total Pageviews, Visits, Unique Visitors of website etc. and summarize data, be also possible to
The average datas such as Avg.time on page, Avg.time on site, the mode of specific data aggregate are unlimited.
(2) Multidimensional Data Model is constructed, carries out the multi-level analysis application of multi-angle, specific processing method, which can be, to be based on
Sale Star Model, the snowflake model of the buildings such as time dimension, region dimension, are also possible to the friendship in each time dimension and region dimension
Fork inquiry can also be the subdivision based on time dimension and region dimension.
(3) business model is constructed, refers to the data model supported and set up based on certain data analysis and decisions, it can
Be user's evaluation model, correlation recommendation model, RFM analysis model etc. it is one such or two kinds and its more than, or certainly
Linear programming model that plan is supported, inventory model etc. it is one such or two kinds and its more than.
104, the pending data is handled according to the processing method.
Pending data is obtained to the step 102 using the data processing method described in step 103 and carries out data processing,
Specific data type includes but is not limited to: bit, int, smallint, tinyint, numeric, decimal, money,
smallmoney、float、datetime、smalldatetime、cursor、timestamp、uniqueidentifier、
Char, text, binary, image etc..
The pending data can be pre-processed before carrying out data processing, method includes but is not limited to: right
Data carry out validity check, carry out integrity checking to data.Specific processing mode can be one kind below and its two
Kind or more:
(1) to the cleaning of repeated data, it can be and directly delete the repeated data from table data store or storage chains,
Alternatively, remaining into the repeat number deleted in other tables of data or data-link while deleting repeated data, it is convenient for
Later to the lookup of repeated data；
It (2) may include: to be realized using ALTER TABLE field, or directly utilize UPDATE to the cleaning of hashed field
Sentence realizes that the sentence of the method or use specifically cleared up is without limitation；
(3) conversion of data may include but be not limited to: the conversion of data type, the conversion of object name, data encoding
Conversion, the conversion etc. of table structure it is one such or a variety of.
In addition, the specific order that the data processing module executes can be each data handling component while execute respectively
Corresponding processing processing task is also possible to execute corresponding task according to each component of execution step of processing task is primary.
105, the third configurable interface file is called, the determining and data name from the data application module
Corresponding data application pattern, and Apply Styles show treated pending data based on the data.
It can realize that interface method either explicitly realizes third configurable interface described in calling interface method using implicit
File, the third interface document standardize the operation of data application interface, and how guide data interface is according to data application
Various components performance in module obtains corresponding data.
The component of the data application pattern provided in the data module includes but is not limited to: query facility component, report
Tool assembly, OLAP component, Data Mining Tools component etc..The form of the displaying result specifically obtained can be but not limited to
Under any one or more:
(1) high performance MAPP includes but is not limited to: complete/incremental data load, heterogeneous data source support, distributed parallel
Calculate etc.；
(2) visual modeling includes but is not limited to: towed modeling, the table association across data warehouse, data dictionary, calculating
Field, model discrimination device, format conversion etc.；
(3) self-service analysis includes but is not limited to: intelligent answer, perspective analysis, chart linkage, dynamic reference line, What-
If analysis, personalized theme, measurement are estimated etc.；
(4) data are made a report on including but not limited to: data collection, examination & approval and publication, customizing form etc. under line.
The processing method of data provided in an embodiment of the present invention, after obtaining data processing request, by calling metadata
First configurable interface file of control centre obtains pending data corresponding with data name from data memory module,
By calling the second configurable interface file, the pending data corresponding with data name is obtained from data processing module
Corresponding processing method, by calling third configurable interface file, determination is corresponding with data name from data application module
Data application pattern, and show treated pending data, compared with prior art, the embodiment of the present invention is according to data bins
Data Layer specification in library adds corresponding data-interface to metadata control centre, each data is controlled by data-interface
The extension and maintenance of each component in layer realize the data exchange of metadata control centre and data Layer, and then improve data warehouse
Scalability and ease for maintenance.
The embodiment of the present invention also provides the processing method of another data, as shown in figure 4, this method comprises:
201, data management layer is added in the data warehouse, and metadata control is constructed in the data management layer
Center.
The data warehouse can be adapted for the computer application system of enterprise there is different hosts and operating system,
The computer environments such as heterogeneous network, isomeric data warehouse, core of the data management layer as whole system, contain first number
According to the functions such as acquisition, management, storage, control the various service interfaces of metadata.It completes and appoints during the entire process of system
The function of " brain " property such as reception, distribution and the management of business, may be implemented transaction management, i.e., provides multiple users correct, same
When access the ability of data warehouse, access control can also may be implemented, that is, limit uncommitted user to the access ability of data,
And ability of detection data validity etc. is not limited to other effects of above-mentioned effect, the data management layer can be by following
Three parts are constituted: data description language (Data Description Language, DDL), data manipulation language (Data
Manipulation Language, DML), data warehouse routine program composition, but be not limited to above-mentioned building form.
Metadata control centre can be constructed using the CWM standard of OIM standard and OMG including but not limited to MDC.It is described
The metadata in storage and maintenance metadata warehouse may be implemented in first control centre, also may be implemented data warehouse modeling tool,
Message transmission between data acquisition tools, front end tool etc. coordinates the work etc. between each module and tool.
202, according to data Layer specification, corresponding data-interface is added for the metadata control centre, and based on described
Data-interface establish the metadata control centre respectively with data memory module, data processing module, data application module it
Between communication connection.
After the metadata control centre is formed by all means, in order to allow the metadata control centre with it is described
Data memory module, the data processing module, the data application module establish connection, so that the metadata control centre
Addition, deletion, the update etc. that can carry out content to the content in the module are related in the operation and metadata of content changing
Hold retrieval, inquiry, place, the routinely metadata operation such as tissue, metadata control centre addition respectively with the number
According to memory module, the data processing module, the corresponding data-interface of the data application module.Wherein, it needs to be advised according to layer
Model is configured, and the layer specification describes the function of corresponding layer, and the function of the layer is greater than and each classification in the layer
The function of data package is added summation.
After the metadata control centre and the module establish a connection, the module can take following management mould
Any one in formula:
(1) System Management Mode is data-oriented warehouse, and by data warehouse management, systems professional is completed, data
User has a right to use, but the not operating rights of metadata, also there is the metadata of newly-generated data set in data application project
Application system passes to data warehouse administrator, is then managed collectively by data warehouse administrator；
(2) user management mode is application-oriented project, that is, allows certain data users in data application metadata
Transition information is fed directly to metadata warehouse, then can guarantee that the dynamic of metadata updates and newly-generated data set metadata in this way
It is timely capture and write-in meta data file；
(3) combination of System Management Mode and user management mode.
203, the data memory module, data processing module, the classification number for including in data application module are obtained respectively
According to component, the classification data component is for recording sorted data in different function module.
Wherein, the data memory module may include but be not limited to, source data component, integrated data component, history number
According to component and current data component etc. it is therein any one cross two kinds and its more than.The data processing module may include but
Be not limited to, algorithm assembly, tool assembly, modelling component and analytic unit etc. it is therein any one cross two kinds and its more than.Institute
Stating data application module may include but be not limited to, and show application component, inquiry application component and Report form application component etc. wherein
Any one cross two kinds and its more than.It is lower fixed that specific acquisition modes are not made, can be successively single acquisition, is also possible to point
Secondary acquisition is multiple.
In addition, can also be monitored in real time to the classification data component, really after obtaining the classification data component
The fixed classification data component is with the presence or absence of update.The form renewal of the classification data component specifically occurred may be but not
It is limited to: deletes existing classification data component, the new classification data component of addition or change existing classification data component.
The concrete mode for updating the data package, can be immediate updating, is also possible to update on time according to the preset time, may be used also
To be that the classification data component updated as needed is updated in batches.
It updates if classification data component exists, configurable interface file is carried out based on updated classification data component
It updates, configurable file is updated, it can be ensured that storing data, newest property, the standard of processing data in data warehouse
True property.The mode being specifically updated to configurable file is to can be that trigger may be joined in data source design,
Data are newly-increased, and modification is just made that effective update during perhaps deleting or joined some check lists, in the increasing of data
The details of operation for recording and having tracked data during revising can also be that self-propagation arranges, Time of Day stamp etc..
204, the data name in the classification data component is recorded in the first configurable interface file, second can
In configuration interface document, third configurable interface file, wherein the data memory module corresponds to first configurable interface
File, the data processing module correspond to the second configurable interface file, and the data application module corresponds to the third
Configurable interface file.
First before obtaining the pending data title, the first configurable interface file, described second be can configure
Will record in interface document, the third configurable interface file with the data memory module, the data processing module and
The corresponding data name of the data application functional module.The stage of the data of record can during design and modeling,
It can also can also be stored from data soft during data migration during data conversion, cleaning and filtering
Metadata is obtained in part and front-end presentation tool.Wherein, the data name is the unique identification of data.
After getting the pending data title, determine in the first configurable interface file with the presence or absence of described
Data name, determining mode can be the form using ergodic data table or data-link, and concrete mode is not required.Work as determination
There are after the data name in the first configurable interface file, then the data are obtained from the data memory module
The corresponding pending data of title.
The processing method of data provided in an embodiment of the present invention, after obtaining data processing request, by calling metadata
First configurable interface file of control centre obtains pending data corresponding with data name from data memory module,
By calling the second configurable interface file, the pending data corresponding with data name is obtained from data processing module
Corresponding processing method, by calling third configurable interface file, determination is corresponding with data name from data application module
Data application pattern, and show treated pending data, compared with prior art, the present invention is according in data warehouse
Data Layer specification adds corresponding data-interface to metadata control centre, is controlled by data-interface each in each data Layer
The extension and maintenance of component realize the data exchange of metadata control centre and data Layer, and then improve expanding for data warehouse
Malleability and ease for maintenance facilitate the data between each data function module to transmit and exchange.
Further, as the realization to method shown in above-mentioned Fig. 3 and Fig. 4, another embodiment of the present invention additionally provides one
The processing unit of kind data.The Installation practice is corresponding with preceding method embodiment, and to be easy to read, present apparatus embodiment is no longer
Detail content in preceding method embodiment is repeated one by one, it should be understood that the device in the present embodiment can correspond to
Realize the full content in preceding method embodiment.
Further, the embodiment of the present invention also provides a kind of data warehouse, as shown in Figure 5, comprising:
Data management layer 31 is provided with metadata control centre, the metadata control centre in the data management layer
Data interaction is carried out by multiple configurable interface files and multiple and different functional modules；
Wherein, each configurable interface file and a functional module be there are mapping relations, it is the multiple can
Configuring interface document includes: the first configurable interface file, the second configurable interface file, third configurable interface file；Institute
Stating multiple and different functional modules includes: data memory module, data processing module, data application module
The data memory module 33, for storing data；
The data processing module 34, for recording the corresponding processing method of the data；
The data application module 35, the Apply Styles for the data that record that treated.
The embodiment of the present invention provides a kind of processing unit of data, as shown in fig. 6, described device is applied to data warehouse
In, the data warehouse includes data management layer, multiple configurable interface files and multiple and different functional modules；The data
Metadata control centre is provided in management level, the multiple configurable interface file includes: the first configurable interface file,
Two configurable interface files, third configurable interface file；The plurality of different functional module includes: data memory module,
Data processing module, data application module；The metadata control centre passes through multiple configurable interface files and the data
Multiple and different functional modules in warehouse carry out data interactions, in each configurable interface file and the data warehouse
A functional module there are mapping relations, record has the corresponding functional module to be included in each configurable interface file
The data name of data；Include:
First acquisition unit 401 includes pending data in the data processing request for obtaining data processing request
Data name；
First call unit 402, for calling the first configurable interface file；
Second acquisition unit 403, for first call unit 402 call the first configurable interface file it
Afterwards, pending data corresponding with the data name is obtained from the data memory module；
Second call unit 404, for calling the second configurable interface file；
Third acquiring unit 405 is used for after second call unit calls the second configurable interface file,
The corresponding processing method of corresponding with the data name pending data is obtained from the data processing module；
Processing unit 406, the processing method for being obtained according to the third acquiring unit 405 is to described to be processed
Data are handled；
Third call unit 407, for calling the third configurable interface file；
First determination unit 408, for the third call unit 407 call the third configurable interface file it
Afterwards, data application pattern corresponding with the data name is determined from the data application module；
Display unit 409, for being based on the number after first determination unit 408 determines data application pattern
Treated pending data is shown according to Apply Styles.
Further, as shown in fig. 7, described device further include:
4th acquiring unit 410, for being obtained to from from the data memory module in the second acquisition unit 403
Before managing data, the data memory module, data processing module, the classification data for including in data application module are obtained respectively
Component, the classification data component is for recording sorted data in different function module；
Recording unit 411, the data name in the classification data component for obtaining the 4th acquiring unit 410
Title is recorded in the first configurable interface file, the second configurable interface file, in third configurable interface file, wherein
The data memory module corresponds to the first configurable interface file, and it is configurable that the data processing module corresponds to described second
Interface document, the data application module correspond to the third configurable interface file.
Further, as shown in fig. 7, described device further include:
Monitoring unit 412, for being monitored in real time to the classification data component；
Second determination unit 413, for determining institute after the monitoring unit is monitored the classification data component
Classification data component is stated with the presence or absence of update；
Updating unit 414, for being based on updated classification data when determining that the classification data component has update
Component is updated configurable interface file.
Further, as shown in fig. 7, the second acquisition unit 403 includes:
First obtains module 4031, for obtaining the data name of pending data；
Determining module 4032, for it is described first obtain module obtain the pending data title after, really
It whether there is the data name in the fixed first configurable interface file；
Second obtains module 4033, and user determines that there are institutes in the first configurable interface file when the determining module
When stating data name, the corresponding pending data of the data name is obtained from the data memory module.
The processing unit of the data includes processor and memory, above-mentioned first acquisition unit 401, the first call unit
402, second acquisition unit 403, the second call unit 404, third acquiring unit 405, processing unit 406, third call unit
407, the first determination unit 408 and display unit 409 etc. store in memory as program unit, are deposited by processor execution
Above procedure unit in memory is stored up to realize corresponding function.
Include kernel in processor, is gone in memory to transfer corresponding program unit by kernel.Kernel can be set one
Or more, it solves the problems, such as traditional data warehouse poor expandability by adjusting kernel parameter and maintainable reduces.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/
Or the forms such as Nonvolatile memory, if read-only memory (ROM) or flash memory (flash RAM), memory include that at least one is deposited
Store up chip.
The embodiment of the invention provides a kind of storage mediums, are stored thereon with program, real when which is executed by processor
The processing method of the existing data.
The embodiment of the invention provides a kind of processor, the processor is for running program, wherein described program operation
The processing method of data described in Shi Zhihang.
The embodiment of the invention provides a kind of equipment, equipment include processor, memory and storage on a memory and can
The program run on a processor, processor execute program when perform the steps of the data warehouse include data management layer,
Multiple configurable interface files and multiple and different functional modules；Metadata control centre is provided in the data management layer,
The multiple configurable interface file includes: the first configurable interface file, the second configurable interface file, third is configurable connects
Mouth file；The plurality of different functional module includes: data memory module, data processing module, data application module；It is described
Metadata control centre is carried out by multiple and different functional modules in multiple configurable interface files and the data warehouse
There are mapping relations for a functional module in data interaction, each configurable interface file and the data warehouse, often
Record has the data name of corresponding the included data of functional module in a configurable interface file；
Data processing request is obtained, includes the data name of pending data in the data processing request；
The first configurable interface file is called, is obtained from the data memory module corresponding with the data name
Pending data；
The second configurable interface file is called, is obtained from the data processing module corresponding with the data name
The corresponding processing method of the pending data；
The pending data is handled according to the processing method of acquisition；
The third configurable interface file is called, determination is corresponding with the data name from the data application module
Data application pattern, and Apply Styles show treated pending data based on the data.
Optionally, in the first configurable interface file by calling metadata control centre, from data memory module
Before obtaining pending data, the method also includes:
The data memory module, data processing module, the classification data group for including in data application module are obtained respectively
Part, the classification data component is for recording sorted data in different function module；
Data name in the classification data component is recorded in the first configurable interface file, second can configure
In interface document, third configurable interface file, wherein the data memory module corresponds to the first configurable interface text
Part, the data processing module correspond to the second configurable interface file, and the data application module corresponds to the third can
Configure interface document.
Optionally, the method also includes:
The classification data component is monitored in real time, determines the classification data component with the presence or absence of update；
If it exists, then configurable interface file is updated based on updated classification data component.
Optionally, the first configurable interface file based on metadata control centre, from data memory module obtain to
Handling data includes:
The data name of pending data is obtained, and determines and whether there is the number in the first configurable interface file
According to title；
If it is determined that in the presence of the corresponding pending data of the data name is then obtained from the data memory module.
Equipment herein can be server, PC, PAD, mobile phone etc..
Present invention also provides a kind of computer program products, when executing on data processing equipment, are adapted for carrying out just
The program code of beginningization there are as below methods step: obtaining data processing request, includes number to be processed in the data processing request
According to data name；
The first configurable interface file is called, is obtained from the data memory module corresponding with the data name
Pending data；
The second configurable interface file is called, is obtained from the data processing module corresponding with the data name
The corresponding processing method of the pending data；
The pending data is handled according to the processing method of acquisition；
The third configurable interface file is called, determination is corresponding with the data name from the data application module
Data application pattern, and Apply Styles show treated pending data based on the data.
It should be understood by those skilled in the art that, embodiments herein can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the application
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the application, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/
Or the forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable Jie
The example of matter.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices
Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
It include so that the process, method, commodity or the equipment that include a series of elements not only include those elements, but also to wrap
Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including element
There is also other identical elements in process, method, commodity or equipment.
It will be understood by those skilled in the art that embodiments herein can provide as method, system or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the application
Form.It is deposited moreover, the application can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The above is only embodiments herein, are not intended to limit this application.To those skilled in the art,
Various changes and changes are possible in this application.It is all within the spirit and principles of the present application made by any modification, equivalent replacement,
Improve etc., it should be included within the scope of the claims of this application.
Claims (11)
1. a kind of data warehouse characterized by comprising
Data management layer, metadata control centre is provided in the data management layer, and the metadata control centre passes through more
A configurable interface file and multiple and different functional modules carry out data interaction；
Wherein, there are mapping relations for each configurable interface file and a functional module, the multiple configurable
Interface document includes: the first configurable interface file, the second configurable interface file, third configurable interface file；It is described more
A different functional module includes: data memory module, data processing module, data application module；
The data memory module, for storing data；
The data processing module, for recording the corresponding processing method of the data；
The data application module, the Apply Styles for the data that record that treated.
2. a kind of processing method of data, the method is applied in data warehouse, which is characterized in that the data warehouse includes
Data management layer, multiple configurable interface files and multiple and different functional modules；First number is provided in the data management layer
According to control centre, the multiple configurable interface file include: the first configurable interface file, the second configurable interface file,
Third configurable interface file；The plurality of different functional module includes: data memory module, data processing module, data
Application module；The metadata control centre passes through multiple and different in multiple configurable interface files and the data warehouse
Functional module carries out data interaction, and a functional module in each configurable interface file and the data warehouse exists
Mapping relations, record has the data name of corresponding the included data of functional module in each configurable interface file；
Data processing request is obtained, includes the data name of pending data in the data processing request；
Call the first configurable interface file, from the data memory module obtain it is corresponding with the data name to
Handle data；
The second configurable interface file is called, institute corresponding with the data name is obtained from the data processing module
State the corresponding processing method of pending data；
The pending data is handled according to the processing method of acquisition；
The third configurable interface file is called, number corresponding with the data name is determined from the data application module
According to Apply Styles, and based on the data, Apply Styles show treated pending data.
3. according to the method described in claim 2, it is characterized in that, can configure by calling the first of metadata control centre
Interface document, before obtaining pending data in data memory module, the method also includes:
The data memory module, data processing module, the classification data component for including in data application module, institute are obtained respectively
Classification data component is stated for recording sorted data in different function module；
Data name in the classification data component is recorded in the first configurable interface file, the second configurable interface
In file, third configurable interface file, wherein the data memory module corresponds to the first configurable interface file, institute
It states data processing module and corresponds to the second configurable interface file, the data application module corresponds to that the third is configurable to be connect
Mouth file.
4. according to the method described in claim 3, it is characterized in that, the method also includes:
The classification data component is monitored in real time, determines the classification data component with the presence or absence of update；
If it exists, then configurable interface file is updated based on updated classification data component.
5. according to the method described in claim 2, it is characterized in that, the first configurable interface text based on metadata control centre
Part, obtaining pending data from data memory module includes:
The data name of pending data is obtained, and determines and whether there is the data name in the first configurable interface file
Claim；
If it is determined that in the presence of the corresponding pending data of the data name is then obtained from the data memory module.
6. a kind of processing unit of data, described device is applied in data warehouse, which is characterized in that the data warehouse includes
Data management layer, multiple configurable interface files and multiple and different functional modules；First number is provided in the data management layer
According to control centre, the multiple configurable interface file include: the first configurable interface file, the second configurable interface file,
Third configurable interface file；The plurality of different functional module includes: data memory module, data processing module, data
Application module；The metadata control centre passes through multiple and different in multiple configurable interface files and the data warehouse
Functional module carries out data interaction, and a functional module in each configurable interface file and the data warehouse exists
Mapping relations, record has the data name of corresponding the included data of functional module in each configurable interface file；Institute
Stating device includes:
First acquisition unit includes the data of pending data for obtaining data processing request, in the data processing request
Title；
First call unit, for calling the first configurable interface file；
Second acquisition unit, for after first call unit calls the first configurable interface file, from described
Pending data corresponding with the data name is obtained in data memory module；
Second call unit, for calling the second configurable interface file；
Third acquiring unit, for after second call unit calls the second configurable interface file, from described
The corresponding processing method of corresponding with the data name pending data is obtained in data processing module；
Processing unit, the processing method for being obtained according to the third acquiring unit to the pending data at
Reason；
Third call unit, for calling the third configurable interface file；
First determination unit, for after the third call unit calls the third configurable interface file, from described
Data application pattern corresponding with the data name is determined in data application module；
Display unit, for after first determination unit determines data application pattern, Apply Styles based on the data
Show treated pending data.
7. device according to claim 6, which is characterized in that described device further include:
4th acquiring unit, for the second acquisition unit obtained from the data memory module pending data it
Before, the data memory module, data processing module, the classification data component for including in data application module, institute are obtained respectively
Classification data component is stated for recording sorted data in different function module；
Recording unit, the data name in the classification data component for obtaining the 4th acquiring unit are recorded in institute
State the first configurable interface file, the second configurable interface file, in third configurable interface file, wherein the data are deposited
Storage module corresponds to the first configurable interface file, and the data processing module corresponds to the second configurable interface file,
The data application module corresponds to the third configurable interface file.
8. device according to claim 7, which is characterized in that described device further include:
Monitoring unit, for being monitored in real time to the classification data component；
Second determination unit, for determining the classification number after the monitoring unit is monitored the classification data component
According to component with the presence or absence of update；
Updating unit, for being based on updated classification data component pair when determining that the classification data component has update
Configurable interface file is updated.
9. device according to claim 6, which is characterized in that the second acquisition unit includes:
First obtains module, for obtaining the data name of pending data；
Determining module after the title for obtaining the pending data that module obtains described first, determines described the
It whether there is the data name in one configurable interface file；
Second obtains module, and user determines that there are the data names in the first configurable interface file when the determining module
When title, the corresponding pending data of the data name is obtained from the data memory module.
10. a kind of storage medium, which is characterized in that the storage medium includes the program of storage, wherein run in described program
When control the storage medium where equipment perform claim require 2 to the data described in any one of claim 5 processing
Method.
11. a kind of processor, which is characterized in that the processor is for running program, wherein right of execution when described program is run
Benefit require 2 to the data described in any one of claim 5 processing method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710764963.2A CN110019440B (en) | 2017-08-30 | 2017-08-30 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710764963.2A CN110019440B (en) | 2017-08-30 | 2017-08-30 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110019440A true CN110019440A (en) | 2019-07-16 |
| CN110019440B CN110019440B (en) | 2021-06-08 |
Family
ID=67186162
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710764963.2A Active CN110019440B (en) | 2017-08-30 | 2017-08-30 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110019440B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111143453A (en) * | 2019-12-24 | 2020-05-12 | 济宁昊瀛天成大数据信息技术有限公司 | Parallel processing method for industrial big data heterogeneous data |
| CN111858467A (en) * | 2020-07-22 | 2020-10-30 | 平安证券股份有限公司 | File data processing method, device, equipment and medium based on artificial intelligence |
| CN115167891A (en) * | 2022-07-07 | 2022-10-11 | 北京蓝天航空科技股份有限公司 | Data updating method, device and equipment of interface control file and storage medium |
| CN111858467B (en) * | 2020-07-22 | 2024-05-03 | 平安证券股份有限公司 | File data processing method, device, equipment and medium based on artificial intelligence |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101535965A (en) * | 2005-10-14 | 2009-09-16 | 塞门铁克操作公司 | Technique for improving scalability and portability of a storage management system |
| CN103902671A (en) * | 2014-03-19 | 2014-07-02 | 北京科技大学 | Dynamic integration method and system of multi-source heterogeneous data |
| CN104572456A (en) * | 2014-12-30 | 2015-04-29 | 北京锐安科技有限公司 | Data collection method and device |
| US20150178367A1 (en) * | 2013-12-20 | 2015-06-25 | Infosys Limited | System and method for implementing online analytical processing (olap) solution using mapreduce |
| CN105867932A (en) * | 2016-04-07 | 2016-08-17 | 浙江工业大学 | Metadata reflection chain based game cloud design method |
| CN105893063A (en) * | 2016-06-13 | 2016-08-24 | 积成电子股份有限公司 | Application program interface based on XML format configuration file and construction method of application program interface |
| CN106202361A (en) * | 2016-07-07 | 2016-12-07 | 上海携程商务有限公司 | Data query method and system |
| CN106462545A (en) * | 2014-03-31 | 2017-02-22 | 亚马逊科技公司 | Scalable file storage service |
| US20170116228A1 (en) * | 2015-10-23 | 2017-04-27 | Oracle International Corporation | System and method for automatic inference of a cube schema from a tabular data for use in a multidimensional database environment |
| CN106649734A (en) * | 2016-12-23 | 2017-05-10 | 中国科学院南京地理与湖泊研究所 | Water quality target management method and system with configurable model |
-
2017
- 2017-08-30 CN CN201710764963.2A patent/CN110019440B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101535965A (en) * | 2005-10-14 | 2009-09-16 | 塞门铁克操作公司 | Technique for improving scalability and portability of a storage management system |
| US20150178367A1 (en) * | 2013-12-20 | 2015-06-25 | Infosys Limited | System and method for implementing online analytical processing (olap) solution using mapreduce |
| CN103902671A (en) * | 2014-03-19 | 2014-07-02 | 北京科技大学 | Dynamic integration method and system of multi-source heterogeneous data |
| CN106462545A (en) * | 2014-03-31 | 2017-02-22 | 亚马逊科技公司 | Scalable file storage service |
| CN104572456A (en) * | 2014-12-30 | 2015-04-29 | 北京锐安科技有限公司 | Data collection method and device |
| US20170116228A1 (en) * | 2015-10-23 | 2017-04-27 | Oracle International Corporation | System and method for automatic inference of a cube schema from a tabular data for use in a multidimensional database environment |
| CN105867932A (en) * | 2016-04-07 | 2016-08-17 | 浙江工业大学 | Metadata reflection chain based game cloud design method |
| CN105893063A (en) * | 2016-06-13 | 2016-08-24 | 积成电子股份有限公司 | Application program interface based on XML format configuration file and construction method of application program interface |
| CN106202361A (en) * | 2016-07-07 | 2016-12-07 | 上海携程商务有限公司 | Data query method and system |
| CN106649734A (en) * | 2016-12-23 | 2017-05-10 | 中国科学院南京地理与湖泊研究所 | Water quality target management method and system with configurable model |
Non-Patent Citations (1)
| Title |
|---|
| 刘志宽: ""分级存储系统中元数据管理的设计与实现"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111143453A (en) * | 2019-12-24 | 2020-05-12 | 济宁昊瀛天成大数据信息技术有限公司 | Parallel processing method for industrial big data heterogeneous data |
| CN111858467A (en) * | 2020-07-22 | 2020-10-30 | 平安证券股份有限公司 | File data processing method, device, equipment and medium based on artificial intelligence |
| CN111858467B (en) * | 2020-07-22 | 2024-05-03 | 平安证券股份有限公司 | File data processing method, device, equipment and medium based on artificial intelligence |
| CN115167891A (en) * | 2022-07-07 | 2022-10-11 | 北京蓝天航空科技股份有限公司 | Data updating method, device and equipment of interface control file and storage medium |
| CN115167891B (en) * | 2022-07-07 | 2023-09-05 | 北京蓝天航空科技股份有限公司 | Data updating method, device and equipment of interface control file and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110019440B (en) | 2021-06-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7574379B2 (en) | Method and system of using artifacts to identify elements of a component business model | |
| US20190102447A1 (en) | System and method for metadata sandboxing and what-if analysis in a multidimensional database environment | |
| CN103299267B (en) | The method and system connecting for carrying out the interleaved of many tenant's storages | |
| CN109997126A (en) | Event-driven is extracted, transformation, loads (ETL) processing | |
| US11487745B2 (en) | Workflow dependency management system | |
| CN105339941B (en) | Projector and selector assembly type are used for ETL Mapping Design | |
| CN105074724A (en) | Efficient query processing using histograms in a columnar database | |
| KR101944973B1 (en) | Method and system for sharing user-defined Enterprise Resource Planning function and computing system performing the same | |
| CN109033113B (en) | Data warehouse and data mart management method and device | |
| CN105956087A (en) | Data and code version management system and method | |
| US9875278B2 (en) | Modified data query function instantiations | |
| Soldatos et al. | A reference architecture model for big data systems in the finance sector | |
| CN110019440A (en) | The processing method and processing device of data | |
| CN112084177B (en) | Data pool application method and device based on data acquisition treatment and mining analysis | |
| CN111708895A (en) | Method and device for constructing knowledge graph system | |
| US11615061B1 (en) | Evaluating workload for database migration recommendations | |
| NOVIKOV et al. | Improving the enterprise resource planning system based on digital modules of the industry 4.0 concept | |
| CN108959309B (en) | Method and device for data analysis | |
| US20140149186A1 (en) | Method and system of using artifacts to identify elements of a component business model | |
| Mathias et al. | A framework for monitoring multiple databases in industries using OPC UA | |
| Martinez-Gil et al. | General model for tracking manufacturing products using graph databases | |
| CN110781430B (en) | Novel virtual data center system of internet and construction method thereof | |
| US20160162814A1 (en) | Comparative peer analysis for business intelligence | |
| US10445299B2 (en) | Optimizing memory by sharing datasets between workspaces | |
| US20140143278A1 (en) | Application programming interface layers for analytical applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| CB02 | Change of applicant information |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: Beijing Guoshuang Technology Co.,Ltd. Address before: 100086 Beijing city Haidian District Shuangyushu Area No. 76 Zhichun Road cuigongfandian 8 layer A Applicant before: Beijing Guoshuang Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |