CN106933914A - The data processing method and device of many tables of data - Google Patents
The data processing method and device of many tables of data Download PDFInfo
- Publication number
- CN106933914A CN106933914A CN201511032517.XA CN201511032517A CN106933914A CN 106933914 A CN106933914 A CN 106933914A CN 201511032517 A CN201511032517 A CN 201511032517A CN 106933914 A CN106933914 A CN 106933914A
- Authority
- CN
- China
- Prior art keywords
- metadata
- metadata table
- query
- data
- query argument
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
- G06F16/24556—Aggregation; Duplicate elimination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses the data processing method and device of a kind of many tables of data.Wherein, the method includes：According to the query argument for getting, it is determined that at least two metadata tables corresponding with query argument, wherein, query argument at least includes：Querying condition and anatomy dimension；The data target in metadata table is inquired about respectively according to querying condition, Query Result corresponding with metadata table is obtained；Query Result is processed according to dimension is dissected, generates extended counter corresponding with metadata table；Extended counter is associated according to dimension is dissected, generates the extended counter table of comparisons.The present invention is solved due to carrying out multi dimensional analysis and/or index analysis to many tables of data, causes the technical problem that resource consumption is excessive.
Description
Technical field
The present invention relates to computer realm, in particular to the data processing method and device of a kind of many tables of data.
Background technology
Data warehouse is the structural data environment of a kind of DSS (dss) and on-line analysis application data source.
It is individual data storage, is created for analytical presentation and decision support purpose.
In data warehouse, it will usually there are multiple physical data tables for data storage content, and each entity number
Corresponding index can be all defined according to table.When a certain index is inquired about, it is only necessary to which a corresponding physical data table is entered
Row data manipulation.
And sometimes, we are when inquiring about a certain index, in addition it is also necessary to which associated index is inquired about,
And represented in one query, presented such that it is able to apparent being gathered information to index.Now, just need
To inquire about pair multiple physical data table corresponding with index simultaneously.
In addition, the multiple indexs in often also needing to multiple physical data tables carry out multi-dimensional free analysis, i.e.,
Any dimension can be filtered and checked according to specifying.
For above-mentioned problem, effective solution is not yet proposed at present.
The content of the invention
The data processing method and device of a kind of many tables of data are the embodiment of the invention provides, at least to solve due to many
Tables of data carries out multi dimensional analysis and/or index analysis, causes the technical problem that resource consumption is excessive.
A kind of one side according to embodiments of the present invention, there is provided data processing method of many tables of data, including：Root
According to the query argument for getting, it is determined that at least two metadata tables corresponding with query argument, wherein, query argument is extremely
Include less：Querying condition and anatomy dimension；The data target in metadata table is inquired about respectively according to querying condition,
Obtain Query Result corresponding with metadata table；Query Result is processed according to dimension is dissected, is generated and metadata
The corresponding extended counter of table；Extended counter is associated according to dimension is dissected, generates the extended counter table of comparisons.
Further, Query Result is processed according to anatomy dimension, generates extended counter corresponding with metadata table,
Including：Query Result corresponding with metadata table is obtained respectively；Polymerization analysis is carried out to Query Result according to dimension is dissected,
Obtain extended counter data word table corresponding with metadata table.
Further, extended counter is associated according to anatomy dimension, generates the extended counter table of comparisons, including：Obtain
Take extended counter data sublist corresponding with metadata table；Table company is carried out to extended counter data sublist according to dimension is dissected
Connect, generate the extended counter table of comparisons.
Further, according to the query argument for getting, it is determined that at least two metadata tables corresponding with query argument,
Including：According to the query argument for getting, the quantity of metadata table corresponding with query argument is judged；Work as query argument
When the quantity of corresponding metadata table is more than one, perform according to querying condition respectively to the data target in metadata table
Inquired about, obtained Query Result step corresponding with metadata table.
Further, according to the query argument for getting, the quantity of metadata table corresponding with query argument, bag are judged
Include：The metadata mapping table that acquisition pre-sets；According to query argument and metadata mapping table, it is determined that for store with
The metadata table of the corresponding aiming field of query argument.
Another aspect according to embodiments of the present invention, additionally provides a kind of data processing equipment of many tables of data, including：
Determining module, for according to the query argument that gets, it is determined that at least two metadata tables corresponding with query argument,
Wherein, query argument at least includes：Querying condition and anatomy dimension；Enquiry module, for distinguishing according to querying condition
Data target in metadata table is inquired about, Query Result corresponding with metadata table is obtained；First generation module,
For being processed Query Result according to anatomy dimension, extended counter corresponding with metadata table is generated；Second generation
Module, for being associated to extended counter according to anatomy dimension, generates the extended counter table of comparisons.
Further, above-mentioned first generation module, including：First sub-acquisition module, for obtaining respectively and metadata
The corresponding Query Result of table；Subprocessing module, for carrying out polymerization analysis to Query Result according to anatomy dimension, obtains
Extended counter data word table corresponding with metadata table.
Further, above-mentioned second generation module, including：Second sub-acquisition module, for obtaining and metadata table pair
The extended counter data sublist answered；Sublist link block, for being carried out to extended counter data sublist according to anatomy dimension
Table is connected, and generates the extended counter table of comparisons.
Further, above-mentioned determining module, including：Sub- judge module, for according to the query argument for getting, sentencing
The quantity of disconnected metadata table corresponding with query argument；Sub- performing module, for when the corresponding metadata table of query argument
Quantity more than one when, execution the data target in metadata table is inquired about respectively according to querying condition, obtain
Query Result step corresponding with metadata table.
Further, above-mentioned sub- judge module, including：3rd sub-acquisition module, for obtaining the first number for pre-setting
According to mapping table；Sub- determining module, for according to query argument and metadata mapping table, it is determined that for storing and inquiring about ginseng
The metadata table of the corresponding aiming field of number.
In embodiments of the present invention, using according to the query argument for getting, it is determined that corresponding with query argument at least two
Individual metadata table, wherein, query argument at least includes：Querying condition and anatomy dimension；It is right respectively according to querying condition
Data target in metadata table is inquired about, and obtains Query Result corresponding with metadata table；According to anatomy dimension pair
Query Result is processed, and generates extended counter corresponding with metadata table；Extended counter is carried out according to dimension is dissected
Association, generates the mode of the extended counter table of comparisons, is grasped simultaneously so as to reach to be reduced when dimension and/or index analysis are carried out
The purpose of the data volume of work, realize improve search efficiency, reduce resource consumption technique effect, and then solve due to
Multi dimensional analysis and/or index analysis are carried out to many tables of data, causes the technical problem that resource consumption is excessive.
Brief description of the drawings
Accompanying drawing described herein is used for providing a further understanding of the present invention, constitutes the part of the application, this hair
Bright schematic description and description does not constitute inappropriate limitation of the present invention for explaining the present invention.In accompanying drawing
In：
Fig. 1 is the flow chart of the data processing method of a kind of optional many tables of data according to embodiments of the present invention；And
Fig. 2 is the schematic diagram of the data processing equipment of a kind of optional many tables of data according to embodiments of the present invention.
Specific embodiment
In order that those skilled in the art more fully understand the present invention program, below in conjunction with the embodiment of the present invention
Accompanying drawing, is clearly and completely described to the technical scheme in the embodiment of the present invention, it is clear that described embodiment
The only embodiment of a present invention part, rather than whole embodiments.Based on the embodiment in the present invention, ability
The every other embodiment that domain those of ordinary skill is obtained under the premise of creative work is not made, should all belong to
The scope of protection of the invention.
It should be noted that term " first ", " in description and claims of this specification and above-mentioned accompanying drawing
Two " it is etc. for distinguishing similar object, without for describing specific order or precedence.It should be appreciated that this
The data that sample is used can be exchanged in the appropriate case, so as to embodiments of the invention described herein can with except
Here the order beyond those for illustrating or describing is implemented.Additionally, term " comprising " and " having " and they
Any deformation, it is intended that covering is non-exclusive to be included, for example, containing process, the side of series of steps or unit
Method, system, product or equipment are not necessarily limited to those steps clearly listed or unit, but may include unclear
List or for these processes, method, product or other intrinsic steps of equipment or unit.
According to embodiments of the present invention, there is provided a kind of embodiment of the method for the data processing method of many tables of data is, it is necessary to say
It is bright, can be in the such as one group computer system of computer executable instructions the step of the flow of accompanying drawing is illustrated
Middle execution, and, although logical order is shown in flow charts, but in some cases, can be being different from
Order herein performs shown or described step.
Fig. 1 is the flow chart of the data processing method of many tables of data according to embodiments of the present invention, as shown in figure 1, should
Method comprises the following steps：
Step S12, according to the query argument for getting, it is determined that at least two metadata tables corresponding with query argument,
Wherein, query argument at least includes：Querying condition and anatomy dimension.
Step S14, inquires about the data target in metadata table respectively according to querying condition, obtains and metadata
The corresponding Query Result of table.
Step S16, is processed Query Result according to dimension is dissected, and generates extended counter corresponding with metadata table.
Step S18, is associated according to dimension is dissected to extended counter, generates the extended counter table of comparisons.
Specifically, in above-mentioned steps S12 to step S18, according to the query demand for pre-setting, it is determined that inquiry ginseng
Number.Control query argument, determines involved metadata table and inquiry index in query argument.When by inquiring about ginseng
After several inquiry indexs in metadata table are inquired about, corresponding Query Result is obtained.Further, according to inquiry
The anatomy dimension included in parameter, is grouped according to dimension is dissected, and is obtained and each metadata by polymerization treatment
The corresponding extended counter of table, and according to treatment obtain extended counter generation with the one-to-one extended counter of metadata table
Subdatasheet.Finally, by each extended counter subdatasheet, table connection is carried out as condition to dissect dimension, so that
The extended counter table of comparisons for ultimately generating.
By above-mentioned steps, the purpose that the data volume of operation simultaneously is reduced when dimension and/or index analysis are carried out is reached,
Realize improving search efficiency, reduce the technique effect of resource consumption, and then solve in the prior art due to many data
Table carries out multi dimensional analysis and/or index analysis, causes the technical problem that resource consumption is excessive.
As a kind of optional implementation method, in the middle of practical application, with high-quality quantity on order in A, B, C website
As dissect dimension as a example by illustrate.Wherein, can be set to for determining the condition of high-quality order：Order
Transaction value is more than 1000 yuan of (ordervalue>1000), and payment status be received payment (orderstatus="
Payment ").
Step 1, it is determined that for storing A tables, B tables and the C tables of A, B, C website bill information as source data table,
Equally order transaction value and payment status are inquired about A tables, B tables and C tables respectively as querying condition.It is logical
Cross inquiry, obtain respectively with A tables, B tables and the corresponding Query Result of C tables.
Step 2, to count the quantity of high-quality order as dimension is dissected, to A tables, B tables and the corresponding inquiry of C tables
Result is processed, and is obtained and A tables, B tables and the corresponding extended counter of C tables.
First, Query Result is dissected by order transaction value and payment status, obtains corresponding Boolean type
Extension columns.Wherein, when order transaction value is met more than 1000 yuan and payment status are received payment, the boolean of extension columns
It is true to be worth, and when being unsatisfactory for, the Boolean of extension columns is false.Then, it is then pair corresponding with each metadata table
In extension columns, numerical value is counted for the quantity of true, obtains high-quality quantity on order corresponding with each website this expansion
Exhibition index.
Step 3, high-quality quantity on order this anatomy anatomy dimension is entered to A tables, B tables and the corresponding extended counter of C tables
Row association, generation includes the extended counter table of comparisons of A tables, B tables and C table high-quality quantity on orders.
As a kind of optional implementation method, in step s 16, the processing mode processed Query Result, extremely
Can include less：Polymerization treatment, mathematical operation etc..
As a kind of optional implementation method, Query Result is processed according to dimension is dissected in step S16, generated
In extended counter corresponding with metadata table, can include：
Step S161, obtains Query Result corresponding with metadata table respectively.
Step S163, polymerization analysis is carried out according to dimension is dissected to Query Result, obtains extension corresponding with metadata table
Achievement data sublist.
Specifically, in above-mentioned steps S161 to step S163, in the anatomy dimension that can be included from query argument,
It is determined that the anatomy dimension and processing mode of the Query Result obtained for inquiry.Then, according to anatomy dimension to inquiry
After result is grouped, then by be polymerized treatment processing mode the Query Result after packet is processed, generation with
The corresponding extended counter data sublist of each metadata table.
As a kind of optional implementation method, extended counter is associated according to dimension is dissected in step S18, generated
In the extended counter table of comparisons, can include：
Step S181, obtains extended counter data sublist corresponding with metadata table.
Step S183, table connection is carried out to extended counter data sublist according to dimension is dissected, and generates the extended counter table of comparisons.
Specifically, after being processed the Query Result of each metadata table with identical anatomy dimension and polymerization methodses,
Can be by step S181 to step S183, to dissect dimension as condition, will expansion corresponding with each metadata table
Exhibition achievement data sublist carries out table connection treatment, ultimately produces and includes all extended counter sublists that this inquiry is obtained
The extended counter table of comparisons.Wherein, the mode of table connection can include：Left outside connection, right outer connection, interior connection,
Complete outer connection etc., is not specifically limited herein.
Used as a kind of optional implementation method, the inquiry dimension required for for generating each index can in advance matching somebody with somebody
The form for putting file is configured.
In the middle of practical application, illustrated as a example by accessing figureofmerit and order figureofmerit.
Index allocation content for calculating visit capacity is as follows：
Index allocation content for calculating order volume is as follows：
In These parameters configuration content, Key is field name, and Name is data name, and Table is table name, SQL
It is treatment type.Specifically, the field to be checked included in above-mentioned visit capacity index allocation content is respectively
Visits fields in Session tables, and processing parameter is to do cumulative.Order volume index allocation content is not done superfluous herein
State.
As a kind of optional implementation method, in step S12 according to the query argument for getting, it is determined that and query argument
In corresponding at least two metadata table, can include：
Step S121, according to the query argument for getting, judges the quantity of metadata table corresponding with query argument.
Step S123, when the quantity of the corresponding metadata table of query argument is more than one, performs according to querying condition point
The other data target in metadata table is inquired about, and obtains Query Result step corresponding with metadata table.
Specifically, in the middle of database, often each tables of data can be defined the corresponding index of storage.Therefore, exist
In step S121 to step S123, first according to the querying condition recorded in query argument, it is determined that needing to which unit
Tables of data is inquired about, and determines to need the quantity in the metadata table of inquiry.When the quantity of metadata table is more than or waits
When two, step S14 treatment is performed.Wherein, querying condition is used to record the field of metadata storage
Name, metadata table name for storing metadata etc..
As a kind of optional implementation method, in step S121 according to the query argument for getting, judge and query argument
In the quantity of corresponding metadata table, can include：
Step S1211, the metadata mapping table that acquisition pre-sets.
Step S1213, according to query argument and metadata mapping table, it is determined that for storing mesh corresponding with query argument
The metadata table of marking-up section.
Specifically, in metadata mapping table, can in advance to for storing the storage field name of each index and depositing
The map information for storing up tables of data is configured, and then by step S1211 to step S1213, it is possible to use set in advance
The metadata mapping table put, and the field name that the index inquired about the need for record in query argument is stored, i.e.,
Can determine that the metadata table for storing the index.
Before being processed, can be in the form of configuration information in advance to extended counter and use with incidence relation
Configured in the tables of data of generation extended counter, and then can be expanded index list according to configuration information.
As a kind of optional implementation method, in the middle of practical application, specifically may include steps of：
Step a, according to incoming query demand, analyzes query argument corresponding with query demand, wherein, looking into
In asking parameter, record the querying condition inquired about metadata table according to query demand and further obtain to inquiring about
The anatomy dimension that is dissected of Query Result.Query argument is obtained after basis carries out Preliminary Analysis treatment to query demand
Arrive.
Step b, it is determined that metadata column corresponding with the anatomy dimension and the metadata table of metadata column storage；
Step c, according to the querying condition to metadata table recorded in query demand, enters to every metadata table respectively
Row inquiry, obtains data acquisition system corresponding with each metadata table；
Step d, dimension anatomy (dimension of group by) is carried out to the data acquisition system, obtains including extended counter
The extended counter sublist of set；
Step e, will dissect dimension (dimension of group by) as Correlation Criteria, to all of corresponding with metadata table
Extended counter sublist be associated, generate extended counter list.
Wherein, when being associated to each extended counter sublist, acquiescence be extended counter is carried out it is fully associative, that is, own
Data can be represented by extended counter list；Can certainly be using certain index as main index, other index conducts
Index of correlation.
Above-mentioned steps can be realized when actually used by code below：
WITH SessionMeasure as(SELECT server_session_id
FROM Session
WHERE source_type=' direct '
),
ECommerceFilter as(SELECT channel_name,server_session_id
FROM ECommerce
WHERE source_type=' direct ' AND order_status LIKE ' completed '
GROUP BY channel_name,server_session_id
),
ECommerceMeasure as(SELECT channel_name AS ChannelName,COUNT(*)AS OrderCount
FROM ECommerce
WHERE source_type=' direct ' AND order_status LIKE ' completed '
GROUP BY channel_name
),
SessionMeasure_ECommerceFilter AS(SELECT Count(*)AS Visits,channel_name AS
ChannelName FROM SessionMeasure INNER JOIN ECommerceFilter ON
SessionMeasure.server_session_id=ECommerceFilter.server_ session_id
Group By channel_name)
SELECT Visits,ISNULL(ECommerceMeasure.ChannelName,
SessionMeasure_ECommerceFilter.ChannelName)AS ChannelName,OrderCount,COUNT(*)
OVER()AS TotalRows
FROM SessionMeasure_ECommerceFilter FULL JOIN ECommerceMeasure ON
SessionMeasure_ECommerceFilter.ChannelName=ECommerceMeas ure.ChannelName
ORDER BY Visits Desc
In above-mentioned sentence, ECommerceMeasure, SessionMeasure_ECommerceFilter is and data
The corresponding extended counter of table.Then, then by SessionMeasure_ECommerceFilter.ChannelName=
ECommerceMeasure.ChannelName conditions are associated to each extended counter, obtain last result.
Particularly, in generation above-mentioned two extended counter corresponding with tables of data, filtered using identical dimension
Chain source_type=' direct ' AND order_status LIKE ' completed ' are to different data
Table is filtered.
By the application implementation method, realize and any dimension and any index are associated, and support across data
The index query function of table.Using identical filter condition, the index in each tables of data is processed, be calculated
Result corresponding with tables of data.Finally all result collection are associated by dissecting dimension again, to reach
All results are associated with the purpose of display.
Fig. 2 is the schematic diagram of the data processing equipment of many tables of data according to embodiments of the present invention.As shown in Fig. 2 should
Device includes：Determining module 22, enquiry module 24, the first generation module 26 and the second generation module 28.
Wherein it is determined that module 22, for according to the query argument for getting, it is determined that corresponding with query argument at least two
Individual metadata table, wherein, query argument at least includes：Querying condition and anatomy dimension；Enquiry module 24, for pressing
The data target in metadata table is inquired about respectively according to querying condition, Query Result corresponding with metadata table is obtained；
First generation module 26, for being processed Query Result according to anatomy dimension, generates expansion corresponding with metadata table
Exhibition index；Second generation module 28, for being associated to extended counter according to anatomy dimension, generates extended counter pair
According to table.
Specifically, in above-mentioned determining module 22, enquiry module 24, the first generation module 26 and the second generation module 28
In, according to the query demand for pre-setting, determine query argument.Control query argument, determines involved in query argument
And metadata table and inquiry index.After being inquired about the inquiry index in metadata table by query argument,
Obtain corresponding Query Result.Further, according to the anatomy dimension included in query argument, enter according to dimension is dissected
Row packet, and extended counter corresponding with each metadata table is obtained by polymerization treatment, and the expansion obtained according to treatment
The generation of exhibition index and the one-to-one extended counter subdatasheet of metadata table.Finally, by each extended counter subdata
Table, table connection is carried out to dissect dimension as condition, so as to the extended counter table of comparisons for ultimately generating.
By said apparatus, the purpose that the data volume of operation simultaneously is reduced when dimension and/or index analysis are carried out is reached,
Realize improving search efficiency, reduce the technique effect of resource consumption, and then solve in the prior art due to many data
Table carries out multi dimensional analysis and/or index analysis, causes the technical problem that resource consumption is excessive.
As a kind of optional implementation method, in above-mentioned first generation module 26, can include：First son obtains mould
Block 261 and subprocessing module 263.
Wherein, the first sub-acquisition module 261, for obtaining Query Result corresponding with metadata table respectively；Subprocessing
Module 263, for carrying out polymerization analysis to Query Result according to anatomy dimension, obtains extension corresponding with metadata table
Achievement data word table.
Specifically, in above-mentioned first sub-acquisition module 261 and subprocessing module 263, can be from query argument in
Comprising anatomy dimension in, it is determined that the anatomy dimension and processing mode of Query Result that are obtained for inquiry.Then,
After being grouped to Query Result according to anatomy dimension, then by the processing mode for processing that is polymerized to the inquiry knot after packet
Fruit is processed, and generates extended counter data sublist corresponding with each metadata table.
As a kind of optional implementation method, in above-mentioned second generation module 28, including：Second sub-acquisition module
281 and sublist link block 283.
Wherein, the second sub-acquisition module 281, for obtaining extended counter data sublist corresponding with metadata table；Son
Table link block 283, for carrying out table connection to extended counter data sublist according to anatomy dimension, generates extended counter
The table of comparisons.
Specifically, after being processed the Query Result of each metadata table with identical anatomy dimension and polymerization methodses,
Can be by the second sub-acquisition module 281 and sublist link block 283, to dissect dimension as condition, will be with each
The corresponding extended counter data sublist of metadata table carries out table connection treatment, ultimately produces and includes what this inquiry was obtained
The extended counter table of comparisons of all extended counter sublists.Wherein, the mode of table connection can include：Left outside connection, the right side
Outer connection, interior connection, complete outer connection etc., are not specifically limited herein.
As a kind of optional implementation method, in above-mentioned determining module 22, can include：Sub- judge module 221
With sub- performing module 223.
Wherein, sub- judge module 221, for according to the query argument for getting, judging unit corresponding with query argument
The quantity of tables of data；Sub- performing module 223, for when the quantity of the corresponding metadata table of query argument is more than one,
Execution is inquired about the data target in metadata table respectively according to querying condition, obtains look into corresponding with metadata table
Ask result step.
Specifically, in the middle of database, often each tables of data can be defined the corresponding index of storage.Therefore, exist
In above-mentioned sub- judge module 221 and sub- performing module 223, first according to the querying condition recorded in query argument, really
Surely need to inquire about which metadata table, and determine to need the quantity in the metadata table of inquiry.Work as metadata table
Quantity be more than or equal to two when, call above-mentioned enquiry module 24 to be processed.
As a kind of optional implementation method, above-mentioned sub- judge module 221, including：The He of 3rd sub-acquisition module 2211
Sub- determining module 2213.
Wherein, the 3rd sub-acquisition module 2211, for obtaining the metadata mapping table for pre-setting；Sub- determining module 2213,
For according to query argument and metadata mapping table, it is determined that the first number for storing aiming field corresponding with query argument
According to table.
Specifically, in metadata mapping table, can in advance to for storing the storage field name of each index and depositing
The map information for storing up tables of data is configured, and then by above-mentioned 3rd sub-acquisition module 2211 and sub- determining module 2213,
The metadata mapping table for pre-setting can be utilized, and the index storage inquired about the need for record in query argument
Field name, you can it is determined that the metadata table for storing the index.
The data processing equipment of many tables of data include processor and memory, above-mentioned acquiring unit, division unit,
Resolution unit, detection unit and judging unit etc. in memory, are deposited as program unit storage by computing device
Storage said procedure unit in memory.The rule that above-mentioned preset rules, metadata mapping table etc. pre-set all may be used
To store in memory.
Kernel is included in processor, is gone in memory to transfer corresponding program unit by kernel.Kernel can set one
Or more, parse content of text by adjusting kernel parameter.
Memory potentially includes the volatile memory in computer-readable medium, random access memory (RAM) and/
Or the form, such as read-only storage (ROM) or flash memory (flash RAM) such as Nonvolatile memory, memory includes at least one
Individual storage chip.
Present invention also provides a kind of embodiment of computer program product, when being performed on data processing equipment, fit
In the program code for performing initialization there are as below methods step：According to the query argument for getting, it is determined that and query argument
Corresponding at least two metadata table, wherein, query argument at least includes：Querying condition and anatomy dimension；According to looking into
Inquiry condition is inquired about the data target in metadata table respectively, obtains Query Result corresponding with metadata table；Press
Query Result is processed according to dimension is dissected, generates extended counter corresponding with metadata table；According to anatomy dimension pair
Extended counter is associated, and generates the extended counter table of comparisons.
The embodiments of the present invention are for illustration only, and the quality of embodiment is not represented.
In the above embodiment of the present invention, the description to each embodiment all emphasizes particularly on different fields, and does not have in certain embodiment
The part of detailed description, may refer to the associated description of other embodiment.
In several embodiments provided herein, it should be understood that disclosed technology contents, can be by other
Mode realize.Wherein, device embodiment described above is only schematical, such as division of described unit,
Can be a kind of division of logic function, there can be other dividing mode when actually realizing, for example multiple units or component
Can combine or be desirably integrated into another system, or some features can be ignored, or do not perform.It is another, institute
Display or the coupling each other for discussing or direct-coupling or communication connection can be by some interfaces, unit or mould
The INDIRECT COUPLING of block or communication connection, can be electrical or other forms.
The unit that is illustrated as separating component can be or may not be it is physically separate, it is aobvious as unit
The part for showing can be or may not be physical location, you can with positioned at a place, or can also be distributed to
On multiple units.Some or all of unit therein can be according to the actual needs selected to realize this embodiment scheme
Purpose.
In addition, during each functional unit in each embodiment of the invention can be integrated in a processing unit, it is also possible to
It is that unit is individually physically present, it is also possible to which two or more units are integrated in a unit.It is above-mentioned integrated
Unit can both be realized in the form of hardware, it would however also be possible to employ the form of SFU software functional unit is realized.
If the integrated unit is to realize in the form of SFU software functional unit and as independent production marketing or when using,
Can store in a computer read/write memory medium.Based on such understanding, technical scheme essence
On all or part of the part that is contributed to prior art in other words or the technical scheme can be with software product
Form is embodied, and the computer software product is stored in a storage medium, including some instructions are used to so that one
Platform computer equipment (can be personal computer, server or network equipment etc.) performs each embodiment institute of the invention
State all or part of step of method.And foregoing storage medium includes：USB flash disk, read-only storage (ROM, Read-Only
Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disc or CD
Etc. it is various can be with the medium of store program codes.
The above is only the preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, under the premise without departing from the principles of the invention, some improvements and modifications can also be made, these improve and moisten
Decorations also should be regarded as protection scope of the present invention.
Claims (10)
1. a kind of data processing method of many tables of data, it is characterised in that including：
According to the query argument for getting, it is determined that at least two metadata tables corresponding with the query argument, its
In, the query argument at least includes：Querying condition and anatomy dimension；
The data target in the metadata table is inquired about respectively according to the querying condition, obtain with it is described
The corresponding Query Result of metadata table；
The Query Result is processed according to the anatomy dimension, generates expansion corresponding with the metadata table
Exhibition index；
The extended counter is associated according to the anatomy dimension, generates the extended counter table of comparisons.
2. method according to claim 1, it is characterised in that enter to the Query Result according to the anatomy dimension
Row treatment, generates extended counter corresponding with the metadata table, including：
The Query Result corresponding with the metadata table is obtained respectively；
Polymerization analysis is carried out to the Query Result according to the anatomy dimension, obtains corresponding with the metadata table
The extended counter data sublist.
3. method according to claim 2, it is characterised in that enter to the extended counter according to the anatomy dimension
Row association, generates the extended counter table of comparisons, including：
Obtain the extended counter data sublist corresponding with the metadata table；
Table connection is carried out to the extended counter data sublist according to the anatomy dimension, the extended counter is generated
The table of comparisons.
4. method according to claim 1, it is characterised in that according to the query argument for getting, it is determined that with it is described
Corresponding at least two metadata table of query argument, including：
According to the query argument for getting, the number of the metadata table corresponding with the query argument is judged
Amount；
When the quantity of the corresponding metadata table of the query argument is more than one, perform according to the inquiry
Condition is inquired about the data target in the metadata table respectively, obtains look into corresponding with the metadata table
Ask result step.
5. method according to claim 4, it is characterised in that according to the query argument for getting, judge with
The quantity of the corresponding metadata table of the query argument, including：
The metadata mapping table that acquisition pre-sets；
According to the query argument and the metadata mapping table, it is determined that corresponding with the query argument for storing
Aiming field the metadata table.
6. a kind of data processing equipment of many tables of data, it is characterised in that including：
Determining module, for according to the query argument for getting, it is determined that corresponding with the query argument at least two
Individual metadata table, wherein, the query argument at least includes：Querying condition and anatomy dimension；
Enquiry module, for being looked into the data target in the metadata table respectively according to the querying condition
Ask, obtain Query Result corresponding with the metadata table；
First generation module, for being processed the Query Result according to the anatomy dimension, generates and institute
State the corresponding extended counter of metadata table；
Second generation module, for being associated to the extended counter according to the anatomy dimension, generation extension
The index table of comparisons.
7. device according to claim 6, it is characterised in that first generation module, including：
First sub-acquisition module, for obtaining the Query Result corresponding with the metadata table respectively；
Subprocessing module, for according to it is described anatomy dimension polymerization analysis is carried out to the Query Result, obtain with
The corresponding extended counter data word table of the metadata table.
8. device according to claim 7, it is characterised in that second generation module, including：
Second sub-acquisition module, for obtaining the extended counter data sublist corresponding with the metadata table；
Sublist link block, for carrying out table connection to the extended counter data sublist according to the anatomy dimension,
Generate the extended counter table of comparisons.
9. device according to claim 6, it is characterised in that the determining module, including：
Sub- judge module, for according to the query argument for getting, judging corresponding with the query argument
The quantity of the metadata table；
Sub- performing module, for when the quantity of the corresponding metadata table of the query argument is more than one,
Execution the data target in the metadata table is inquired about respectively according to the querying condition, obtain with it is described
The corresponding Query Result step of metadata table.
10. device according to claim 9, it is characterised in that the sub- judge module, including：
3rd sub-acquisition module, for obtaining the metadata mapping table for pre-setting；
Sub- determining module, for according to the query argument and the metadata mapping table, it is determined that for store with
The metadata table of the corresponding aiming field of the query argument.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511032517.XA CN106933914A (en) | 2015-12-31 | 2015-12-31 | The data processing method and device of many tables of data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511032517.XA CN106933914A (en) | 2015-12-31 | 2015-12-31 | The data processing method and device of many tables of data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106933914A true CN106933914A (en) | 2017-07-07 |
Family
ID=59443688
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201511032517.XA Pending CN106933914A (en) | 2015-12-31 | 2015-12-31 | The data processing method and device of many tables of data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106933914A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106933902A (en) * | 2015-12-31 | 2017-07-07 | 北京国双科技有限公司 | Querying method and device that data multidimensional degree is freely dissected |
| CN109254966A (en) * | 2018-08-23 | 2019-01-22 | 平安科技（深圳）有限公司 | Tables of data querying method, device, computer equipment and storage medium |
| CN109710681A (en) * | 2018-12-29 | 2019-05-03 | 亚信科技（南京）有限公司 | Data output method, device, computer equipment and storage medium |
| CN109800225A (en) * | 2018-12-24 | 2019-05-24 | 北京奇艺世纪科技有限公司 | Acquisition methods, device, server and the computer readable storage medium of operational indicator |
| CN110008211A (en) * | 2019-02-21 | 2019-07-12 | 北京奇艺世纪科技有限公司 | Data query method, apparatus, electronic equipment and storage medium |
| CN110334114A (en) * | 2018-02-12 | 2019-10-15 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus for inquiring data |
| CN110471935A (en) * | 2019-08-15 | 2019-11-19 | 上海达梦数据库有限公司 | A kind of execution method, apparatus, equipment and the storage medium of data manipulation |
| CN110471954A (en) * | 2019-07-29 | 2019-11-19 | 北京百分点信息科技有限公司 | A kind of data digging method and device |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101957849A (en) * | 2010-09-21 | 2011-01-26 | 用友软件股份有限公司 | Static statement generation method and device |
| CN102024062A (en) * | 2011-01-06 | 2011-04-20 | 中国工商银行股份有限公司 | Device and method for realizing data dynamic cache |
| CN102194164A (en) * | 2011-04-18 | 2011-09-21 | 上海实时数据软件有限公司 | Index-system-model-based agile business intelligence data construction method |
| CN102323942A (en) * | 2011-09-01 | 2012-01-18 | 北京中创信测科技股份有限公司 | Statistical query method |
| CN102999506A (en) * | 2011-09-13 | 2013-03-27 | 阿里巴巴集团控股有限公司 | Method and device for obtaining unique visitor (UV) |
| CN103605651A (en) * | 2013-08-28 | 2014-02-26 | 杭州顺网科技股份有限公司 | Data processing showing method based on on-line analytical processing (OLAP) multi-dimensional analysis |
| CN104112008A (en) * | 2014-07-16 | 2014-10-22 | 深圳市国泰安信息技术有限公司 | Multi-table data association inquiry optimizing method and device |
| CN104408180A (en) * | 2014-12-15 | 2015-03-11 | 北京国双科技有限公司 | Stored data inquiring method and device |
| CN104408179A (en) * | 2014-12-15 | 2015-03-11 | 北京国双科技有限公司 | Method and device for processing data from data table |
| CN104699788A (en) * | 2015-03-16 | 2015-06-10 | 百度在线网络技术（北京）有限公司 | Database query method and device |
| CN104899199A (en) * | 2014-03-04 | 2015-09-09 | 阿里巴巴集团控股有限公司 | Data processing method and system for data warehouse |
-
2015
- 2015-12-31 CN CN201511032517.XA patent/CN106933914A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101957849A (en) * | 2010-09-21 | 2011-01-26 | 用友软件股份有限公司 | Static statement generation method and device |
| CN102024062A (en) * | 2011-01-06 | 2011-04-20 | 中国工商银行股份有限公司 | Device and method for realizing data dynamic cache |
| CN102194164A (en) * | 2011-04-18 | 2011-09-21 | 上海实时数据软件有限公司 | Index-system-model-based agile business intelligence data construction method |
| CN102323942A (en) * | 2011-09-01 | 2012-01-18 | 北京中创信测科技股份有限公司 | Statistical query method |
| CN102999506A (en) * | 2011-09-13 | 2013-03-27 | 阿里巴巴集团控股有限公司 | Method and device for obtaining unique visitor (UV) |
| CN103605651A (en) * | 2013-08-28 | 2014-02-26 | 杭州顺网科技股份有限公司 | Data processing showing method based on on-line analytical processing (OLAP) multi-dimensional analysis |
| CN104899199A (en) * | 2014-03-04 | 2015-09-09 | 阿里巴巴集团控股有限公司 | Data processing method and system for data warehouse |
| CN104112008A (en) * | 2014-07-16 | 2014-10-22 | 深圳市国泰安信息技术有限公司 | Multi-table data association inquiry optimizing method and device |
| CN104408180A (en) * | 2014-12-15 | 2015-03-11 | 北京国双科技有限公司 | Stored data inquiring method and device |
| CN104408179A (en) * | 2014-12-15 | 2015-03-11 | 北京国双科技有限公司 | Method and device for processing data from data table |
| CN104699788A (en) * | 2015-03-16 | 2015-06-10 | 百度在线网络技术（北京）有限公司 | Database query method and device |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106933902B (en) * | 2015-12-31 | 2020-02-07 | 北京国双科技有限公司 | Data multidimensional free analysis query method and device |
| CN106933902A (en) * | 2015-12-31 | 2017-07-07 | 北京国双科技有限公司 | Querying method and device that data multidimensional degree is freely dissected |
| CN110334114B (en) * | 2018-02-12 | 2024-05-17 | 北京京东尚科信息技术有限公司 | Method and device for inquiring data |
| CN110334114A (en) * | 2018-02-12 | 2019-10-15 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus for inquiring data |
| CN109254966A (en) * | 2018-08-23 | 2019-01-22 | 平安科技（深圳）有限公司 | Tables of data querying method, device, computer equipment and storage medium |
| CN109800225A (en) * | 2018-12-24 | 2019-05-24 | 北京奇艺世纪科技有限公司 | Acquisition methods, device, server and the computer readable storage medium of operational indicator |
| CN109710681A (en) * | 2018-12-29 | 2019-05-03 | 亚信科技（南京）有限公司 | Data output method, device, computer equipment and storage medium |
| CN109710681B (en) * | 2018-12-29 | 2021-09-17 | 亚信科技（南京）有限公司 | Data output method and device, computer equipment and storage medium |
| CN110008211B (en) * | 2019-02-21 | 2021-07-06 | 北京奇艺世纪科技有限公司 | Data query method and device, electronic equipment and storage medium |
| CN110008211A (en) * | 2019-02-21 | 2019-07-12 | 北京奇艺世纪科技有限公司 | Data query method, apparatus, electronic equipment and storage medium |
| CN110471954A (en) * | 2019-07-29 | 2019-11-19 | 北京百分点信息科技有限公司 | A kind of data digging method and device |
| CN110471954B (en) * | 2019-07-29 | 2022-05-03 | 北京百分点科技集团股份有限公司 | Data mining method and device |
| CN110471935A (en) * | 2019-08-15 | 2019-11-19 | 上海达梦数据库有限公司 | A kind of execution method, apparatus, equipment and the storage medium of data manipulation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106933914A (en) | The data processing method and device of many tables of data | |
| CN110147483B (en) | Title reconstruction method and device | |
| CN106557498A (en) | Date storage method and device and data query method and apparatus | |
| EP2732416A2 (en) | System and method for advertisement placement | |
| CN107888716A (en) | A kind of sort method of domain name resolution server, terminal device and storage medium | |
| CN107180093A (en) | Information search method and device and ageing inquiry word recognition method and device | |
| CN102591876A (en) | Sequencing method and device of search results | |
| CN106933893A (en) | The querying method and device of multi-dimensional data | |
| CN110162566A (en) | Association analysis method, device, computer equipment and the storage medium of business datum | |
| CN107784070A (en) | A kind of method, apparatus and equipment for improving data cleansing efficiency | |
| CN107832444A (en) | Event based on search daily record finds method and device | |
| CN106649363A (en) | Data query method and device | |
| CN106933897A (en) | Data query method and apparatus | |
| CN107358456A (en) | Data show method and apparatus | |
| CN105653661A (en) | Search result re-ranking method and device | |
| CN106933935A (en) | task storage method and device | |
| CN110532282A (en) | Data query method and device | |
| CN106933916A (en) | The processing method and processing device of JSON character strings | |
| CN106933894A (en) | The querying method and device of multi-dimensional data | |
| CN110851737A (en) | Recommendation method and device, electronic equipment and computer storage medium | |
| CN106933918A (en) | The querying method and device of tables of data | |
| CN106933902A (en) | Querying method and device that data multidimensional degree is freely dissected | |
| CN106933904A (en) | The filter method and device of data | |
| CN106933920A (en) | The screening technique and device of session | |
| CN109165282A (en) | A kind of network data grasping means and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: Beijing Guoshuang Technology Co.,Ltd. Address before: 100086 Cuigong Hotel, 76 Zhichun Road, Shuangyushu District, Haidian District, Beijing Applicant before: Beijing Guoshuang Technology Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170707 |