CN106649368A - Data storage method and device and data query method and device - Google Patents
Data storage method and device and data query method and device Download PDFInfo
- Publication number
- CN106649368A CN106649368A CN201510729753.0A CN201510729753A CN106649368A CN 106649368 A CN106649368 A CN 106649368A CN 201510729753 A CN201510729753 A CN 201510729753A CN 106649368 A CN106649368 A CN 106649368A
- Authority
- CN
- China
- Prior art keywords
- data
- data storage
- achievement data
- achievement
- row
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data storage method and device and a data query method and device. The data storage method comprises the steps that an index definition of index data is added into a metadata table; a data storage location where data is not stored is retrieved in an index data storage table, wherein at the data storage location, columns in the index data storage table are empty and/or rows are empty; a mapping relation between the index definition and the retrieved data storage location is established to obtain an associated storage location; the index data is added into the associated storage location. Through the data storage method and device and the data query method and device, the technical problem of low query efficiency of the index data in the prior art is solved.
Description
Technical field
The application is related to database field, in particular to a kind of date storage method, device and data query side
Method, device.
Background technology
The data stored in relevant database data analysis system, it will usually have many indexs, be different clients
Different dimension indicators portfolio analysis are provided.In many cases, in addition it is also necessary to import other systems in data analysis system
The achievement data of statistics, in order to carry out comprehensive analysis.By taking monitoring of the advertisement as an example, generally have in data analysis system
The data monitoring indexs, also referred to as data target such as exposure, click, clicking rate.When need analysis cost with exposure or
Click on and the relation between order volume, then need to import such as order volume index, then carry out query statistic, in order to
It is associated analysis.
Because the index for importing is relevant with the demand of analysis, different customer demands are also differed, therefore, it is to be imported
Index is unpredictable.In order to reach analytical effect, data analysis system can introduce dynamic indicator in prior art
Storage and inquiry are supported.Also by taking the data analysiss of monitoring of the advertisement as an example, the data such as table 1 recorded in data analysis system
It is shown, when needing to import the index of " cost " and " order ", first add the definition of the index, as shown in table 2,
Regeneration dynamic indicator tables of data, such as table 3.
Table 1
| Dimension ID | Exposure | Click on | Clicking rate |
| 1 | 100 | 3 | 3.00% |
| … |
Table 2
| Index ID | Title |
| 1 | Cost |
| 2 | Order |
| … |
Table 3
| Dimension ID | Desired value | Index ID |
| 1 | 100 | 1 |
| 2 | 1 | 2 |
| … |
When achievement data is inquired about, its query statement is as follows：
SELECT tables of data. [ID]
, SUM ([Impress]) [exposure]
, SUM ([Click]) [click]
, SUM ([Ctr]) [clicking rate]
, SUM ([1]) cost
, SUM ([2]) order
FROM(SELECT*
FROM dynamic indicator tables of data
PIVOT (MAX ([Value]) FOR index ID IN ([1], [2])) a
)Pvt
INNER JOIN E ON tables of data. [ID]=Pvt. [ID]
GROUP BY tables of data. [ID]；
It follows that after index is imported, during the achievement data of the former index of inquiry and importing, first will need
Row storage index switchs to row storage, by index relation and former index conjunctive query.Although the program overcomes dynamic and refers to
The inquiry problem of implementation of mark addition, can be dynamically completed data analysiss work in the case where predicting index quantity,
But, but accumulation over time, the rising of data volume, " first row storage index is converted into into row storage " in such scheme
This query steps will become slow, and search efficiency is gradually reduced.
For above-mentioned problem, effective solution is not yet proposed at present.
The content of the invention
The embodiment of the present application provides a kind of date storage method, device and data query method, device, at least to solve
The certainly low technical problem of index efficiency data query in prior art.
According to the one side of the embodiment of the present application, there is provided a kind of date storage method, including：In metadata table,
The index definition of addition achievement data；In achievement data storage table, the data storage location of non-data storage is retrieved,
Wherein, the data storage location is classified as sky and/or behavior sky comprising in the achievement data storage table；Set up described
The mapping relations of index definition and the data storage location for retrieving, to obtain associated storage position；By the index number
According to storage in the associated storage position.
Further, set up the index definition includes with the mapping relations of the data storage location for retrieving：Obtain inspection
The mark of the data storage location that rope is arrived；The corresponding mapping relations row of the index definition are inquired about from the metadata table
Data form；The mark of the data storage location for retrieving is stored in the data form to set up the data
The mapping relations of storage location and the index definition, using the data storage location as the associated storage position.
Further, before the empty row that inquiry pre-sets in being set to arrange the achievement data storage table of storage, institute
Stating method also includes：The empty row of N row or N row nulls are set in the achievement data storage table, the sky is arranged into institute
Null is stated as the data storage location, wherein, 0 ＜ N≤Max, the Max are the data base for storing achievement data
The maximum supported；The achievement data storage table is set in the database for row storage.
Further, methods described also includes：When receiving for the deletion instruction for deleting achievement data, institute is deleted
State the achievement data deleted described in metadata table in the indicated index definition of instruction and the achievement data storage table
Mapping relations；It is superfluous that the metadata token for deleting the indicated index definition of instruction will be recorded in the metadata table
It is remaining.
According to the another aspect of the embodiment of the present application, a kind of data query method is additionally provided, including：Receive for looking into
Ask the query statement of achievement data；The index definition indicated by the query statement is determined from metadata table, institute is determined
State the mapping relations of the achievement data in the index definition and achievement data storage table indicated by query statement；Using described
Mapping relations inquire about the achievement data indicated by the query statement from the achievement data storage table.
According to the another aspect of the embodiment of the present application, a kind of data storage device is additionally provided, including：Adding device,
For in metadata table, adding the index definition of achievement data；Retrieval unit, in achievement data storage table,
The data storage location of the non-data storage of retrieval, wherein, the data storage location includes the achievement data storage table
In to be classified as sky and/or behavior empty；Unit is set up, for setting up reflecting for the index definition and the data storage location
Relation is penetrated, to obtain associated storage position；And import unit, for achievement data storage to be arrived into the association
In storage location.
Further, the unit of setting up includes：Acquisition module, for obtaining the data storage location for retrieving
Mark；Enquiry module, for inquiring about the number of the corresponding mapping relations row of the index definition from the metadata table
According to form；Writing module, for the mark of the data storage location for retrieving being stored in the data form to build
The mapping relations of the data storage location and the index definition are found, using the data storage location as the association
Storage location.
Further, described device also includes：First setting unit, in the data storage for retrieving non-data storage
Before position, the empty row of N row or N row nulls are set in the achievement data storage table, by the empty row or institute
Null is stated as the data storage location, wherein, 0 ＜ N≤Max, the Max are the data base for storing achievement data
The maximum supported；Second setting unit, for arranging the achievement data storage table in the database for row
Storage.
Further, described device also includes：Unit is deleted, for receiving the deletion for deleting achievement data
During instruction, delete and delete described in the metadata table the indicated index definition of instruction and the achievement data storage table
In achievement data mapping relations；Indexing unit, for will record described deletion in the metadata table indication is instructed
The metadata token of the index definition for showing is redundancy.
According to the another aspect of the embodiment of the present application, a kind of data query arrangement is additionally provided, including：Receiving unit,
For receiving the query statement for inquiring about achievement data；Determining unit, for determining the inquiry from metadata table
The indicated index definition of instruction, in determining the index definition and achievement data storage table indicated by the query statement
The mapping relations of achievement data；Query unit, for being looked into from the achievement data storage table using the mapping relations
Ask the achievement data indicated by the query statement.
According to the embodiment of the present application, in metadata table, add the index definition of achievement data；In achievement data storage
In table, the data storage location of non-data storage is retrieved, wherein, data storage location is comprising in achievement data storage table
To be classified as sky and/or behavior empty；The mapping relations of index definition and the data storage location for retrieving are set up, to be closed
Connection storage location；Achievement data is stored in associated storage position, row storage index is reduced and is converted into row storage
IO performance costs, improve search efficiency, solve the low technical problem of index efficiency data query in prior art.
Description of the drawings
Accompanying drawing described herein is used for providing further understanding of the present application, constitutes the part of the application, this Shen
Schematic description and description please does not constitute the improper restriction to the application for explaining the application.In accompanying drawing
In：
Fig. 1 is the flow chart of the date storage method according to the embodiment of the present application；
Fig. 2 is the flow chart of the data query method according to the embodiment of the present application；
Fig. 3 is the schematic diagram of the data storage device according to the embodiment of the present application.
Fig. 4 is the schematic diagram of the data query arrangement according to the embodiment of the present application.
Specific embodiment
In order that those skilled in the art more fully understand application scheme, below in conjunction with the embodiment of the present application
Accompanying drawing, is clearly and completely described to the technical scheme in the embodiment of the present application, it is clear that described embodiment
The only embodiment of the application part, rather than the embodiment of whole.Based on the embodiment in the application, ability
The every other embodiment that domain those of ordinary skill is obtained under the premise of creative work is not made, all should belong to
The scope of the application protection.
It should be noted that the description and claims of this application and the term " first " in above-mentioned accompanying drawing, "
Two " it is etc. the object for distinguishing similar, without for describing specific order or precedence.It should be appreciated that this
The data that sample is used can be exchanged in the appropriate case, so as to embodiments herein described herein can with except
Here the order beyond those for illustrating or describing is implemented.Additionally, term " comprising " and " having " and they
Any deformation, it is intended that covering is non-exclusive to be included, and for example, contains process, the side of series of steps or unit
Method, system, product or equipment are not necessarily limited to those steps clearly listed or unit, but may include unclear
List or other steps intrinsic for these processes, method, product or equipment or unit.
According to the embodiment of the present application, there is provided a kind of embodiment of the method for date storage method, it should be noted that
The step of flow process of accompanying drawing is illustrated can perform in the such as computer system of one group of computer executable instructions, and
And, although show logical order in flow charts, but in some cases, can be with different from order herein
Perform shown or described step.
Fig. 1 is the flow chart of the date storage method according to the embodiment of the present application, as shown in figure 1, the method include as
Lower step：
Step S102, in metadata table, adds the index definition of achievement data.
Wherein, metadata table for record index metadata tables of data, can record on the metadata table index mark,
The metadata such as title, data type.In the present embodiment, when needing to import index in data analysis system, first exist
Add the index definition of the achievement data in metadata table, can include adding mark, title, the number of the achievement data
According to type etc. for the data inquiring about and store (for example：Data type is for data memory format verification).
Step S104, in achievement data storage table, retrieves the data storage location of non-data storage, wherein, data
Storage location is classified as sky and/or behavior sky comprising in achievement data storage table.
Step S106, sets up the mapping relations of index definition and the data storage location for retrieving, to obtain associated storage
Position.
Achievement data storage table is the tables of data of the statistical data that record has indices.In the embodiment of the present application, index
Table data store could be arranged to row storage, it is also possible to be set to go storage, i.e. achievement data by the storage of data row or
Row storage, each row represent statistical data of the index in different dimensions, and set in advance in achievement data storage table
Put the data storage location for not storing achievement data, such as null or empty row, in order to after achievement data is imported,
During achievement data to be imported to the data storage location for not storing achievement data for pre-setting.
Step S108, achievement data is stored in associated storage position.
After set up mapping relations of the index definition with the data storage location for retrieving, i.e. agriculture products data are led
Enter position, as associated storage position, therefore, it can import to the achievement data of acquisition and above-mentioned set up mapping relations
In the associated storage position for being mapped.Do not store because the embodiment of the present application pre-sets in achievement data storage table
The data storage location of achievement data is used to import achievement data so that the achievement data and data analysis system after importing
In original achievement data in same achievement data storage table, carry out achievement data inquire about when, without the need for will row storage
Index is converted into row storage, reduces the IO performance costs that row storage index is converted into row storage, improves search efficiency.
According to the embodiment of the present application, in metadata table, add the index definition of achievement data；In achievement data storage
In table, the data storage location of non-data storage is retrieved, wherein, data storage location is comprising in achievement data storage table
To be classified as sky and/or behavior empty；The mapping relations of index definition and the data storage location for retrieving are set up, to be closed
Connection storage location；Achievement data is stored in associated storage position, when carrying out achievement data and inquiring about, without the need for by row
Storage index is converted into row storage, reduces the IO performance costs that row storage index is converted into row storage, improves inquiry
Efficiency, solves the low technical problem of index efficiency data query in prior art.
Preferably, before the empty row that inquiry pre-sets in being set to arrange the achievement data storage table of storage, method
Also include：The empty row of N row or N row nulls are set in achievement data storage table, using empty row or null as data
Storage location, wherein, 0 ＜ N≤Max, Max are to store the maximum that the data place of achievement data is supported；In data
Setting target table data store is row storage in storehouse.
In the embodiment of the present application, the empty row of N row are set in the achievement data storage table prestored in data analysis system
Or N row nulls, wherein, N can from 0 to Max in arbitrarily select.Each empty row or null can add one
Individual corresponding mark, for representing corresponding empty row or null.If the achievement data storage table is set to into row
Storage, then it is each to arrange the statistical data for representing an index in different dimensions, as shown in table 4；If the achievement data
Storage table is set to go storage, then represent statistical data of the index in different dimensions per a line.
Table 4
| Dimension ID | Exposure | Click on | Clicking rate | Preset 1 | … | Preset N |
Preferably, set up index definition includes with the mapping relations of the data storage location for retrieving：What acquisition was retrieved
The mark of data storage location；The data form of the corresponding mapping relations row of index definition is inquired about from metadata table；Will
The mark storage of the data storage location for retrieving is in data form setting up reflecting for data storage location and index definition
Relation is penetrated, using data storage location as associated storage position.
In the embodiment of the present application, each data storage location is provided with corresponding in preset achievement data storage table
Mark, it is as shown in table 4 " preset 1 " ... " preset N ", during achievement data is imported, if it is determined that
The empty row of achievement data are imported, then obtains its mark, then the mark be written to the finger of the importing in metadata table
In marking the data form of corresponding mapping relations row, to set up the mapping relations between empty row and the index of importing.With sky
Row is identical with the principle of empty row as data storage location, does not repeat here.
Table 5
| Index ID | State | Title | Mapping relations | Data type |
| 1 | Normally | Exposure | Exposure | Integer |
| 2 | Normally | Click on | Click on | Integer |
| 3 | Normally | Clicking rate | Clicking rate | Percentage ratio |
| 4 | Normally | Order | Preset 1 | Integer |
| 5 | Normally | Cost | Preset 2 | Currency |
| 6 | It is idle | |||
| … |
As shown in table 4, table 5, wherein, the index of entitled " order " and " cost " is the index for importing, its
In, the mark of the corresponding empty row of These parameters is followed successively by " preset 1 " and " preset 2 ", when will refer in mapping relations row
Mark data are imported to after table 4, then there are mapping relations between the These parameters and its achievement data in table 5.
Preferably, method also includes：When receiving for the deletion instruction for deleting achievement data, metadata table is deleted
The middle mapping relations for deleting the indicated index definition of instruction and the achievement data in achievement data storage table；By metadata
The metadata token of the indicated index definition of record deletion instruction is redundancy in table.
In the embodiment of the present application, when needing to delete certain achievement data, will first the index definition indicated by instruction be deleted
Delete with the mapping relations of achievement data, and be redundancy by the metadata token of the index definition recorded in metadata table,
So, data analysis system will be deleted at one's leisure.
It should be noted that in the embodiment of the present application, the data in each example are only with monitoring of the advertisement data instance, ability
Field technique personnel could be aware that the technical scheme of the embodiment of the present application can be used after the description of the present application is read
In the data storage and query of various multi objectives, the embodiment of the present application does not have improper restriction.
According to the embodiment of the present application, a kind of embodiment of the method for data query method is additionally provided, it should be noted that
Can perform in the such as computer system of one group of computer executable instructions the step of the flow process of accompanying drawing is illustrated,
And, although show logical order in flow charts, but in some cases, can be with suitable different from herein
Sequence performs shown or described step.The data query method of the embodiment of the present application can be based on the embodiment of the present application
The achievement data that date storage method stored and imported inquired about, that is to say the data query of the embodiment of the present application
Method can be used for the achievement data inquired about the date storage method storage of the embodiment of the present application and import.
As shown in Fig. 2 the data query method of the embodiment of the present application includes：
Step S202, receives the query statement for inquiring about achievement data.
Step S204, determines the index definition indicated by query statement from metadata table, determines indicated by query statement
Index definition and achievement data storage table in achievement data mapping relations.
Step S206, using mapping relations the achievement data indicated by query statement is inquired about from achievement data storage table.
In the date storage method using the embodiment of the present application achievement data is imported (referring specifically to the number of the embodiment of the present application
According to the description of storage method) after, the achievement data of importing is stored in achievement data storage with original achievement data
In table.
After the query statement for receiving inquiry achievement data, with reference to metadata table and achievement data storage table, from finger
Corresponding achievement data is directly inquired about in mark table data store.Due to the embodiment of the present application it is pre- in achievement data storage table
Data storage location is first provided with for importing achievement data so that in the achievement data and data analysis system after importing
Original achievement data in same achievement data storage table, carry out achievement data inquire about when, without the need for row storage is referred to
Mark is converted into row storage, reduces the IO performance costs that row storage index is converted into row storage, improves search efficiency.
For example, by taking above-mentioned table 4, table 5 as an example, when achievement data inquiry is carried out, query statement is as follows：
SELECT[ID]
, SUM ([exposure]) [exposure]
, SUM ([click]) [click]
, SUM ([clicking rate]) [clicking rate]
, SUM (preset 1) costs
, SUM (preset 2) orders
FROM tables of data (preset)
GROUP BY[ID]
It follows that in the achievement data query script of the embodiment of the present application, it is only necessary to inquire about index in metadata table
The row at place, directly carry out Querying by group, and performance has obtained significant increase.
The embodiment of the present application additionally provides a kind of data storage device, and the device can be used for performing the embodiment of the present application
Date storage method, as shown in figure 3, the device includes：Adding device 301, retrieval unit 303, set up unit
305 and import unit 307.
Adding device 301 is used in metadata table, adds the index definition of achievement data.
Wherein, metadata table for record index metadata tables of data, can record on the metadata table index mark,
The metadata such as title, data type.In the present embodiment, when needing to import index in data analysis system, first exist
Add the index definition of the achievement data in metadata table, can include adding mark, title, the number of the achievement data
According to type etc. for the data inquiring about and store (for example：Data type is for data memory format verification).
Retrieval unit 303 is used in achievement data storage table, retrieves the data storage location of non-data storage, wherein,
Data storage location is classified as sky and/or behavior sky comprising in achievement data storage table.
Unit 305 is set up for setting up the mapping relations of index definition and data storage location, to obtain associated storage position
Put.
Achievement data storage table is the tables of data of the statistical data that record has indices.In the embodiment of the present application, index
Table data store could be arranged to row storage, it is also possible to be set to go storage, i.e. achievement data by the storage of data row or
Row storage, each row represent statistical data of the index in different dimensions, and set in advance in achievement data storage table
Put the data storage location for not storing achievement data, such as null or empty row, in order to after achievement data is imported,
During achievement data to be imported to the data storage location for not storing achievement data for pre-setting.
Import unit 307 is used to achievement data is stored in associated storage position.
After set up mapping relations of the index definition with the data storage location for retrieving, i.e. agriculture products data are led
Enter position, as associated storage position, therefore, it can import to the achievement data of acquisition and above-mentioned set up mapping relations
In the associated storage position for being mapped.Do not store because the embodiment of the present application pre-sets in achievement data storage table
The data storage location of achievement data is used to import achievement data so that the achievement data and data analysis system after importing
In original achievement data in same achievement data storage table, carry out achievement data inquire about when, without the need for will row storage
Index is converted into row storage, reduces the IO performance costs that row storage index is converted into row storage, improves search efficiency.
According to the embodiment of the present application, in metadata table, add the index definition of achievement data；In achievement data storage
In table, the data storage location of non-data storage is retrieved, wherein, data storage location is comprising in achievement data storage table
To be classified as sky and/or behavior empty；The mapping relations of index definition and the data storage location for retrieving are set up, to be closed
Connection storage location；Achievement data is stored in associated storage position, when carrying out achievement data and inquiring about, without the need for by row
Storage index is converted into row storage, reduces the IO performance costs that row storage index is converted into row storage, improves inquiry
Efficiency, solves the low technical problem of index efficiency data query in prior art.
Preferably, device also includes：First setting unit, for retrieve non-data storage data storage location it
Before, the empty row of N row or N row nulls are set in achievement data storage table, using empty row or null as data storage
Position, wherein, 0 ＜ N≤Max, Max are to store the maximum that the data place of achievement data is supported；Second arranges single
Unit, is row storage for the setting target table data store in data base.
In the embodiment of the present application, the empty row of N row are set in the achievement data storage table prestored in data analysis system
Or N row nulls, wherein, N can from 0 to Max in arbitrarily select.Each empty row or null can add one
Individual corresponding mark, for representing corresponding empty row or null.If the achievement data storage table is set to into row
Storage, then it is each to arrange the statistical data for representing an index in different dimensions, as shown in table 4；If the achievement data
Storage table is set to go storage, then represent statistical data of the index in different dimensions per a line.
Preferably, setting up unit includes：Acquisition module, for obtaining the mark of the data storage location for retrieving；Look into
Module is ask, for inquiring about the data form of the corresponding mapping relations row of index definition from metadata table；Writing module,
For the storage of the mark of the data storage location for retrieving to be determined with setting up data storage location in data form with index
The mapping relations of justice, using data storage location as associated storage position.
In the embodiment of the present application, each data storage location is provided with corresponding in preset achievement data storage table
Mark, it is as shown in table 4 " preset 1 " ... " preset N ", during achievement data is imported, if it is determined that
The empty row of achievement data are imported, then obtains its mark, then the mark be written to the finger of the importing in metadata table
In marking the data form of corresponding mapping relations row, to set up the mapping relations between empty row and the index of importing.With sky
Row is identical with the principle of empty row as data storage location, does not repeat here.
As shown in table 4, table 5, wherein, the index of entitled " order " and " cost " is the index for importing, its
In, the mark of the corresponding empty row of These parameters is followed successively by " preset 1 " and " preset 2 ", when will refer in mapping relations row
Mark data are imported to after table 4, then there are mapping relations between the These parameters and its achievement data in table 5.
Preferably, device also includes：Delete unit, for receive for delete achievement data deletion instruct when,
Deleting to be deleted in metadata table instructs the mapping of the achievement data in indicated index definition and achievement data storage table to close
System；Indexing unit, the metadata token for record deletion in metadata table to be instructed indicated index definition is superfluous
It is remaining.
In the embodiment of the present application, when needing to delete certain achievement data, will first the index definition indicated by instruction be deleted
Delete with the mapping relations of achievement data, and be redundancy by the metadata token of the index definition recorded in metadata table,
So, data analysis system will be deleted at one's leisure.
It should be noted that the data storage device of the embodiment of the present application can also include：Metadata storage unit, is used for
The definition of addition index, the mapping relations set up between index and preset table row, and storage other metadata information,
For inquiring about and storing (for example：Data type is for data memory format verification)；And data cell, for updating
A column data in achievement data storage table or cell data (including newly-increased data or update the data), delete
A column data or a cell data in achievement data storage table, and batch operation is supported, with lifting operation
Can, also support with affairs.Wherein, metadata storage unit equivalent to the adding device in the embodiment of the present application, build
Vertical unit, data cell is equivalent to retrieval unit, import unit, deletion unit, indexing unit etc..
The data storage device includes processor and memorizer, and above-mentioned adding device 301, retrieval unit 303 are set up
Unit 305 and import unit 307 etc. are stored in memory as program unit, are stored in by computing device
Said procedure unit in reservoir.
Kernel is included in processor, is gone in memorizer to transfer corresponding program unit by kernel.Kernel can arrange one
Or more, import achievement data by adjusting kernel parameter.
Memorizer potentially includes the volatile memory in computer-readable medium, random access memory (RAM) and/
Or the form, such as read only memory (ROM) or flash memory (flash RAM) such as Nonvolatile memory, memorizer includes at least one
Individual storage chip.
Present invention also provides a kind of embodiment of computer program, when performing in data handling equipment, fits
In the program code for performing initialization there are as below methods step：In metadata table, add the index definition of achievement data；
In achievement data storage table, the data storage location of non-data storage is retrieved, wherein, data storage location includes finger
Sky and/or behavior sky are classified as in mark table data store；Set up the mapping of index definition and the data storage location for retrieving
Relation, to obtain associated storage position；Achievement data is stored in associated storage position.
The embodiment of the present application additionally provides a kind of data query arrangement, and the device can be used for performing the embodiment of the present application
Data query method, as shown in figure 4, the device includes：Receiving unit 401, determining unit 403 and query unit
405。
Receiving unit 401 is used to receive the query statement for inquiring about achievement data.
Determining unit 403 is used for the index definition from metadata table indicated by determination query statement, determines query statement
The mapping relations of the achievement data in indicated index definition and achievement data storage table.
Query unit 405 is used to inquire about the index indicated by query statement from achievement data storage table using mapping relations
Data.
In the date storage method using the embodiment of the present application achievement data is imported (referring specifically to the number of the embodiment of the present application
According to the description of storage method) after, the achievement data of importing is stored in achievement data storage with original achievement data
In table.
After the query statement for receiving inquiry achievement data, with reference to metadata table and achievement data storage table, from finger
Corresponding achievement data is directly inquired about in mark table data store.Due to the embodiment of the present application it is pre- in achievement data storage table
Data storage location is first provided with for importing achievement data so that in the achievement data and data analysis system after importing
Original achievement data in same achievement data storage table, carry out achievement data inquire about when, without the need for row storage is referred to
Mark is converted into row storage, reduces the IO performance costs that row storage index is converted into row storage, improves search efficiency.
The data query arrangement includes processor and memorizer, above-mentioned receiving unit 401, determining unit 403 and looks into
Ask unit 405 grade to store in memory as program unit, by computing device storage in memory above-mentioned
Program unit.
Kernel is included in processor, is gone in memorizer to transfer corresponding program unit by kernel.Kernel can arrange one
Or more, inquire about achievement data by adjusting kernel parameter.
Memorizer potentially includes the volatile memory in computer-readable medium, random access memory (RAM) and/
Or the form, such as read only memory (ROM) or flash memory (flash RAM) such as Nonvolatile memory, memorizer includes at least one
Individual storage chip.
Present invention also provides a kind of embodiment of computer program, when performing in data handling equipment, fits
In the program code for performing initialization there are as below methods step：The query statement for inquiring about achievement data is received, from unit
The index definition indicated by query statement is determined in tables of data, index definition and index number indicated by query statement is determined
According to the mapping relations of the achievement data in storage table, query statement is inquired about from achievement data storage table using mapping relations
Indicated achievement data.
Above-mentioned the embodiment of the present application sequence number is for illustration only, does not represent the quality of embodiment.
In above-described embodiment of the application, the description to each embodiment all emphasizes particularly on different fields, and does not have in certain embodiment
The part of detailed description, may refer to the associated description of other embodiment.
In several embodiments provided herein, it should be understood that disclosed technology contents, other can be passed through
Mode realize.Wherein, device embodiment described above is only schematic, such as division of described unit,
Can be a kind of division of logic function, can there is other dividing mode, such as multiple units or component when actually realizing
Can with reference to or be desirably integrated into another system, or some features can be ignored, or not perform.It is another, institute
The coupling each other for showing or discussing or direct-coupling or communication connection can be by some interfaces, unit or mould
The INDIRECT COUPLING of block or communication connection, can be electrical or other forms.
The unit as separating component explanation can be or may not be it is physically separate, it is aobvious as unit
The part for showing can be or may not be physical location, you can with positioned at a place, or can also be distributed to
On multiple units.Some or all of unit therein can be according to the actual needs selected to realize this embodiment scheme
Purpose.
In addition, each functional unit in the application each embodiment can be integrated in a processing unit, it is also possible to
It is that unit is individually physically present, it is also possible to which two or more units are integrated in a unit.It is above-mentioned integrated
Unit both can be realized in the form of hardware, it would however also be possible to employ the form of SFU software functional unit is realized.
If the integrated unit is realized and as independent production marketing or when using using in the form of SFU software functional unit,
During a computer read/write memory medium can be stored in.Based on such understanding, the technical scheme essence of the application
On all or part of prior art is contributed part in other words or the technical scheme can be with software product
Form is embodied, and the computer software product is stored in a storage medium, including some instructions are used so that one
Platform computer equipment (can be personal computer, server or network equipment etc.) performs each embodiment institute of the application
State all or part of step of method.And aforesaid storage medium includes：USB flash disk, read only memory (ROM, Read-Only
Memory), random access memory (RAM, Random Access Memory), portable hard drive, magnetic disc or CD
Etc. it is various can be with the medium of store program codes.
The above is only the preferred implementation of the application, it is noted that for the ordinary skill people of the art
For member, on the premise of without departing from the application principle, some improvements and modifications can also be made, these improve and moisten
Decorations also should be regarded as the protection domain of the application.
Claims (10)
1. a kind of date storage method, it is characterised in that include：
In metadata table, add the index definition of achievement data；
In achievement data storage table, the data storage location of non-data storage is retrieved, wherein, the data are deposited
Storage space put comprising the achievement data storage table in be classified as sky and/or behavior empty；
The mapping relations of the index definition and the data storage location for retrieving are set up, to obtain associated storage position
Put；
The achievement data is stored in the associated storage position.
2. method according to claim 1, it is characterised in that set up the index definition and deposit with the data for retrieving
The mapping relations that storage space is put include：
The mark of the data storage location that acquisition is retrieved；
The data form of the corresponding mapping relations row of the index definition is inquired about from the metadata table；
The mark of the data storage location for retrieving is stored in the data form to set up the data storage
The mapping relations of position and the index definition, using the data storage location as the associated storage position.
3. method according to claim 1, it is characterised in that retrieve non-data storage data storage location it
Before, methods described also includes：
The empty row of N row or N row nulls are set in the achievement data storage table, by the empty row or described
Null as the data storage location, wherein, 0 ＜ N≤Max, the Max be store achievement data data
The maximum that place is supported；
The achievement data storage table is set in the database for row storage.
4. method according to claim 1, it is characterised in that methods described also includes：
When receiving for the deletion instruction for deleting achievement data, delete described in the deletion metadata table and refer to
The mapping relations of the achievement data in the indicated index definition of order and the achievement data storage table；
It is redundancy that the metadata token for deleting the indicated index definition of instruction will be recorded in the metadata table.
5. a kind of data query method, it is characterised in that include：
Receive the query statement for inquiring about achievement data；
The index definition indicated by the query statement is determined from metadata table, the query statement indication is determined
The mapping relations of the achievement data in the index definition for showing and achievement data storage table；
The index indicated by the query statement is inquired about from the achievement data storage table using the mapping relations
Data.
6. a kind of data storage device, it is characterised in that include：
Adding device, in metadata table, adding the index definition of achievement data；
Retrieval unit, in achievement data storage table, retrieving the data storage location of non-data storage, its
In, the data storage location is classified as sky and/or behavior sky comprising in the achievement data storage table；
Unit is set up, for setting up the mapping relations of the index definition and the data storage location, to obtain
Associated storage position；And
Import unit, for the achievement data to be stored in the associated storage position.
7. device according to claim 6, it is characterised in that the unit of setting up includes：
Acquisition module, for obtaining the mark of the data storage location for retrieving；
Enquiry module, for inquiring about the number of the corresponding mapping relations row of the index definition from the metadata table
According to form；
Writing module, for the mark of the data storage location for retrieving being stored in the data form to build
The mapping relations of the data storage location and the index definition are found, using the data storage location as described
Associated storage position.
8. device according to claim 6, it is characterised in that described device also includes：
First setting unit, for before the data storage location of non-data storage is retrieved, in the index number
According to the empty row of N row or N row nulls is arranged in storage table, using the empty row or the null as the data
Storage location, wherein, 0 ＜ N≤Max, the Max are to store the maximum that the data place of achievement data is supported；
Second setting unit, for arranging the achievement data storage table in the database for row storage.
9. device according to claim 6, it is characterised in that described device also includes：
Unit is deleted, for when receiving for the deletion instruction for deleting achievement data, deleting the metadata
The mapping of the achievement data in the indicated index definition of instruction and the achievement data storage table is deleted described in table
Relation；
Indexing unit, for the unit for deleting the indicated index definition of instruction will to be recorded in the metadata table
Data markers are redundancy.
10. a kind of data query arrangement, it is characterised in that include：
Receiving unit, for receiving the query statement for inquiring about achievement data；
Determining unit, for determining the index definition indicated by the query statement from metadata table, determines institute
State the mapping relations of the achievement data in the index definition and achievement data storage table indicated by query statement；
Query unit, refers to for the inquiry to be inquired about from the achievement data storage table using the mapping relations
The indicated achievement data of order.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510729753.0A CN106649368A (en) | 2015-10-30 | 2015-10-30 | Data storage method and device and data query method and device |
| PCT/CN2016/102764 WO2017071525A1 (en) | 2015-10-30 | 2016-10-20 | Data storage method and apparatus and data query method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510729753.0A CN106649368A (en) | 2015-10-30 | 2015-10-30 | Data storage method and device and data query method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106649368A true CN106649368A (en) | 2017-05-10 |
Family
ID=58629833
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510729753.0A Pending CN106649368A (en) | 2015-10-30 | 2015-10-30 | Data storage method and device and data query method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106649368A (en) |
| WO (1) | WO2017071525A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021618A (en) * | 2017-11-13 | 2018-05-11 | 北京天元创新科技有限公司 | A kind of data query method and system |
| CN108829897A (en) * | 2018-07-03 | 2018-11-16 | 沈文策 | Date storage method and data-storage system |
| CN112860696A (en) * | 2021-02-07 | 2021-05-28 | 中国邮政储蓄银行股份有限公司 | Data query method and device and data query model |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111291037A (en) * | 2018-11-22 | 2020-06-16 | 中国移动通信集团重庆有限公司 | Data storage and query method, device, equipment and computer storage medium |
| CN112199393B (en) * | 2020-09-18 | 2024-05-10 | 深圳希施玛数据科技有限公司 | Data table generation method, device, equipment and storage medium based on cross-table query |
| CN113422796A (en) * | 2021-05-06 | 2021-09-21 | 北京沃东天骏信息技术有限公司 | Information processing method and device and storage medium |
| CN113535734B (en) * | 2021-07-26 | 2023-07-07 | 安徽容知日新科技股份有限公司 | Data storage method, data query method and computing device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101320388A (en) * | 2008-07-18 | 2008-12-10 | 金蝶软件（中国）有限公司 | Method and apparatus for filtering documents according to documents state |
| CN104376042A (en) * | 2014-10-11 | 2015-02-25 | 武汉天量数据技术有限公司 | Method and system for acquiring test questions from test question database |
| CN104850565A (en) * | 2014-02-19 | 2015-08-19 | 北京京东尚科信息技术有限公司 | Metadata management method based on K-V storage system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7689631B2 (en) * | 2005-05-31 | 2010-03-30 | Sap, Ag | Method for utilizing audience-specific metadata |
-
2015
- 2015-10-30 CN CN201510729753.0A patent/CN106649368A/en active Pending
-
2016
- 2016-10-20 WO PCT/CN2016/102764 patent/WO2017071525A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101320388A (en) * | 2008-07-18 | 2008-12-10 | 金蝶软件（中国）有限公司 | Method and apparatus for filtering documents according to documents state |
| CN104850565A (en) * | 2014-02-19 | 2015-08-19 | 北京京东尚科信息技术有限公司 | Metadata management method based on K-V storage system |
| CN104376042A (en) * | 2014-10-11 | 2015-02-25 | 武汉天量数据技术有限公司 | Method and system for acquiring test questions from test question database |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021618A (en) * | 2017-11-13 | 2018-05-11 | 北京天元创新科技有限公司 | A kind of data query method and system |
| CN108021618B (en) * | 2017-11-13 | 2020-10-16 | 北京天元创新科技有限公司 | Data query method and system |
| CN108829897A (en) * | 2018-07-03 | 2018-11-16 | 沈文策 | Date storage method and data-storage system |
| CN112860696A (en) * | 2021-02-07 | 2021-05-28 | 中国邮政储蓄银行股份有限公司 | Data query method and device and data query model |
| CN112860696B (en) * | 2021-02-07 | 2024-04-12 | 中国邮政储蓄银行股份有限公司 | Data query method and device and data query model |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2017071525A1 (en) | 2017-05-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106649368A (en) | Data storage method and device and data query method and device | |
| US11132346B2 (en) | Information processing method and apparatus | |
| KR101806055B1 (en) | Generating a multi-column index for relational databases by interleaving data bits for selectivity | |
| EP2946333B1 (en) | Efficient query processing using histograms in a columnar database | |
| US11573941B2 (en) | Systems, methods, and data structures for high-speed searching or filtering of large datasets | |
| CN105094707B (en) | A kind of data storage, read method and device | |
| CN109558404B (en) | Data storage method, device, computer equipment and storage medium | |
| US9256665B2 (en) | Creation of inverted index system, and data processing method and apparatus | |
| CN102024046B (en) | Data repeatability checking method and device as well as system | |
| EP3701480B1 (en) | Systems and methods for intelligently grouping financial product users into cohesive cohorts | |
| CN106933897A (en) | Data query method and apparatus | |
| CN109801693B (en) | Medical records grouping method and device, terminal and computer readable storage medium | |
| CN106933935A (en) | task storage method and device | |
| CN108681866B (en) | Waybill processing method, system, device and storage medium | |
| CN105159950A (en) | Mass data real-time sorting and inquiring method and system | |
| US20070239663A1 (en) | Parallel processing of count distinct values | |
| CN103455497A (en) | Method and system for providing integrated search results | |
| CN109739854A (en) | A kind of date storage method and device | |
| CN113298145A (en) | Label filling method and device | |
| CN113761185A (en) | Main key extraction method, equipment and storage medium | |
| CN106855866A (en) | XML document storage method and device | |
| CN104463627A (en) | Data processing method and device | |
| CN106933928A (en) | Task storage method and device based on external data file | |
| CN106815228A (en) | The class name choosing method and device of search keyword | |
| US20230325372A1 (en) | Hierarchical to matrix data management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: Beijing Guoshuang Technology Co.,Ltd. Address before: 100086 Cuigong Hotel, 76 Zhichun Road, Shuangyushu District, Haidian District, Beijing Applicant before: Beijing Guoshuang Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170510 |
|
| RJ01 | Rejection of invention patent application after publication |