CN103902614B - A kind of data processing method, equipment and system - Google Patents
A kind of data processing method, equipment and system Download PDFInfo
- Publication number
- CN103902614B CN103902614B CN201210584674.1A CN201210584674A CN103902614B CN 103902614 B CN103902614 B CN 103902614B CN 201210584674 A CN201210584674 A CN 201210584674A CN 103902614 B CN103902614 B CN 103902614B
- Authority
- CN
- China
- Prior art keywords
- data
- hstore
- column
- master server
- inquiry request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
Abstract
The embodiment of the invention discloses a kind of data processing method, equipment and system, this method includes：Sliced service device receives the data inquiry request of master server forwarding, wherein carrying the list field of the key fields and the data column read for indicating request for indicating to ask the data read to be expert at；The sliced service device inquires about corresponding column data according to the key fields and list field in the data that itself is stored, and the column data inquired is returned to the master server in the form of array.In the present invention, the performance consumption of data processing in distributed column storage database system is reduced, improves data-handling efficiency.
Description
Technical field
The present invention relates to field of communication technology, more particularly to a kind of data processing method, equipment and system.
Background technology
Distributed column storage database is a kind of suitable quick search, distributed excellent fix, it is being provided
While mass data storage, the inquiry velocity to data can also be effectively improved.
Focused primarily in existing distributed column memory technology scheme and how to realize data query, without focused data
The demand of analysis.And in practical application, the major function of database is largely analytic type demand in addition to inquiry.For example,
Count under certain conditions, the summation of some row；Calculated for certain several row, as calculated local telephone network minute and long-distance call minute
Ratio etc..
In view of the above-mentioned problems, the solution in distributed system can be using the method for Distributed Calculation come real at present
It is existing.For example, the system based on Hadoop, using Mapreduce as Computational frame, its Map interface is dbinputformat,
The interface provides the reading to data row.It is specific as follows：
1）, inputformat several bursts can be divided into according to key；
2）, each Map read in a burst；
3）, the read-write interface that provides of Map intrinsic calls distributed data base, according to key, read a line record.
When analysis is realized inside Map, reading is that going for a line a line records, it is necessary first to according to field location
Specific field to be processed is distinguished, is then handled again；Some operations need to enter the reduce stages, such as sum.It is aobvious
So, it is this in the way of row reading process, do not utilize the advantage of column storage.
In the implementation of the present invention, inventor has found to have at least the following problems in the prior art：
Since distributed column stores, each column family preserves in one file, so the interface of a line record is read every time,
Need to read the field responded from multiple files according to key, be then combined with returning for a record；Meanwhile in the Map stages, by
In needing to be operated for some row, it is also necessary to row record is decomposed according to field, could further operate, cause merging
It is lost with the performance twice of fractionation.
The content of the invention
The embodiment of the present invention provides a kind of data processing method, equipment and system, to reduce based on distributed column storage number
According to the performance consumption of the data processing of storehouse system, data-handling efficiency is improved.
In order to reach object above, an embodiment of the present invention provides a kind of data processing method, applied to including main service
In the distributed column storage database system of device and sliced service device, this method includes：
Sliced service device receives the data inquiry request of master server forwarding, wherein carrying what is read for indicating request
The key fields and the list field of the data column read for indicating request that data are expert at；
The sliced service device inquires about corresponding row according to the key fields and list field in the data that itself is stored
Data, and the column data inquired is returned into the master server in the form of array.
The embodiment of the present invention additionally provides a kind of distributed column storage database system, including master server and sliced service
Device,
The master server is used for, and receives the data inquiry request that client is initiated, and the data inquiry request is forwarded
Give sliced service device；And receive the data for the array form that sliced service device returns；
The sliced service device is used for, and receives the data inquiry request of master server forwarding, wherein carrying for indicating
The key fields and the list field of the data column read for indicating request that the data that request is read are expert at；According to institute
State key fields and list field and corresponding column data is inquired about in the data that itself is stored, and by the column data inquired with number
The form of group returns to the master server.
The embodiment of the present invention also provides a kind of sliced service device, applied to the distributed column storage including master server
In the system of storehouse, the distribution server includes：One data slice block Hregion, at least one row module Hstore, Yi Jizhi
A few row storage file HstoreFile；Wherein：
The Hregion is used for, and receives the data inquiry request of main service forwarding, wherein carrying for indicating that request is read
The key fields and the list field of the data column read for indicating request that the data taken are expert at；According to the list
Field determines corresponding Hstore, and the data inquiry request is transmitted to the Hstore；Receive the data texts that Hstore is returned
Part, according to the data file generated data arrays, and returns to master server by the data arrays；
The Hstore is used for, true according to the key fields when receiving the data inquiry request of Hregion forwardings
Fixed corresponding HstoreFile, and the data inquiry request is transmitted to the HstoreFile；Receive what HstoreFile was returned
Data files, and the data files are returned into Hregion；
The HstoreFile is used for, and when receiving the data inquiry request of Hstore forwardings, is returned to Hstore whole
A data files.
In the above embodiment of the present invention, after sliced service device receives the data inquiry request of master server forwarding, according to key
Field and list field inquire about corresponding column data in the data that itself is stored, and by the column data inquired with the shape of array
Formula returns to master server, reduces the performance consumption of data processing in distributed column storage database system, improves data
Treatment effeciency.
Brief description of the drawings
Fig. 1 is existing distributed column storage database system structure diagram；
Fig. 2 is the flow diagram that existing distributed data base reads data；
Fig. 3 is the flow diagram that existing Map tasks handle data；
Fig. 4 is a kind of flow diagram of data processing method provided in an embodiment of the present invention；
Fig. 5 is a kind of flow diagram of data processing method provided in an embodiment of the present invention；
Fig. 6 is a kind of flow diagram of data processing method provided in an embodiment of the present invention；
Fig. 7 is a kind of structure diagram of distributed column storage database system provided in an embodiment of the present invention；
Fig. 8 is a kind of structure diagram of split blade type server provided in an embodiment of the present invention.
Embodiment
The technical solution that embodiment provides for a better understanding of the present invention, below to existing distributed column storage database
System architecture and conventional data processing method based on existing distributed column storage database system architecture carry out simple
Description.
Referring to Fig. 1, existing distributed column storage database system includes master server（Master）With sliced service device
（Tablet Server）, which includes：One data slice block（Hregion）, at least one row module
（Hstore）, and at least one row storage file（HstoreFile）；Wherein：
One or more fragment datas can be stored in one Hregion；The fragment data include former tables of data a line or
The total data of multirow, the number of burst can be determined according to the quantity of the equipment of parallel data processing；
In one sliced service device, the data stored in Hregion are stored in different Hstore by row or column family（I.e.
The data of a row or a Ge Lie families are stored with one Hstore）；The data branch stored in Hstore is stored in
In HstoreFile.Wherein, in distributed column storage database, the several row being often accessed simultaneously are defined as row man
Race.
Based on above-mentioned distributed column storage database system, flow chart of data processing can be such as Fig. 2 and Fig. 3 institutes in the prior art
Show.Wherein, which relates generally to two flows：First be distributed data base read data process；The
Two be Map tasks processing data flow.
Referring to Fig. 2, distributed data base reads the processes of data and may comprise steps of in the prior art：
Step 201, master server receive the data inquiry request that client is sent, please according to the instruction that is used for wherein carried
Ask the key fields that the data of reading are expert at that the data inquiry request is transmitted to corresponding Hregion.
Step 202, Hregion receive data inquiry request, travel through Hstore, key fields are corresponded to inquire about in respective column
Data.
Step 203, Hstore determine corresponding HstoreFile according to key fields；
Step 204, HstoreFile determine the offset of the data of request reading according to the corresponding index of key fields
（offset）, and the offset is returned into Hstore.
Step 205, Hstore read corresponding data according to the offset, and the data of reading are returned to Hregion.
Step 206, Hregion splice the result that all Hstore are returned.
Spliced result is returned to master server by step 207, Hregion.
Wherein, after master server obtains result, Map tasks are outputed it to.
Referring to Fig. 3, the flow that data are handled for Map tasks in the prior art may comprise steps of：
Step 301, Map read in a record（That is data line, Map read in data in a manner of reading line by line）.
Step 302, according to metadata information split out corresponding field value from the record of reading.
Wherein, due to being carried out in a manner of being read by row when Map reads in data, and need what is analyzed and handled
Data are the data of certain row or a few row in tables of data, and therefore, Map is read in after data, it is necessary to according to metadata information from reading
Data in split out corresponding field value（Such as the age）.
Step 303, carry out respective handling to obtained field value（Such as summation）.
In such scheme, remain as when carrying out digital independent and read by row, and due to distributed column storage database
In system, each row or column family are saved in one file, read the interface of data by row, it is necessary to according to key from multiple texts
Part reads the field of response, is then combined with returning for a record, data reading performance using redundancy is relatively low；Further, at Map tasks
In the reason stage, due to needing to be operated for some row, decomposed, just as after row reading data, it is necessary to record row according to field
It can further operate, add the performance consumption of data processing.
In view of the above-mentioned problems, an embodiment of the present invention provides a kind of number being applied in distributed column storage database system
According to the technical solution of processing.In the technical scheme, client is sent to the master server of distributed column storage database system
Data inquiry request in, not only carry the key fields be expert at of data read for indicating request, further include for referring to
Show the list field for the data column that request is read；, should according to key fields after master server receives data inquiry request
Data inquiry request is transmitted to corresponding sliced service device；Sliced service device receives the data inquiry request of master server forwarding
Afterwards, corresponding column data, and the column data that will be inquired are inquired about in the data that itself is stored according to key fields and list field
Master server is returned in the form of array, reduces the performance consumption of data processing in distributed column storage database system,
Improve data-handling efficiency.
Below in conjunction with the attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is carried out clear
Chu, be fully described by, it is clear that the embodiments described below only part of the embodiment of the present invention, rather than whole realities
Apply example.Based on the embodiments of the present invention, those of ordinary skill in the art are obtained without creative efforts
Every other embodiment, belong to the embodiment of the present invention protection scope.
As shown in figure 4, be a kind of flow diagram of data processing method provided in an embodiment of the present invention, can include with
Lower step：
Step 401, sliced service device receive the data inquiry request of master server forwarding, wherein carrying for indicating to ask
The list field of data column sought key fields that the data of reading are expert at and read for indicating request.
Specifically, for distributed column storage database system, when user needs to carry out data query, can by
Client inputs corresponding query argument to initiate data inquiry request to the master server of distributed column storage database system.
In order to make full use of the advantage of distributed column storage database system, in embodiments of the present invention, client to point
In the data inquiry request that the master server of cloth column storage database system is sent, it is used for instruction request except carrying routine
Outside the key fields that the data of reading are expert at, the list field of the data column read for indicating request is also carried.
After master server receives the data inquiry request of client transmission, institute is determined according to the key fields wherein carried
Sliced service device where the data of requesting query, and the data inquiry request is transmitted to corresponding sliced service device.
Step 402, sliced service device inquire about corresponding row according to key fields and list field in the data that itself is stored
Data, and the column data inquired is returned into master server in the form of array.
Specifically, in embodiments of the present invention, the number that sliced service device is stored according to key fields and list field at itself
According to the corresponding column data of middle inquiry, and the specific implementation that the column data inquired is returned to master server in the form of array can
To comprise the following steps：
Step 4021, Hregion determine corresponding Hstore according to the list field carried in data inquiry request, and will
The data inquiry request is transmitted to the Hstore.
Specifically, the fragment data stored in Hregion is stored in Hstore by row or column family, when Hregion is received
To after data inquiry request, the row where the data of request reading are determined according to the list field wherein carried, and then determine to deposit
The Hstore of the column data is stored up, and the data inquiry request is transmitted to the Hstore.
Step 4022, Hstore determine corresponding HstoreFile according to the key fields carried in data inquiry request, and
The data inquiry request is transmitted to the HstoreFile.
Specifically, the data stored in Hstore are stored in HstoreFile by row, looked into when Hstore receives data
After asking request, the row where the data of request reading are determined according to the key fields wherein carried, and then determine storage the row data
HstoreFile, and the data inquiry request is transmitted to the HstoreFile.
After step 4023, HstoreFile receive data inquiry request, whole data files are returned to Hstore.
Specifically, in the prior art, HstoreFile is received after data inquiry request, it is necessary to be corresponded to according to key fields
Index determine the offset of the data that request is read, and the offset is returned into Hstore, by Hstore according to being somebody's turn to do
Offset reads the full line data of corresponding row.
In order to improve data-handling efficiency, in embodiments of the present invention, after HstoreFile receives data inquiry request,
Whole data files are directly returned into Hstore, Hstore is directly obtained corresponding column data, without according to offset
Go to read full line data.
The data files received are returned to Hregion by step 4024, Hstore.
Step 4025, Hregion return to the data arrays according to the data file generated data arrays received
Master server.
In the above manner, realizing the reading of distributed column storage database system midrange evidence, row are taken full advantage of
The advantage of data is stored, reduces the performance consumption of digital independent, improves the efficiency of data processing.
Master server, it is necessary to which data are exported gives Map tasks after receiving the data of sliced service device return, with further
Ground carries out Map task processing.
As shown in fig. 6, data processing method provided in an embodiment of the present invention can also comprise the following steps：
Step 601, Map read in a ColRecord.
Specifically, in embodiments of the present invention, it is defined as follows structure：
ColRecord（Coldata [1], coldata [2] ... ... coldata [n]）
Wherein, n is the columns for the column data that the sliced service device inquires, and coldata [i] is the sliced service device
A column data in the column data inquired, i are the positive integer no more than n.
After Map receives the data array datas of master server output, data are read according to above-mentioned data structure.
Step 602, Map obtain each column data according to the ColRecord.
Step 603, Map carry out data processing according to obtained column data by row.
No longer it is full line data since master server is exported to the data of Map tasks, but data arrays；Map is received
After the data array datas of master server output, data can be read according to ColRecord structures, directly obtained to be treated
Each column data, so that each column data is analyzed and handled by row, without being decomposed again to the row record of reading according to field,
The performance consumption of data processing is reduce further, improves data-handling efficiency.
By above description as can be seen that in technical solution provided in an embodiment of the present invention, client is to distributed column
In the data inquiry request of master server transmission for storing Database Systems, the data read for indicating request are not only carried
The key fields being expert at, further include the list field of the data column read for indicating request；Master server receives number
After inquiry request, which is transmitted to by corresponding sliced service device according to key fields；Sliced service device receives
After the data inquiry request of master server forwarding, correspondence is inquired about in the data that itself is stored according to key fields and list field
Column data, and the column data inquired is returned into master server in the form of array, reduces distributed column storage
The performance consumption of data processing, improves data-handling efficiency in the system of storehouse.
Based on the identical technical concept of above method embodiment, an embodiment of the present invention provides a kind of distributed column to store number
According to storehouse system.
As shown in fig. 7, be a kind of structure diagram of distributed column storage database system provided in an embodiment of the present invention,
It can include master server 71 and sliced service device 72, wherein：
The master server 71 can be used for, and receive the data inquiry request that client is initiated, and please by the data query
Ask and be transmitted to sliced service device 72；And receive the data for the array form that sliced service device 72 returns；
The sliced service device 72 is used for, and receives the data inquiry request that master server 71 forwards, is used for wherein carrying
Indicate the list field for the key fields and data column read for indicating request for asking the data read to be expert at；Root
Corresponding column data, and the column data that will be inquired are inquired about in the data that itself is stored according to the key fields and list field
The master server 71 is returned in the form of array.
Wherein, the sliced service device 72 includes data slice block Hregion, an at least one row module Hstore,
And at least one row storage file HstoreFile；Wherein：
The Hregion is used for, and receives the data inquiry request of main service forwarding, wherein carrying for indicating that request is read
The key fields and the list field of the data column read for indicating request that the data taken are expert at；According to the list
Field determines corresponding Hstore, and the data inquiry request is transmitted to the Hstore；Receive the data texts that Hstore is returned
Part, according to the data file generated data arrays, and returns to master server by the data arrays；
The Hstore is used for, true according to the key fields when receiving the data inquiry request of Hregion forwardings
Fixed corresponding HstoreFile, and the data inquiry request is transmitted to the HstoreFile；Receive what HstoreFile was returned
Data files, and the data files are returned into Hregion；
The HstoreFile is used for, and when receiving the data inquiry request of Hstore forwardings, is returned to Hstore whole
A data files.
Wherein, the master server 71 can be also used for, and the data arrays are exported to Map so that the Map according to
The data arrays read data, and are analyzed and processed according to obtained column data by row.
Wherein, the master server is specifically used for, and data arrays are exported to Map so that the Map according to
ColRecord structures read the data array datas；
The ColRecord structures are specially：
ColRecord（Coldata [1], coldata [2] ... ... coldata [n]）
Wherein, n is the columns for the column data that the sliced service device inquires, and coldata [i] is the sliced service device
A column data in the column data inquired, i are the positive integer no more than n.
Wherein, in distributed column storage database system provided in an embodiment of the present invention, a master server can be right
Should one or more sliced service devices.
Based on the identical technical concept of above method embodiment, the embodiment of the present invention additionally provides a kind of sliced service device,
It can be applied to above method embodiment.
As shown in figure 8, for a kind of structure diagram of sliced service device provided in an embodiment of the present invention, can include：One
A data slice block Hregion81, at least one row module Hstore82, and at least one row storage file
HstoreFile83；Wherein：
The Hregion81 is used for, and the data inquiry request of main service forwarding is received, wherein carrying for indicating request
The key fields and the list field of the data column read for indicating request that the data of reading are expert at；According to the row
Literary name section determines corresponding Hstore82, and the data inquiry request is transmitted to the Hstore82；Hstore82 is received to return
Data files, according to the data file generated data arrays, and the data arrays are returned into master server；
The Hstore82 is used for, when receiving the data inquiry request of Hregion81 forwardings, according to the key words
Section determines corresponding HstoreFile83, and the data inquiry request is transmitted to the HstoreFile83；Receive
The data files that HstoreFile83 is returned, and the data files are returned into Hregion81；
The HstoreFile83 is used for, when receiving the data inquiry request of Hstore82 forwardings, to Hstore82
Return to whole data files.
Through the above description of the embodiments, those skilled in the art can be understood that the embodiment of the present invention
It by hardware realization, can also add the mode of necessary general hardware platform by software to realize.Based on such reason
Solution, the technical solution of the embodiment of the present invention can be embodied in the form of software product, which can be stored in one
A non-volatile memory medium（Can be CD-ROM, USB flash disk, mobile hard disk etc.）In, including some instructions are used so that a meter
Calculate machine equipment（Can be personal computer, server, or network equipment etc.）Perform each implement scene institute of the embodiment of the present invention
The method stated.
It will be appreciated by those skilled in the art that attached drawing is a schematic diagram for being preferable to carry out scene, module in attached drawing or
Flow is not necessarily implemented necessary to the embodiment of the present invention.
It will be appreciated by those skilled in the art that the module in device in implement scene can be described according to implement scene into
Row is distributed in the device of implement scene, can also carry out one or more dresses that respective change is disposed other than this implement scene
In putting.The module of above-mentioned implement scene can be merged into a module, can also be further split into multiple submodule.
The embodiments of the present invention are for illustration only, do not represent the quality of implement scene.
Disclosed above is only several specific implementation scenes of the embodiment of the present invention, and still, the embodiment of the present invention is not office
It is limited to this, the changes that any person skilled in the art can think of should all fall into the business limitation scope of the embodiment of the present invention.
Claims (7)
- A kind of 1. data processing method, applied to the distributed column storage database system for including master server and sliced service device In, it is characterised in that this method includes：Sliced service device receives the data inquiry request of master server forwarding, wherein carrying the data read for indicating request The key fields and the list field of the data column read for indicating request being expert at；The sliced service device inquires about corresponding columns according to the key fields and list field in the data that itself is stored According to, and the column data inquired is returned into the master server in the form of array；Wherein, the sliced service device includes data slice block a Hregion, at least one row module Hstore, Yi Jizhi A few row storage file HstoreFile；The sliced service device inquires about corresponding columns according to the key fields and list field in the data that itself is stored According to, and the column data inquired is returned into the master server in the form of array, it is specially：The Hregion determines corresponding Hstore according to the list field, and the data inquiry request is transmitted to this Hstore；The Hstore determines corresponding HstoreFile according to the key fields, and the data inquiry request is transmitted to this HstoreFile；After the HstoreFile receives data inquiry request, whole data files are returned to Hstore；The data files received are returned to Hregion by the Hstore；The data arrays are returned to master server by the Hregion according to the data file generated data arrays received.
- 2. the method as described in claim 1, it is characterised in that this method further includes：The master server is by the data numbers Group output so that the Map reads data according to the data arrays, and is analyzed to Map according to obtained column data by row Processing.
- 3. method as claimed in claim 2, it is characterised in that the Map reads data according to data arrays, is specially：The Map reads the data array datas according to ColRecord structures；The ColRecord structures are specially：ColRecord (coldata [1], coldata [2] ... ... coldata [n])Wherein, n is the columns for the column data that the sliced service device inquires, and coldata [i] inquires about for the sliced service device To column data in a column data, i is positive integer no more than n.
- 4. a kind of distributed column storage database system, including master server and sliced service device, it is characterised in thatThe master server is used for, and receives the data inquiry request that client is initiated, and the data inquiry request is transmitted to point Piece server；And receive the data for the array form that sliced service device returns；The sliced service device is used for, and receives the data inquiry request of master server forwarding, wherein carrying for indicating request The key fields and the list field of the data column read for indicating request that the data of reading are expert at；According to the key Field and list field inquire about corresponding column data in the data that itself is stored, and by the column data inquired with the shape of array Formula returns to the master server；Wherein, it is characterised in that the sliced service device includes a data slice block Hregion, At least one row module Hstore, and at least one row storage file HstoreFile；Wherein：The Hregion is used for, and receives the data inquiry request of master server forwarding, wherein carrying for indicating that request is read The key fields be expert at of data and the data column read for indicating request list field；According to the list word Section determines corresponding Hstore, and the data inquiry request is transmitted to the Hstore；The data files that Hstore is returned are received, According to the data file generated data arrays, and the data arrays are returned into master server；The Hstore is used for, and when receiving the data inquiry request of Hregion forwardings, is determined pair according to the key fields The HstoreFile answered, and the data inquiry request is transmitted to the HstoreFile；Receive the data that HstoreFile is returned File, and the data files are returned into Hregion；The HstoreFile is used for, and when receiving the data inquiry request of Hstore forwardings, is returned to Hstore whole Data files.
- 5. system as claimed in claim 4, it is characterised in that the master server is additionally operable to, and the data arrays are exported To Map, so that the Map reads data according to the data arrays, and analyzed and processed according to obtained column data by row.
- 6. system as claimed in claim 5, it is characterised in thatThe master server is specifically used for, and data arrays are exported to Map, so that the Map is read according to ColRecord structures The data array datas；The ColRecord structures are specially：ColRecord (coldata [1], coldata [2] ... ... coldata [n])Wherein, n is the columns for the column data that the sliced service device inquires, and coldata [i] inquires about for the sliced service device To column data in a column data, i is positive integer no more than n.
- A kind of 7. sliced service device, applied in the distributed column storage database system including master server, it is characterised in that The sliced service device includes：One data slice block Hregion, at least one row module Hstore, and at least one row Storage file HstoreFile；Wherein：The Hregion is used for, and receives the data inquiry request of master server forwarding, wherein carrying for indicating that request is read The key fields be expert at of data and the data column read for indicating request list field；According to the list word Section determines corresponding Hstore, and the data inquiry request is transmitted to the Hstore；The data files that Hstore is returned are received, According to the data file generated data arrays, and the data arrays are returned into master server；The Hstore is used for, and when receiving the data inquiry request of Hregion forwardings, is determined pair according to the key fields The HstoreFile answered, and the data inquiry request is transmitted to the HstoreFile；Receive the data that HstoreFile is returned File, and the data files are returned into Hregion；The HstoreFile is used for, and when receiving the data inquiry request of Hstore forwardings, is returned to Hstore whole Data files.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210584674.1A CN103902614B (en) | 2012-12-28 | 2012-12-28 | A kind of data processing method, equipment and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210584674.1A CN103902614B (en) | 2012-12-28 | 2012-12-28 | A kind of data processing method, equipment and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103902614A CN103902614A (en) | 2014-07-02 |
| CN103902614B true CN103902614B (en) | 2018-05-04 |
Family
ID=50993942
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210584674.1A Active CN103902614B (en) | 2012-12-28 | 2012-12-28 | A kind of data processing method, equipment and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103902614B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3138025A4 (en) * | 2015-03-28 | 2017-06-14 | Huawei Technologies Co., Ltd. | Apparatus and method for creating user defined variable size tags on records in rdbms |
| CN105404638A (en) * | 2015-09-28 | 2016-03-16 | 高新兴科技集团股份有限公司 | Method for solving correlated query of distributed cross-database fragment table |
| CN106802891A (en) * | 2015-11-26 | 2017-06-06 | 中国电信股份有限公司 | The querying method of the non-burst field of distributed data base, system and equipment |
| CN111090618B (en) * | 2019-10-29 | 2023-08-18 | 厦门网宿有限公司 | Data reading method, system and equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101727465A (en) * | 2008-11-03 | 2010-06-09 | 中国移动通信集团公司 | Methods for establishing and inquiring index of distributed column storage database, device and system thereof |
| CN101828182A (en) * | 2007-09-21 | 2010-09-08 | 哈索-普拉特纳-研究所软件系统有限责任公司 | ETL-less zero redundancy system and method for reporting OLTP data |
| CN102521367A (en) * | 2011-12-16 | 2012-06-27 | 清华大学 | Distributed type processing method based on massive data |
| CN102156714B (en) * | 2011-03-22 | 2012-11-14 | 清华大学 | Method for realizing self-adaptive vertical divided relational database and system thereof |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8671111B2 (en) * | 2011-05-31 | 2014-03-11 | International Business Machines Corporation | Determination of rules by providing data records in columnar data structures |
-
2012
- 2012-12-28 CN CN201210584674.1A patent/CN103902614B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101828182A (en) * | 2007-09-21 | 2010-09-08 | 哈索-普拉特纳-研究所软件系统有限责任公司 | ETL-less zero redundancy system and method for reporting OLTP data |
| CN101727465A (en) * | 2008-11-03 | 2010-06-09 | 中国移动通信集团公司 | Methods for establishing and inquiring index of distributed column storage database, device and system thereof |
| CN102156714B (en) * | 2011-03-22 | 2012-11-14 | 清华大学 | Method for realizing self-adaptive vertical divided relational database and system thereof |
| CN102521367A (en) * | 2011-12-16 | 2012-06-27 | 清华大学 | Distributed type processing method based on massive data |
Non-Patent Citations (1)
| Title |
|---|
| 巧用数组实现多表数据的更新;李作主;《科技信息》;20081225(第36期);第111页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103902614A (en) | 2014-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103678408B (en) | A kind of method and device of inquiry data | |
| US11580168B2 (en) | Method and system for providing context based query suggestions | |
| US11169959B2 (en) | Lineage data for data records | |
| CN103838867A (en) | Log processing method and device | |
| CN105989076A (en) | Data statistical method and device | |
| CN103902614B (en) | A kind of data processing method, equipment and system | |
| US20180373760A1 (en) | Parallel compute offload to database accelerator | |
| CN112835904A (en) | Data processing method and data processing device | |
| US7606906B2 (en) | Bundling and sending work units to a server based on a weighted cost | |
| CN108062384A (en) | The method and apparatus of data retrieval | |
| CN107679158A (en) | Data managing method, device, computer-readable medium and electronic equipment | |
| CN106407442A (en) | Massive text data processing method and apparatus | |
| US20110225287A1 (en) | Method and system for distributed processing of web traffic analytics data | |
| CN109726219A (en) | The method and terminal device of data query | |
| KR20140031429A (en) | Item recommend system and method thereof, apparatus supporting the same | |
| KR101914784B1 (en) | Skyline querying method based on quadtree | |
| CN110909072B (en) | Data table establishment method, device and equipment | |
| CN117235069A (en) | Index creation method, data query method, device, equipment and storage medium | |
| CN110443068A (en) | Method for secret protection and device | |
| CN111723063A (en) | Method and device for processing offline log data | |
| CN113590322A (en) | Data processing method and device | |
| CN112711588B (en) | Method and device for multi-table connection | |
| CN109299112B (en) | Method and apparatus for processing data | |
| CN113706064A (en) | Order processing method and device | |
| CN112799863A (en) | Method and apparatus for outputting information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |