CN101639851B - Method for storing and querying data and devices thereof - Google Patents
Method for storing and querying data and devices thereof Download PDFInfo
- Publication number
- CN101639851B CN101639851B CN 200910089187 CN200910089187A CN101639851B CN 101639851 B CN101639851 B CN 101639851B CN 200910089187 CN200910089187 CN 200910089187 CN 200910089187 A CN200910089187 A CN 200910089187A CN 101639851 B CN101639851 B CN 101639851B
- Authority
- CN
- China
- Prior art keywords
- data
- index
- inquire
- component
- list
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 21
- 238000013500 data storage Methods 0.000 claims description 20
- 238000010586 diagram Methods 0.000 description 5
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for storing and querying data and devices therefor, belonging to the field of a computer. The storing method comprises: decomposing form fields, and obtaining elements forming the fields; setting up an index of the elements; and storing the form which is decomposed and set with the index. The querying method comprises: decomposing the data to be queried, and obtaining elements forming the data to be queried; determining a single index of the data to be queried; setting up the single index of the data to be queried and the corresponding relationship of the elements forming the data to be queried; querying the data to be queried in the stored data which comprises the elements and the index of the elements. The storage device comprises a decomposition module, an index establishing module and a storage module. The querying device includes a decomposition module, an index determining module and a querying module. The invention can improve the data querying efficiency and shorten the data querying time.
Description
Technical field
The present invention relates to computer realm, particularly the method and apparatus of a kind of data storage, inquiry.
Background technology
Along with the day by day surge of data volume, how effectively management data becomes the problem that people are concerned about.Therefore, database technology, particularly the high-volume database technology becomes the hot issue of research.
Current database is often stored data with the form of list.List comprises a plurality of fields, comprises one or more record, and the content of record is the data of storage.During data query, often take field as unit, travel through all records of this list, until find qualified record.
Existing data enquire method needs all data of traversal until find qualified data, and when the data volume of storing in the database was larger, this querying method efficient was lower, and query time is longer.
Summary of the invention
In order to improve efficiency data query, shorten the data query time, the embodiment of the invention provides the method and apparatus of a kind of data storage, inquiry.Described technical scheme is as follows:
A kind of method of data storage, described method comprises:
The field of decomposing list obtains the component of described field;
Set up the single index of described component;
List after storing described decomposition and setting up single index;
The field of described decomposition list comprises:
Judge the data type of described field;
Decompose described field according to the byte that described data type takies.
A kind of method of data query, described method comprises:
Decompose data to inquire, obtain the component of described data to inquire;
Determine the single index of described data to inquire;
Set up the corresponding relation of the component of the single index of described data to inquire and described data to inquire, the described data to inquire of inquiry in the storage data, described storage data storage be the single index of component and described component;
Described decomposition data to inquire comprises: the data type of judging field; Decompose described field according to the byte that described data type takies.
A kind of device of data storage, described device comprises:
Decomposing module for the field of decomposing list, obtains the component of described field;
Module set up in index, is used for setting up the single index of the component that described decomposing module obtains;
Memory module is used for storing the list after described index is set up the decomposition that module obtains and set up single index;
Described decomposing module comprises:
Judging unit is for the data type of judging described field;
Resolving cell decomposes described field for the byte that the data type according to described judgment unit judges takies.
A kind of device of data query, described device comprises:
Decomposing module is used for decomposing data to inquire, obtains the component of described data to inquire;
The index determination module is for the single index of determining described data to inquire;
Enquiry module, be used for setting up the corresponding relation of the component of the single index of the data to inquire that described index determination module determines and the data to inquire that described decomposing module obtains, the described data to inquire of inquiry in the storage data, what described storage data were stored is the single index of component and described component;
Described decomposing module is used for judging the data type of field when decomposing data to inquire; Decompose described field according to the byte that described data type takies.
The beneficial effect that the technical scheme that the embodiment of the invention provides is brought is: improved efficiency data query, shortened the data query time.
Description of drawings
Fig. 1 is the method flow diagram of the data storage that provides of the embodiment of the invention;
Fig. 2 is the other method process flow diagram of the data storage that provides of the embodiment of the invention;
Fig. 3 is the method flow diagram of the data query that provides of another embodiment of the present invention;
Fig. 4 is the apparatus structure synoptic diagram of the data storage that provides of another embodiment of the present invention;
Fig. 5 is another structural representation of device of the data storage that provides of another embodiment of the present invention;
Fig. 6 is the apparatus structure synoptic diagram of the data query that provides of another embodiment of the present invention;
Fig. 7 is another structural representation of device of the data query that provides of another embodiment of the present invention.
Embodiment
For making the purpose, technical solutions and advantages of the present invention clearer, embodiment of the present invention is described further in detail below in conjunction with accompanying drawing.
Referring to Fig. 1, the embodiment of the invention provides a kind of method of data storage, and the method is decomposed form fields, sets up the index that decomposes the component that obtains, and the list after storing this decomposition and setting up index specifically comprises:
101: decompose the field of list, obtain the component of this field;
102: the index of setting up this component;
103: the list after storing this decomposition and setting up index.
The date storage method that the embodiment of the invention provides has improved efficiency data query, has shortened the data query time.
In the database, data are with the form storage of list.List comprises different records, generally distinguishes different records with id, and id can be generated automatically by system.Article one, record comprises several fields.Referring to Fig. 2, below elaborate in database, the method that how using present embodiment provides is stored data.
201: the data of obtaining each field store in the original list.
Concrete, can obtain by manual input or the mode that imports from the external world data of each field store the original list.
For example: invoice list of table 1 expression, formed by " id ", " number ", " password " three fields, store n bar invoice record.Wherein, " id " is used for representing different records; The numeral of " number " expression invoice is comprised of 4 bit digital; The password of " password " expression invoice is comprised of 4 letters.
Table 1
| id | Number | Password |
| 1 | a 1b 1c 1d 1 | A 1B 1C 1D 1 |
| 2 | a 2b 2c 2d 2 | A 2B 2C 2D 2 |
| ... | ... | ... |
| n | a nb nc nd n | A nB nC nD n |
Wherein, 0≤a
i, b
i, c
i, d
i≤ 9, A
i, B
i, C
i, D
i∈ a, b ..., z}, 1≤i≤n, in the mass memory situation, general 2.5 * 10
9≤ n≤∞.
202: decompose the field of original list, obtain the corresponding component of each field.
Concrete, the data type of judgement field is decomposed this field according to the byte that this data type takies, and obtains the component of this field.
Concrete, the data type of field can be numeral, character etc.For example, the data type of " number " is numeral in the table 1, and the data type of " password " is character.The byte that different types of data takies in database is different.For example, individual digit generally takies a byte in database, and single letter generally takies two bytes in database.Therefore, numeral is decomposed according to a byte, and letter decomposes according to two bytes.In addition, numeral can also be decomposed according to the mathematical operation rule.For example, a
1b
1c
1d
1=a
1* 10
3+ b
1* 10
2+ c
1* 10
1+ d
1, a after then decomposing
1b
1c
1d
1=(a
1) (b
1) (c
1) (d
1).Accordingly, the invoice list after the decomposition is as shown in table 2.
Table 2
203: the single index of setting up each component.
For example, the single index of each component foundation of table 2 is as shown in table 3.
Index(a)＝C
a
Index(b)＝C
b
Index(D)＝C
D
Table 3
| id | C a | C b | C c | C d | C A | C B | C C | C D |
| 1 | a 1 | b 1 | c 1 | d 1 | A 1 | B 1 | C 1 | D 1 |
| 2 | a 2 | b 2 | c 2 | d 2 | A 2 | B 2 | C 2 | D 2 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... |
| n | a n | b n | c n | d n | A n | B n | C n | D n |
204: according to single index, set up the combined index of each component.
Concrete, according to single index, in conjunction with actual query demand, for each component is set up combined index.Some single index-group become combined index.For example, during actual the use, there is the query demand of invoice number, then increases the index of invoice number; The query demand that has the invoice password, the index of increase invoice password; The query demand that has invoice number and invoice password, the index of increase invoice number and invoice password, formula is expressed as follows:
Index
Number+password=C
aC
bC
cC
dC
AC
BC
CC
D
Index
Number=C
aC
bC
cC
d
Index
Password=C
AC
BC
CC
D
205: the list that will decompose and set up behind the index stores in the database.
Concrete, if do not store original list in the database, the list after then can directly storing decomposition and setting up index; If stored original list in the database, then replace original list with the list that decomposes and set up behind the index.In order to save the replacement time, the mode that can adopt itemize to replace is that id is 1 record in 1 the record substitution table 1 with id in the table 3 for example, is that id is 2 record in 2 the record substitution table 1 with id in the table 3.
The method that the embodiment of the invention provides by decomposing form fields, is set up the index that decomposes the component that obtains, and the list after storing this decomposition and setting up index so that inquire about when storing data, improves efficiency data query, shortens the data query time.
Based on the method for above-mentioned data storage, referring to Fig. 3, the embodiment of the invention also provides a kind of method of data query, specifically comprises:
301: data to inquire is decomposed, obtain the component of data to inquire;
Wherein, decompose the method for data to inquire, consistent with the method for decomposition field in the data storage implementation example, do not repeat them here.
Take the invoice list as example, whether having invoice number in the data to inquire storehouse is " a
1b
1c
1d
1" record.With a
1b
1c
1d
1Be decomposed into a
1, b
1, c
1, d
1Four components.
302: the single index of determining data to inquire;
Concrete, the combined index of at first definite data to inquire according to this combined index, is further determined the single index that this combined index relates to.
For example: the combined index of determining invoice number is Index
Number, the single index that this combined index comprises is C
aC
bC
cC
d
303: set up the corresponding relation of the component of the single index of data to inquire and data to inquire, inquire about data to inquire in the storage data, what these storage data were stored is the index of component and component.
Wherein, storage data storage method can adopt the method in the data storage implementation example, repeats no more here.
For example: set up wish inquiry invoice number a
1b
1c
1d
1Single index C
aC
bC
cC
dWith set up wish inquiry invoice number a
1b
1c
1d
1Component a
1, b
1, c
1, d
1Corresponding relation be:
“C
a”＝a
1
“C
b”＝b
1
“C
c”＝c
1
“C
d”＝d
1
Then whether having invoice number in the Query Database is " a
1b
1c
1d
1" record, use SQL (Structure Query Language, structured query sentence) inquiry to be specially:
Select*from invoice list where " C
a"=a
1And
“C
b”＝b
1and
“C
c”＝c
1and
“C
d”＝d
1
The method of the data query that present embodiment provides can improve efficiency data query, shortens the data query time.For example: the data v of a n position, v1, v2,,, vn represents respectively each bit digital.Suppose data to inquire (target data) for Y, Y1, Y2,,, Yn is its each bit digital.When the method for using present embodiment to provide is inquired about, at first filter out the data of v1 ≠ Y1, average probability is 9/10, retains the data of v1=Y1, average probability is 1/10; Then filter out the data of v2 ≠ Y2 in the data of retaining, average probability is
The retention probability is
The like, last position inquiry 1 time, hit rate is 100%, then the poorest hit rate of this n bit data is
The below defines several performance index: most optimistic time, most pessimistic time and setup time.
Most optimistic time: there is data to inquire in the database, and within 10,000 data.
Most pessimistic time: do not have data to inquire in the database, need the whole database lookup of traversal.
Setup time: the time that data database storing is finished.
High-volume database has 2.5 * 10 at least
9The bar data, our tentation data storehouse has 2.5 * 10 here
9The bar data, and the hypothesis prior art adopts Select*from invoice list where " number=a
1b
1c
1d
1" mode inquires about, then the contrast of the performance index of test is as follows:
| Test result | Data volume | Most optimistic time | Most pessimistic time | Setup time |
| Prior art | 2.5×10 9 | 1.474s | 124.35s | 1700.5s |
| The present invention | 2.5×10 9 | 0.0005s | 0.0014s | 9623.16s |
From then on as can be known, with respect to prior art, the method for the data query that the embodiment of the invention provides, although setup time is longer, efficiency data query is higher, the data query time is shorter.
Referring to Fig. 4, the embodiment of the invention provides a kind of device of data storage, and this device comprises:
Decomposing module 401 for the field of decomposing list, obtains the component of this field;
Wherein, referring to Fig. 5, decomposing module 401 comprises:
Judging unit 401a is for the data type of judging this field;
Resolving cell 401b is used for decomposing this field according to the byte that the data type that this judging unit 401a judges takies.
Wherein, this index is set up module 402 and is comprised:
Combined index is set up unit 402b, is used for setting up the single index that unit 402a sets up according to this single index, sets up the combined index of this component.
Wherein, this memory module 403, if specifically be used for storing this list, the list after setting up the decomposition that module 402 obtains and set up index with this index is replaced this list.
The data storage device that the embodiment of the invention provides so that inquire about when storing data, has improved efficiency data query, has shortened the data query time.
Referring to Fig. 6, the embodiment of the invention provides a kind of device of data query, and this device comprises:
Decomposing module 501 is used for decomposing data to inquire, obtains the component of this data to inquire;
Wherein, referring to Fig. 7, this index determination module 502 comprises:
Combined index determining unit 502a is for the combined index of determining this data to inquire;
Single index determining unit 502b is used for the combined index definite according to this combined index determining unit 502a, determines the single index of this data to inquire.
The data query device that the embodiment of the invention provides has improved efficiency data query, has shortened the data query time.
All or part of content in the technical scheme that above embodiment provides can realize that by software programming its software program is stored in the storage medium that can read, storage medium for example: the hard disk in the computing machine, CD or floppy disk.
The above only is preferred embodiment of the present invention, and is in order to limit the present invention, within the spirit and principles in the present invention not all, any modification of doing, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (10)
1. the method for data storage is characterized in that described method comprises:
The field of decomposing list obtains the component of described field;
Set up the single index of described component;
List after storing described decomposition and setting up single index;
The field of described decomposition list comprises:
Judge the data type of described field;
Decompose described field according to the byte that described data type takies.
2. the method for claim 1 is characterized in that, the described single index of setting up described component also comprises:
According to described single index, set up the combined index of described component.
3. the method for claim 1 is characterized in that, the described decomposition of described storage and set up single index after list comprise:
If store described list, with described decomposition and the list after setting up single index replace described list.
4. the method for a data query is characterized in that, described method comprises:
Decompose data to inquire, obtain the component of described data to inquire;
Determine the single index of described data to inquire;
Set up the corresponding relation of the component of the single index of described data to inquire and described data to inquire, the described data to inquire of inquiry in the storage data, described storage data storage be the single index of component and described component;
Described decomposition data to inquire comprises: the data type of judging field; Decompose described field according to the byte that described data type takies.
5. method as claimed in claim 4 is characterized in that, the single index of described definite described data to inquire comprises:
Determine the combined index of described data to inquire;
According to described combined index, determine the single index of described data to inquire.
6. the device of data storage is characterized in that described device comprises:
Decomposing module for the field of decomposing list, obtains the component of described field;
Module set up in index, is used for setting up the single index of the component that described decomposing module obtains;
Memory module is used for storing the list after described index is set up the decomposition that module obtains and set up single index;
Described decomposing module comprises:
Judging unit is for the data type of judging described field;
Resolving cell decomposes described field for the byte that the data type according to described judgment unit judges takies.
7. device as claimed in claim 6 is characterized in that, described index is set up module and comprised:
Combined index is set up the unit, is used for setting up the single index that the unit is set up according to described single index, sets up the combined index of described component.
8. device as claimed in claim 6 is characterized in that, described memory module, if specifically be used for storing described list, the list after setting up the decomposition that module obtains and set up single index with described index is replaced described list.
9. the device of a data query is characterized in that, described device comprises:
Decomposing module is used for decomposing data to inquire, obtains the component of described data to inquire;
The index determination module is for the single index of determining described data to inquire;
Enquiry module, be used for setting up the corresponding relation of the component of the single index of the data to inquire that described index determination module determines and the data to inquire that described decomposing module obtains, the described data to inquire of inquiry in the storage data, what described storage data were stored is the single index of component and described component;
Described decomposing module is used for judging the data type of field when being used for decomposing data to inquire; Decompose described field according to the byte that described data type takies.
10. device as claimed in claim 9 is characterized in that, described index determination module comprises:
The combined index determining unit is for the combined index of determining described data to inquire; Single index determining unit is used for the combined index definite according to described combined index determining unit, determines the single index of described data to inquire.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910089187 CN101639851B (en) | 2009-08-03 | 2009-08-03 | Method for storing and querying data and devices thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910089187 CN101639851B (en) | 2009-08-03 | 2009-08-03 | Method for storing and querying data and devices thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101639851A CN101639851A (en) | 2010-02-03 |
| CN101639851B true CN101639851B (en) | 2013-01-02 |
Family
ID=41614835
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200910089187 Expired - Fee Related CN101639851B (en) | 2009-08-03 | 2009-08-03 | Method for storing and querying data and devices thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101639851B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102236659B (en) * | 2010-04-27 | 2014-01-15 | 中国银联股份有限公司 | Method and system for filtering data from data source by using complex conditions |
| CN103942205B (en) * | 2013-01-18 | 2018-06-05 | 深圳市腾讯计算机系统有限公司 | Storage, the method, apparatus and system for reading directory index |
| CN103246738A (en) * | 2013-05-15 | 2013-08-14 | 大唐淮北发电厂 | Query system and method based on input validation |
| CN105447030B (en) * | 2014-08-29 | 2019-06-11 | 阿里巴巴集团控股有限公司 | A kind of index process method and apparatus |
| CN107562743B (en) * | 2016-06-30 | 2022-12-02 | 中兴通讯股份有限公司 | Data storage method and device, and data search request processing method and device |
| CN106649483A (en) * | 2016-10-08 | 2017-05-10 | 浪潮软件股份有限公司 | Dynamic data query method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1588362A (en) * | 2004-09-16 | 2005-03-02 | 卢利华 | Changing method for book machine-readable catalog |
| US20080222104A1 (en) * | 2007-03-06 | 2008-09-11 | Nitrosecurity, Inc. | Clustered index with differentiated subfields |
-
2009
- 2009-08-03 CN CN 200910089187 patent/CN101639851B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1588362A (en) * | 2004-09-16 | 2005-03-02 | 卢利华 | Changing method for book machine-readable catalog |
| US20080222104A1 (en) * | 2007-03-06 | 2008-09-11 | Nitrosecurity, Inc. | Clustered index with differentiated subfields |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101639851A (en) | 2010-02-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101639851B (en) | Method for storing and querying data and devices thereof | |
| EP2263180B1 (en) | Indexing large-scale gps tracks | |
| CN103514201B (en) | Method and device for querying data in non-relational database | |
| CN108376143B (en) | Novel OLAP pre-calculation system and method for generating pre-calculation result | |
| CN105373541A (en) | Processing method and system for data operation request of database | |
| CN108446391A (en) | Processing method, device, electronic equipment and the computer-readable medium of data | |
| CN111061758B (en) | Data storage method, device and storage medium | |
| US20140244654A1 (en) | Data migration | |
| CN103714086A (en) | Method and device used for generating non-relational data base module | |
| CN104090901A (en) | Method, device and server for processing data | |
| CN108205571B (en) | Key value data table connection method and device | |
| CN107577787B (en) | Method and system for storing associated data information | |
| CN102629269A (en) | Searching and storing method for embedded database | |
| WO2017092444A1 (en) | Log data mining method and system based on hadoop | |
| WO2023160137A1 (en) | Graph data storage method and system, and computer device | |
| CN102867049A (en) | Chinese PINYIN quick word segmentation method based on word search tree | |
| CN104408128B (en) | A kind of reading optimization method indexed based on B+ trees asynchronous refresh | |
| CN109213751B (en) | Spark platform based Oracle database parallel migration method | |
| CN101916254A (en) | Form statistical method and device | |
| CN104794237A (en) | Web page information processing method and device | |
| CN112765178A (en) | Real-time indexing method and system for efficient combined query of big electric power data | |
| CN109542912B (en) | Interval data storage method, device, server and storage medium | |
| CN103092838B (en) | A kind of method and device for obtaining English words | |
| CN116541403A (en) | Method, system, electronic device and storage medium for capturing database data change in real time | |
| TWI785724B (en) | Method for creating data warehouse, electronic device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20180123 Address after: 100176 Beijing Beijing economic and Technological Development Zone Culture Park, No. 6, courtyard No. 30, No. 18, 1803 Patentee after: BEIJING MFOX SCIENCE & TECHNOLOGY Co.,Ltd. Address before: 100020 room 1809, room No. 6, Wantong center, Chaoyang District, Beijing Patentee before: Beijing Grandison Information Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20130102 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |