CN115878612B - Database structure and retrieval method thereof - Google Patents
Database structure and retrieval method thereof Download PDFInfo
- Publication number
- CN115878612B CN115878612B CN202211441840.2A CN202211441840A CN115878612B CN 115878612 B CN115878612 B CN 115878612B CN 202211441840 A CN202211441840 A CN 202211441840A CN 115878612 B CN115878612 B CN 115878612B
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- data file
- unit
- storing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 10
- 238000013500 data storage Methods 0.000 claims description 36
- 238000013507 mapping Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims 1
- 230000007547 defect Effects 0.000 abstract description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a database structure, which comprises: the storage units are used for storing data files, and the storage format of each storage unit is the same; the index unit is used for establishing and storing a public index table applicable to all the storage units, and establishing and storing storage table heads corresponding to the storage units one by one; the storage address searching unit is used for searching a storage unit where the target data file is located; and the data file searching unit is used for searching the target data file in the storage unit. The invention can improve the defects of the prior art and improve the retrieval efficiency of the small database.
Description
Technical Field
The invention relates to the technical field of databases, in particular to a database structure and a retrieval method thereof.
Background
For databases with smaller data volume and fast data updating speed, the reading efficiency is an important parameter index. Because the data size is small and the updating speed is high, if a retrieval system is set up as in the case of traditional medium and large data, a great deal of time is spent on updating and maintaining the system, and the system is not suitable for the use scene of the small database. Designing a set of retrieval systems for small databases has become one of the hot spots of current research.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a database structure and a retrieval method thereof, which can solve the defects of the prior art and improve the retrieval efficiency of a small database.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows.
A database structure comprising, in a database, a plurality of data structures,
the storage units are used for storing data files, and the storage format of each storage unit is the same;
the index unit is used for establishing and storing a public index table applicable to all the storage units, and establishing and storing storage table heads corresponding to the storage units one by one;
the storage address searching unit is used for searching a storage unit where the target data file is located;
and the data file searching unit is used for searching the target data file in the storage unit.
The method for searching the database structure comprises the following steps:
A. allocating storage units for the data files according to the data types, wherein each data type is allocated with at least one storage unit;
B. storing the data files into corresponding storage units according to the data types, and when a certain data type corresponds to a plurality of storage units, respectively storing the data files of the data types into different storage units to ensure that the data quantity deviation of the different storage units is less than 3%;
C. the index unit establishes a public index table and a storage table head corresponding to the storage unit one by one;
D. the storage address searching unit searches the storage unit where the target data file is located according to the storage table head, and meanwhile, the data file searching unit searches the target data file in the storage unit according to the public index table.
Preferably, in step a, the number of allocated storage units is determined according to the data amount of the data file of each type of data, and the data amount is proportional to the number of allocated storage units.
Preferably, in step B, when the data amount deviation occurs in different storage units corresponding to the same data type, the temporary filling data file is added to the storage unit lacking the data file, and when the new data file needs to be stored, the temporary filling data file is preferentially deleted and then stored in the new data file.
Preferably, in step C, before the common index table is established, common features are extracted from the data files in different storage units, then a data storage format is set according to the common features, the data files in the storage units are restored according to the data storage format, and finally the common index table is established according to the data storage format.
Preferably, the data storage format includes,
a data storage bit for storing a data file;
the data association bits are in one-to-one correspondence with the data storage bits and are used for storing the association mapping relation between the data files in the corresponding data storage bits and the data files in other data storage bits;
a data reconstruction bit for storing reconstruction information of the data file;
when the data file is restored, firstly, the data file is restored according to the data storage format to enable the data file to meet the requirement of the current data storage format, then the restored data file is stored in the data storage bit, meanwhile, the restoring information of the data file is stored in the data restoring bit, and then the corresponding association mapping relation of the original data file is stored in the corresponding data association bit according to the association relation of the original data file.
Preferably, in step C, the data files in the storage unit are screened using the common characteristics, and the storage header is created using the data files not related to the common characteristics.
Preferably, creating the memory header using a data file that is not related to the common characteristic includes the steps of,
extracting a storage address and an association mapping relation of a data file which are not related to the common characteristics, inputting the storage address and the association mapping relation into a hash function to generate a hash value, and forming the address and the hash value of a storage unit into a storage table head.
The beneficial effects brought by adopting the technical scheme are as follows: the invention designs a data storage format specifically aiming at the characteristics of a small database, and then effectively improves the data retrieval efficiency through synchronous two-dimensional retrieval on the basis. The database retrieval system does not depend on the absolute correspondence between the index table and the data file, has high flexibility, small operand in data updating, and is very suitable for small databases with high data updating speed.
Drawings
Fig. 1 is a schematic diagram of one embodiment of the present invention.
Detailed Description
Referring to fig. 1, one embodiment of the present invention includes,
the storage units 1 are used for storing data files, and the storage format of each storage unit 1 is the same;
an index unit 2, configured to create and store a common index table applicable to all the storage units 1, and create and store a storage table header corresponding to the storage units 1 one by one;
a storage address searching unit 3, configured to search a storage unit 1 in which the target data file is located;
a data file searching unit 4 for searching the storage unit 1 for the target data file.
The method for searching the database structure comprises the following steps:
A. allocating storage units 1 for the data files according to the data types, and allocating at least one storage unit 1 for each data type;
B. storing the data files into the corresponding storage units 1 according to the data types, and when a certain data type corresponds to a plurality of storage units 1, storing the data files of the data types into different storage units 1 respectively to ensure that the data quantity deviation of different storage units 1 is less than 3%;
C. the index unit 2 establishes a public index table and a storage table head corresponding to the storage unit 1 one by one;
D. the storage address searching unit 3 searches the storage unit 1 where the target data file is located according to the storage table header, and at the same time, the data file searching unit 4 searches the target data file in the storage unit 1 according to the common index table.
In step a, the number of allocation memory units 1 is determined according to the data amount of the data file of each type of data, and the data amount is proportional to the number of allocation memory units 1.
In step B, when the data amount deviation occurs in different storage units 1 corresponding to the same data type, the temporary filling data file is added into the storage unit 1 lacking the data file, and when the new data file needs to be stored, the temporary filling data file is preferentially deleted and then stored into the new data file.
In step C, before the common index table is established, common features are extracted from the data files in different storage units 1, then a data storage format is set according to the common features, the data files in the storage units 1 are restored according to the data storage format, and finally the common index table is established according to the data storage format.
The data storage format includes the data storage format of,
a data storage bit for storing a data file;
the data association bits are in one-to-one correspondence with the data storage bits and are used for storing the association mapping relation between the data files in the corresponding data storage bits and the data files in other data storage bits;
a data reconstruction bit for storing reconstruction information of the data file;
when the data file is restored, firstly, the data file is restored according to the data storage format to enable the data file to meet the requirement of the current data storage format, then the restored data file is stored in the data storage bit, meanwhile, the restoring information of the data file is stored in the data restoring bit, and then the corresponding association mapping relation of the original data file is stored in the corresponding data association bit according to the association relation of the original data file.
In step C, the data files in the storage unit 1 are filtered using the common characteristics, and a storage header is created using data files that are not related to the common characteristics.
The creation of a memory header using a data file that is not related to a common feature includes the steps of,
extracting a storage address and an association mapping relation of a data file which are not related to the common characteristics, inputting the storage address and the association mapping relation into a hash function to generate a hash value, and forming the address and the hash value of the storage unit 1 into a storage table head.
In the description of the present invention, it should be understood that the terms "longitudinal," "transverse," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like indicate or are based on the orientation or positional relationship shown in the drawings, merely to facilitate description of the present invention, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be construed as limiting the present invention.
The foregoing has shown and described the basic principles and main features of the present invention and the advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that the above embodiments and descriptions are merely illustrative of the principles of the present invention, and various changes and modifications may be made without departing from the spirit and scope of the invention, which is defined in the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (3)
1. A method of retrieving a database structure, the database structure comprising,
the storage units are used for storing data files, and the storage format of each storage unit is the same;
the index unit is used for establishing and storing a public index table applicable to all the storage units, and establishing and storing storage table heads corresponding to the storage units one by one;
the storage address searching unit is used for searching a storage unit where the target data file is located;
the data file searching unit is used for searching the target data file in the storage unit;
the method is characterized by comprising the following steps of:
A. allocating storage units for the data files according to the data types, wherein each data type is allocated with at least one storage unit;
B. storing the data files into corresponding storage units according to the data types, and when a certain data type corresponds to a plurality of storage units, respectively storing the data files of the data types into different storage units to ensure that the data quantity deviation of the different storage units is less than 3%;
C. the index unit establishes a public index table and a storage table head corresponding to the storage unit one by one;
before a common index table is established, common features are extracted from data files in different storage units, then a data storage format is set according to the common features, the data files in the storage units are restored according to the data storage format, and finally the common index table is established according to the data storage format; the data storage format includes the data storage format of,
a data storage bit for storing a data file;
the data association bits are in one-to-one correspondence with the data storage bits and are used for storing the association mapping relation between the data files in the corresponding data storage bits and the data files in other data storage bits;
a data reconstruction bit for storing reconstruction information of the data file;
when the data file is restored, firstly, the data file is restored according to the data storage format to enable the data file to meet the requirement of the current data storage format, then the restored data file is stored in the data storage bit, meanwhile, the restoring information of the data file is stored in the data restoring bit, and then the corresponding association mapping relation of the original data file is stored in the corresponding data association bit according to the association relation of the original data file;
screening the data files in the storage unit by using the common characteristics, and establishing a storage header by using the data files which are not related to the common characteristics;
the creation of a memory header using a data file that is not related to a common feature includes the steps of,
extracting a storage address and an association mapping relation of a data file which are not related to the common characteristics, inputting the storage address and the association mapping relation into a hash function to generate a hash value, and forming the address and the hash value of a storage unit into a storage table head;
D. the storage address searching unit searches the storage unit where the target data file is located according to the storage table head, and meanwhile, the data file searching unit searches the target data file in the storage unit according to the public index table.
2. The method for retrieving a database structure according to claim 1, wherein: in step a, the number of allocated storage units is determined according to the data amount of the data file of each type of data, and the data amount is proportional to the number of allocated storage units.
3. The method for retrieving a database structure according to claim 2, wherein: in the step B, when the data quantity deviation occurs in different storage units corresponding to the same data type, the temporary filling data file is supplemented into the storage unit lacking the data file, and when a new data file needs to be stored, the temporary filling data file is preferentially deleted and then stored into the new data file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211441840.2A CN115878612B (en) | 2022-11-17 | 2022-11-17 | Database structure and retrieval method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211441840.2A CN115878612B (en) | 2022-11-17 | 2022-11-17 | Database structure and retrieval method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115878612A CN115878612A (en) | 2023-03-31 |
| CN115878612B true CN115878612B (en) | 2023-12-15 |
Family
ID=85760140
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211441840.2A Active CN115878612B (en) | 2022-11-17 | 2022-11-17 | Database structure and retrieval method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115878612B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101286157A (en) * | 2007-09-28 | 2008-10-15 | 深圳市天朗时代科技有限公司 | File retrieval method and apparatus and time stream file processor |
| CN103186617A (en) * | 2011-12-30 | 2013-07-03 | 北京新媒传信科技有限公司 | Data storage method and device |
| CN105426408A (en) * | 2015-11-02 | 2016-03-23 | 北京锐安科技有限公司 | Multi-index data processing method and apparatus |
| CN106709851A (en) * | 2016-11-30 | 2017-05-24 | 中体彩科技发展有限公司 | Big data retrieval method and apparatus |
| CN106960020A (en) * | 2017-03-09 | 2017-07-18 | 星环信息科技（上海）有限公司 | A kind of method and apparatus for creating concordance list |
| CN111651372A (en) * | 2020-05-13 | 2020-09-11 | 湖北三江航天万峰科技发展有限公司 | Flash retrieval method based on Hash search and storage medium |
| CN112434190A (en) * | 2020-12-22 | 2021-03-02 | 南京云悦欣自动化工程有限公司 | Method for carrying out query management on planned operation |
| CN114356851A (en) * | 2022-01-12 | 2022-04-15 | 北京字节跳动网络技术有限公司 | Data file storage method and device, electronic equipment and storage medium |
-
2022
- 2022-11-17 CN CN202211441840.2A patent/CN115878612B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101286157A (en) * | 2007-09-28 | 2008-10-15 | 深圳市天朗时代科技有限公司 | File retrieval method and apparatus and time stream file processor |
| CN103186617A (en) * | 2011-12-30 | 2013-07-03 | 北京新媒传信科技有限公司 | Data storage method and device |
| CN105426408A (en) * | 2015-11-02 | 2016-03-23 | 北京锐安科技有限公司 | Multi-index data processing method and apparatus |
| CN106709851A (en) * | 2016-11-30 | 2017-05-24 | 中体彩科技发展有限公司 | Big data retrieval method and apparatus |
| CN106960020A (en) * | 2017-03-09 | 2017-07-18 | 星环信息科技（上海）有限公司 | A kind of method and apparatus for creating concordance list |
| CN111651372A (en) * | 2020-05-13 | 2020-09-11 | 湖北三江航天万峰科技发展有限公司 | Flash retrieval method based on Hash search and storage medium |
| CN112434190A (en) * | 2020-12-22 | 2021-03-02 | 南京云悦欣自动化工程有限公司 | Method for carrying out query management on planned operation |
| CN114356851A (en) * | 2022-01-12 | 2022-04-15 | 北京字节跳动网络技术有限公司 | Data file storage method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115878612A (en) | 2023-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8099421B2 (en) | File system, and method for storing and searching for file by the same | |
| CN105320775B (en) | The access method and device of data | |
| US8225029B2 (en) | Data storage processing method, data searching method and devices thereof | |
| JP6262874B2 (en) | Database implementation method | |
| CN112395212B (en) | Method and system for reducing garbage recovery and write amplification of key value separation storage system | |
| CN104346357A (en) | File accessing method and system for embedded terminal | |
| CN103995855A (en) | Method and device for storing data | |
| CN106682110B (en) | Image file storage and management system and method based on Hash grid index | |
| CN110888837B (en) | Object storage small file merging method and device | |
| CN106502587A (en) | Data in magnetic disk management method and magnetic disk control unit | |
| CN110109927A (en) | Oracle database data processing method based on LSM tree | |
| CN106201778B (en) | Information processing method and storage equipment | |
| CN102609492B (en) | Metadata management method supporting variable table modes | |
| CN107463333B (en) | Method, device and system for recovering network hard disk space | |
| CN112395288B (en) | R-tree index merging and updating method, device and medium based on Hilbert curve | |
| CN1845093A (en) | Attribute extensible object file system | |
| CN104765754A (en) | Data storage method and device | |
| US20130339401A1 (en) | Method for Reliable and Efficient Filesystem Metadata Conversion | |
| CN110597805B (en) | Memory index structure processing method | |
| CN103530067B (en) | A kind of method and apparatus of data manipulation | |
| CN115878612B (en) | Database structure and retrieval method thereof | |
| CN115576956B (en) | Data processing method, system, equipment and storage medium | |
| CN107786599B (en) | Memory cloud system | |
| CN113326262B (en) | Data processing method, device, equipment and medium based on key value database | |
| CN111190545B (en) | Method for carrying out LPM rule compression storage on trie structure based on software implementation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20231123 Address after: Room 6331, 6332, 6333, 6334, Beijing Exhibition Hall Hotel, No. 135 Xizhimenwai Street, Xicheng District, Beijing, 100032 Applicant after: Beijing Oriental Jingrong Education Technology Co.,Ltd. Address before: 050000 No. 7, Hebian Lane, Wangjiazhuang Village, Shangzhuang Town, Luquan District, Shijiazhuang City, Hebei Province Applicant before: Shijiazhuang Zongyu Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |