CN117520384A - Metadata query optimization method and terminal - Google Patents
Metadata query optimization method and terminal Download PDFInfo
- Publication number
- CN117520384A CN117520384A CN202311686764.6A CN202311686764A CN117520384A CN 117520384 A CN117520384 A CN 117520384A CN 202311686764 A CN202311686764 A CN 202311686764A CN 117520384 A CN117520384 A CN 117520384A
- Authority
- CN
- China
- Prior art keywords
- information
- metadata
- index
- data
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000005457 optimization Methods 0.000 title claims abstract description 28
- 238000012163 sequencing technique Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000001174 ascending effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a metadata query optimization method and a terminal, which are used for carrying out index query on an ordered index sequence in a memory by a binary search method to obtain index information; acquiring associated metadata information in a memory according to the queried index information; acquiring complete target data from a data source according to the metadata information; the index information, the metadata information and the target data are sequentially and uniquely associated; the invention establishes the index for the metadata information in the memory and sorts the index, so that the index data can be quickly inquired through a binary search method when the data is inquired, the metadata can be quickly acquired according to the inquired index information, and then the target data can be acquired, and compared with traversing the metadata in a cache, the inquiry of the target data can be more quickly realized, and the effective improvement of the data inquiry efficiency can be realized.
Description
Technical Field
The present invention relates to the field of metadata query technologies, and in particular, to a method and a terminal for metadata query optimization.
Background
In the existing database and information query software, a rapid query function is generally provided. The main realization logic is that each piece of data has related metadata information, when specific information is queried, the metadata information is queried preferentially, and after a result is queried, the complete information is queried again in a real data source according to the metadata information. In order to improve the quick query function of metadata, in general, various software or databases and the like store and load metadata information into a memory, and the quick query function of metadata is realized through the memory.
The metadata information is stored in a memory in a certain cache block, that is, some or some metadata list information exists in one cache block. Metadata information is typically stored in such cache blocks in a list, and when a metadata query is made, the metadata in the cache blocks is polled and corresponding metadata information is found. Because of the pure memory operation, even the polling operation in the cache block can quickly realize metadata searching.
However, with another increase in performance requirements, how to further increase the search efficiency is also a technical problem to be studied.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the metadata query optimization method and the terminal are provided, and the data query efficiency is effectively improved.
In order to solve the technical problems, the invention adopts the following technical scheme:
a method of metadata query optimization, comprising the steps of:
s1, index inquiry is carried out on an ordered index sequence in a memory through a binary search method, and index information is obtained;
s2, acquiring associated metadata information in the memory according to the queried index information;
s3, acquiring complete target data from a data source according to the metadata information;
the index information, the metadata information and the target data are sequentially and uniquely associated.
In order to solve the technical problems, the invention adopts another technical scheme that:
a terminal for metadata query optimization comprising a processor, a memory and a computer program stored in the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
s1, index inquiry is carried out on an ordered index sequence in a memory through a binary search method, and index information is obtained;
s2, acquiring associated metadata information in the memory according to the queried index information;
s3, acquiring complete target data from a data source according to the metadata information;
the index information, the metadata information and the target data are sequentially and uniquely associated.
The invention has the beneficial effects that: according to the metadata query optimization method and terminal, indexes are built for metadata information in a memory, and the indexes are ordered, so that index data can be quickly queried through a binary search method when data is queried, metadata can be quickly obtained according to queried index information, and then target data can be obtained, and compared with traversing the metadata in a cache, query of the target data can be more quickly achieved, and the data query efficiency is effectively improved.
Drawings
FIG. 1 is a flow chart of a method of metadata query optimization in accordance with an embodiment of the present invention;
FIG. 2 is a block diagram of a terminal for metadata query optimization according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a comparison of a data storage structure with a legacy storage structure in a metadata query optimization method according to an embodiment of the present invention;
description of the reference numerals:
1. a terminal for metadata query optimization; 2. a processor; 3. a memory.
Detailed Description
In order to describe the technical contents, the achieved objects and effects of the present invention in detail, the following description will be made with reference to the embodiments in conjunction with the accompanying drawings.
Referring to fig. 1 and 3, a method for optimizing metadata query includes the steps of:
s1, index inquiry is carried out on an ordered index sequence in a memory through a binary search method, and index information is obtained;
s2, acquiring associated metadata information in the memory according to the queried index information;
s3, acquiring complete target data from a data source according to the metadata information;
the index information, the metadata information and the target data are sequentially and uniquely associated.
From the above description, the beneficial effects of the invention are as follows: according to the metadata query optimization method and terminal, indexes are built for metadata information in a memory, and the indexes are ordered, so that index data can be quickly queried through a binary search method when data is queried, metadata can be quickly obtained according to queried index information, and then target data can be obtained, and compared with traversing the metadata in a cache, query of the target data can be more quickly achieved, and the data query efficiency is effectively improved.
Further, the index information includes unique identification information in the metadata information and pointer information pointing to the metadata information storage location;
the unique identification information carries out unique identification on the target data;
the index sequence is obtained by sequencing the index information according to the unique identification information.
As can be seen from the above description, the index information is ordered by the unique identification information and includes pointer information for pointing to the storage location of the metadata in the memory.
Further, the step S1 specifically includes:
and acquiring the unique identification information to be queried, and carrying out index query on the ordered index sequence by a binary search method to obtain index information.
From the above description, the index information is ordered according to the unique identification information, so that when data is queried, binary query can be performed through the unique identification, the corresponding index information can be quickly obtained, and the query efficiency is improved.
Further, the step S2 specifically includes:
and obtaining the storage position of the metadata information in the memory according to the pointer information in the index information, and obtaining the metadata information from the storage position.
As can be seen from the above description, the index points to the associated metadata information location through the pointer, so that information can be directly read from the location to obtain corresponding metadata, thereby avoiding unnecessary data query.
Further, the method further comprises the steps of:
s4, modifying the metadata information and the index sequence according to the data change information of the data source, and re-sequencing the index sequence.
It can be seen from the above description that when there is a data change in the data source, the metadata and the index sequence are correspondingly adjusted, and the index sequence is reordered, so as to ensure the searching efficiency, and avoid the influence of the data change on the efficiency of data searching.
Referring to fig. 2, a terminal for metadata query optimization includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:
s1, index inquiry is carried out on an ordered index sequence in a memory through a binary search method, and index information is obtained;
s2, acquiring associated metadata information in the memory according to the queried index information;
s3, acquiring complete target data from a data source according to the metadata information;
the index information, the metadata information and the target data are sequentially and uniquely associated.
From the above description, the beneficial effects of the invention are as follows: according to the metadata query optimization method and terminal, indexes are built for metadata information in a memory, and the indexes are ordered, so that index data can be quickly queried through a binary search method when data is queried, metadata can be quickly obtained according to queried index information, and then target data can be obtained, and compared with traversing the metadata in a cache, query of the target data can be more quickly achieved, and the data query efficiency is effectively improved.
Further, the index information includes unique identification information in the metadata information and pointer information pointing to the metadata information storage location;
the unique identification information carries out unique identification on the target data;
the index sequence is obtained by sequencing the index information according to the unique identification information.
As can be seen from the above description, the index information is ordered by the unique identification information and includes pointer information for pointing to the storage location of the metadata in the memory.
Further, the step S1 specifically includes:
and acquiring the unique identification information to be queried, and carrying out index query on the ordered index sequence by a binary search method to obtain index information.
From the above description, the index information is ordered according to the unique identification information, so that when data is queried, binary query can be performed through the unique identification, the corresponding index information can be quickly obtained, and the query efficiency is improved.
Further, the step S2 specifically includes:
and obtaining the storage position of the metadata information in the memory according to the pointer information in the index information, and obtaining the metadata information from the storage position.
As can be seen from the above description, the index points to the associated metadata information location through the pointer, so that information can be directly read from the location to obtain corresponding metadata, thereby avoiding unnecessary data query.
Further, the method further comprises the steps of:
s4, modifying the metadata information and the index sequence according to the data change information of the data source, and re-sequencing the index sequence.
It can be seen from the above description that when there is a data change in the data source, the metadata and the index sequence are correspondingly adjusted, and the index sequence is reordered, so as to ensure the searching efficiency, and avoid the influence of the data change on the efficiency of data searching.
The metadata query optimization method and the terminal are suitable for scenes in which quick query of data is required.
Referring to fig. 1 and 3, a first embodiment of the present invention is as follows:
a method of metadata query optimization, comprising the steps of:
s1, index inquiry is carried out on an ordered index sequence in a memory through a binary search method, and index information is obtained;
the index information includes unique identification information in the metadata information and pointer information pointing to the metadata information storage location;
the index information, the metadata information and the target data are sequentially and uniquely associated, and the unique identification information carries out unique identification on the target data;
the index sequence is obtained by sequencing the index information according to the unique identification information.
In this embodiment, a small memory space is newly and independently opened in the buffer block, and the space is used to store the pointer sequence of the buffer block. When a piece of metadata is newly added in the cache block, the metadata is required to be written into a pointer sequence of the cache block according to key information in the metadata, namely unique identification information (for example, service primary key id or service unique number information), and ascending sort processing is required to be performed in the pointer sequence aiming at the added key information. Each key will have a pointer to the actual metadata information.
The step S1 specifically comprises the following steps:
and acquiring the unique identification information to be queried, and carrying out index query on the ordered index sequence by a binary search method to obtain index information.
When the metadata information needs to be queried from the cache block, a binary search method is preferably used to query related data from a pointer sequence (i.e. an index sequence), and in general, the specific metadata pointer (index information) can be quickly located after about 3 times of query.
S2, acquiring associated metadata information in the memory according to the queried index information;
the step S2 specifically comprises the following steps:
and obtaining the storage position of the metadata information in the memory according to the pointer information in the index information, and obtaining the metadata information from the storage position.
In this embodiment, according to the metadata pointer (index information) obtained by the query, wherein the pointer is a physical address, the direct recording of the pointer is actually the direct physical address where the data is directly recorded, and the metadata can be found directly according to the address.
S3, acquiring complete target data from a data source according to the metadata information.
S4, modifying the metadata information and the index sequence according to the data change information of the data source, and re-sequencing the index sequence.
In this embodiment, when there is a change in the data source, and thus metadata needs to be changed, the index sequence is changed synchronously, and the index sequence is reordered. For example, when one piece of metadata information is reduced in the cache block, the index sequence is also reordered.
Referring to fig. 2, a second embodiment of the present invention is as follows:
a terminal 1 for metadata query optimization comprising a processor 2, a memory 3 and a computer program stored in the memory 3 and executable on the processor 2, the steps of the above embodiment being implemented when the processor 2 executes the computer program.
In summary, according to the metadata query optimization method and terminal provided by the invention, indexes are established for metadata information in a memory, and the indexes are ordered, so that index data can be quickly queried through a binary search method during data query, metadata can be quickly obtained according to the queried index information, and further target data can be obtained, and compared with traversing the metadata in a cache, the target data query can be more quickly realized. According to the invention, the structure of the buffer block is further modified, the pointer sequence data structure is newly added in the buffer block by increasing the capacity of the buffer block, the original polling inquiry mode is modified into a mode of two-way inquiry, so that the quick inquiry in the buffer block is realized, the inquiry times in the buffer block are further reduced rapidly by changing the structure, and the inquiry efficiency of metadata in the buffer block is further effectively improved.
The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent changes made by the specification and drawings of the present invention, or direct or indirect application in the relevant art, are included in the scope of the present invention.
Claims (10)
1. A method of metadata query optimization, comprising the steps of:
s1, index inquiry is carried out on an ordered index sequence in a memory through a binary search method, and index information is obtained;
s2, acquiring associated metadata information in the memory according to the queried index information;
s3, acquiring complete target data from a data source according to the metadata information;
the index information, the metadata information and the target data are sequentially and uniquely associated.
2. The method of claim 1, wherein the index information includes unique identification information in the metadata information and pointer information to the metadata information storage location;
the unique identification information carries out unique identification on the target data;
the index sequence is obtained by sequencing the index information according to the unique identification information.
3. The method of metadata query optimization according to claim 2, wherein step S1 is specifically:
and acquiring the unique identification information to be queried, and carrying out index query on the ordered index sequence by a binary search method to obtain index information.
4. The method of metadata query optimization according to claim 2, wherein step S2 is specifically:
and obtaining the storage position of the metadata information in the memory according to the pointer information in the index information, and obtaining the metadata information from the storage position.
5. The method of metadata query optimization of claim 1, further comprising the steps of:
s4, modifying the metadata information and the index sequence according to the data change information of the data source, and re-sequencing the index sequence.
6. A terminal for metadata query optimization comprising a processor, a memory and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the following steps when executing the computer program:
s1, index inquiry is carried out on an ordered index sequence in a memory through a binary search method, and index information is obtained;
s2, acquiring associated metadata information in the memory according to the queried index information;
s3, acquiring complete target data from a data source according to the metadata information;
the index information, the metadata information and the target data are sequentially and uniquely associated.
7. The terminal for metadata query optimization as claimed in claim 6, wherein the index information includes unique identification information in the metadata information and pointer information pointing to the metadata information storage location;
the unique identification information carries out unique identification on the target data;
the index sequence is obtained by sequencing the index information according to the unique identification information.
8. The terminal for metadata query optimization according to claim 7, wherein step S1 is specifically:
and acquiring the unique identification information to be queried, and carrying out index query on the ordered index sequence by a binary search method to obtain index information.
9. The terminal for metadata query optimization according to claim 7, wherein step S2 is specifically:
and obtaining the storage position of the metadata information in the memory according to the pointer information in the index information, and obtaining the metadata information from the storage position.
10. The terminal for metadata query optimization of claim 6, further comprising the steps of:
s4, modifying the metadata information and the index sequence according to the data change information of the data source, and re-sequencing the index sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311686764.6A CN117520384A (en) | 2023-12-08 | 2023-12-08 | Metadata query optimization method and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311686764.6A CN117520384A (en) | 2023-12-08 | 2023-12-08 | Metadata query optimization method and terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117520384A true CN117520384A (en) | 2024-02-06 |
Family
ID=89749417
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311686764.6A Pending CN117520384A (en) | 2023-12-08 | 2023-12-08 | Metadata query optimization method and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117520384A (en) |
-
2023
- 2023-12-08 CN CN202311686764.6A patent/CN117520384A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6725223B2 (en) | Storage format for encoded vector indexes | |
| CN105117417A (en) | Read-optimized memory database Trie tree index method | |
| US9535940B2 (en) | Intra-block partitioning for database management | |
| CN107341178B (en) | Data retrieval method based on self-adaptive binary quantization Hash coding | |
| CN105975587A (en) | Method for organizing and accessing memory database index with high performance | |
| WO2013152678A1 (en) | Method and device for metadata query | |
| US11221999B2 (en) | Database key compression | |
| CN102819586A (en) | Uniform Resource Locator (URL) classifying method and equipment based on cache | |
| CN116450656B (en) | Data processing method, device, equipment and storage medium | |
| CN109299143B (en) | Knowledge fast indexing method of data interoperation test knowledge base based on Redis cache | |
| CN112131218A (en) | Hash table look-up method, device and equipment for gene comparison and storage medium | |
| CN114691721A (en) | Graph data query method and device, electronic equipment and storage medium | |
| CN111080290A (en) | Block chain transaction execution and data storage method based on account classification | |
| CN114969069B (en) | Heat perception local updating method applied to key value storage system | |
| CN112732725A (en) | NVM (non volatile memory) hybrid memory-based adaptive prefix tree construction method, system and medium | |
| CN109213760B (en) | High-load service storage and retrieval method for non-relational data storage | |
| CN111723092B (en) | Data processing method and device | |
| CN113688130A (en) | Memory database storage engine management method | |
| CN113392089A (en) | Database index optimization method and readable storage medium | |
| CN109325022A (en) | A kind of data processing method and device | |
| US7792825B2 (en) | Fast select for fetch first N rows with order by | |
| CN116975006A (en) | Data deduplication method, system and medium based on disk cache and B-tree index | |
| CN117520384A (en) | Metadata query optimization method and terminal | |
| CN113625967B (en) | Data storage method, data query method and server | |
| JPH09305622A (en) | Method and system for managing data base having document retrieval function |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |