CN112434025B - Method, system, device and medium for optimizing index persistence - Google Patents
Method, system, device and medium for optimizing index persistence Download PDFInfo
- Publication number
- CN112434025B CN112434025B CN202011181128.4A CN202011181128A CN112434025B CN 112434025 B CN112434025 B CN 112434025B CN 202011181128 A CN202011181128 A CN 202011181128A CN 112434025 B CN112434025 B CN 112434025B
- Authority
- CN
- China
- Prior art keywords
- thread
- persistent
- memory
- persistent memory
- memory device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000002688 persistence Effects 0.000 title claims abstract description 38
- 230000002085 persistent effect Effects 0.000 claims abstract description 163
- 230000015654 memory Effects 0.000 claims abstract description 114
- 230000005055 memory storage Effects 0.000 claims abstract description 40
- 230000004044 response Effects 0.000 claims abstract description 13
- 230000036541 health Effects 0.000 claims abstract description 11
- 238000004590 computer program Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 description 10
- 238000007726 management method Methods 0.000 description 6
- 238000011161 development Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 239000008186 active pharmaceutical agent Substances 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2272—Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a method, a system, equipment and a storage medium for optimizing index persistence, wherein the method comprises the following steps: initializing the persistent memory device into a nonvolatile persistent mode, and creating a namespace in the persistent memory device; in response to receiving a request for applying for memory resources by a thread, judging whether the thread is a thread of a persistent index file type; judging whether the state of the persistent memory device is healthy or not in response to the fact that the thread is of the persistent index file type; and responding to the state health of the persistent memory device, providing persistent memory storage type memory resources for the thread, and allocating the index cache memory in the name space according to the size of the index file. The invention utilizes the persistent memory device to replace the traditional disk as the storage medium for index persistence, and improves the performance of inquiring or retrieving data and the efficiency of reading and writing the index file through the name space established in the persistent memory device.
Description
Technical Field
The present invention relates to the field of persistent memory, and more particularly, to a method, system, computer device, and readable medium for optimizing index persistence.
Background
The index is an independent storage structure for sequencing values of one or more fields in the data table, is very important for efficient operation of a relational database, and can greatly improve query or retrieval efficiency by reasonably using the index. The index persistence is to store the index file in the memory in a permanent storage medium, so as to avoid the loss of the index file after power failure, save the time consumption caused by newly building an index after downtime restart, and improve the query or retrieval performance. Index persistence is usually to store the serialized index file in a permanent storage medium such as a disk, and this process will generate a large number of disk IO; in addition, when data is queried or retrieved by using an index, a persistent index file needs to be written into a memory through operations such as deserialization, and a large number of disk IO are also generated in the process.
Generally, the size of the index file occupies about 8% of the space of the associated data table, and the space occupied by the index file cannot be ignored in the task of querying or retrieving massive data. At this time, the disk IO generated by reading and writing the persistent index file will affect the performance of querying or retrieving data.
Disclosure of Invention
In view of this, an object of the embodiments of the present invention is to provide a method, a system, a computer device, and a computer-readable storage medium for optimizing index persistence, in which a persistent memory device replaces a conventional disk as a storage medium for index persistence, and a namespace created in the persistent memory device is used to improve performance of querying or retrieving data and improve efficiency of reading and writing an index file.
Based on the above object, an aspect of the embodiments of the present invention provides a method for optimizing index persistence, including the following steps: initializing persistent memory equipment into a nonvolatile persistent mode, and creating a namespace in the persistent memory equipment; in response to receiving a request for applying for memory resources by a thread, judging whether the thread is a thread of a persistent index file type; responding to the thread of the type of the persistent index file, and judging whether the state of the persistent memory device is healthy or not; and responding to the state health of the persistent memory equipment, providing persistent memory storage type memory resources for the thread, and allocating the index cache memory in the namespace according to the size of the index file.
In some embodiments, said providing persistent memory storage memory resources to said thread comprises: and reconstructing a memory storage engine and an index cache memory of the relational database.
In some embodiments, said providing persistent memory storage memory resources to said thread comprises: and rewriting the pointing address of the cache index file in the memory storage engine, and adding a hierarchy of the persistent memory storage.
In some embodiments, said providing persistent memory storage memory resources to said thread comprises: and pointing the storage mode of the cache index file to the persistent memory device.
In some embodiments, the method further comprises: in response to the thread not being a thread of a persistent index file type, determining whether the thread is a thread that queries or retrieves a data type; and responding to the thread which inquires or retrieves the data type, and allocating the Dynamic Random Access Memory (DRAM) operation memory resource for the thread.
In some embodiments, said allocating dram run memory resources for said threads comprises: and allocating different types of used memory according to the types of the query or retrieval data statements.
In some embodiments, said creating a namespace in said persistent memory device comprises: and acquiring the actual capacity of the persistent memory device, and creating a plurality of namespaces which have the same size and are isolated from each other according to the actual capacity.
In another aspect of the embodiments of the present invention, there is also provided an optimized index persistence system, including: the control module is configured to initialize a persistent memory device to a nonvolatile persistent mode and create a namespace in the persistent memory device; the first judgment module is configured to respond to a received request for applying for memory resources by a thread and judge whether the thread is a thread of a persistent index file type; the second judgment module is configured to respond to the thread being of the persistent index file type, and judge whether the state of the persistent memory device is healthy; and the persistent module is configured to respond to the state health of the persistent memory device, provide persistent memory storage type memory resources to the thread, and allocate the index cache memory in the namespace according to the size of the index file.
In another aspect of the embodiments of the present invention, there is also provided a computer device, including: at least one processor; and a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method as above.
In a further aspect of the embodiments of the present invention, a computer-readable storage medium is also provided, in which a computer program for implementing the above method steps is stored when the computer program is executed by a processor.
The invention has the following beneficial technical effects: the persistent memory device replaces a traditional disk to serve as a storage medium for index persistence, and the naming space created in the persistent memory device is used for improving the performance of inquiring or retrieving data and improving the efficiency of reading and writing index files.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
FIG. 1 is a diagram illustrating an embodiment of a method for optimizing index persistence according to the present invention;
FIG. 2 is a schematic hardware structure diagram of an embodiment of a computer apparatus for optimizing index persistence according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
In view of the above objects, a first aspect of the embodiments of the present invention proposes an embodiment of a method for optimizing index persistence. FIG. 1 is a diagram illustrating an embodiment of a method for optimizing index persistence according to the present invention. As shown in fig. 1, the embodiment of the present invention includes the following steps:
s1, initializing the persistent memory device into a nonvolatile persistent mode, and creating a namespace in the persistent memory device;
s2, responding to the received request of the thread for applying for the memory resource, and judging whether the thread is a thread of a persistent index file type;
s3, responding to the thread with the persistent index file type, and judging whether the state of the persistent memory device is healthy or not; and
and S4, responding to the state health of the persistent memory device, providing the persistent memory storage type memory resource for the thread, and allocating the index cache memory in the name space according to the size of the index file.
In this embodiment, a persistent memory device is used as a storage medium of a persistent index file. Unlike a conventional disk-type persistent storage device, the persistent memory device is installed in a server memory slot, and the processor accesses the persistent memory device in a byte-addressing manner. The persistent Memory device has high capacity, low cost, and persistent characteristics compared to a DRAM (Dynamic Random Access Memory), and has high performance and low latency compared to a magnetic disk.
Currently, the persistent memory device has three capacities of 128GB, 256GB and 512GB, and the operating system of this embodiment uses the persistent memory device as a "disk". When the persistent memory device is used as a disk, the processor can access the persistent memory device only by means of the related development library, and compared with a common disk, the persistent memory device has higher IO performance and lower delay while the persistent characteristic is guaranteed, so that performance optimization of a read-write persistent index stage is realized.
The persistent memory device is initialized to a non-volatile persistent mode and a namespace is created in the persistent memory device. When initializing the persistent memory device, the persistent memory device may be initialized to an APP Direct (non-volatile persistent) mode by using an ipmctl create-good persistent memorytype ═ APP Direct command, where the persistent memory device has a persistent characteristic similar to a disk and IO performance and latency are better than the disk. The Namespace command may be utilized to create a Namespace. After the steps, the configuration of the persistent memory device can be completed, and a precondition is provided for the optimization of the index persistence in the next step.
In some embodiments, said creating a namespace in said persistent memory device comprises: and acquiring the actual capacity of the persistent memory device, and creating a plurality of namespaces which have the same size and are isolated from each other according to the actual capacity. When Namespace is created, the size of each Namespace needs to be specified, a piece of persistent memory equipment can create a plurality of Namespaces, each Namespace is isolated from each other, but the sum of the Namespaces is smaller than or equal to the actual capacity of the persistent memory equipment.
And responding to the received request of the thread for applying for the memory resource, and judging whether the thread is a thread of a persistent index file type. All threads need to apply for memory resources from the memory management and optimization module, and the memory management and optimization module allocates corresponding memory resources to the threads according to the types of the threads.
And judging whether the state of the persistent memory device is healthy or not in response to the fact that the thread is a thread of the type of the persistent index file. The status of the persistent memory device can be queried by using an ipmctl show-dim command, and when the status field shows that the status of the persistent memory device is not health, it indicates that a certain problem exists in the persistent memory device, and errors need to be eliminated according to specific information until the status is available for use in the health.
And responding to the state health of the persistent memory device, providing persistent memory storage type memory resources for the thread, and allocating the index cache memory in the name space according to the size of the index file.
In some embodiments, said providing persistent memory storage memory resources to said thread comprises: and reconstructing a memory storage engine and an index cache memory of the relational database.
In some embodiments, said providing persistent memory storage memory resources to said thread comprises: and rewriting the pointing address of the cache index file in the memory storage engine, and adding a hierarchy of the persistent memory storage.
In some embodiments, said providing persistent memory storage memory resources to said thread comprises: and pointing the storage mode of the cache index file to the persistent memory device.
When the operating system judges that the current thread is a thread of a persistent index file type, the memory management and optimization module provides a persistent memory storage type memory resource for the thread to realize index persistence optimization. Specifically, the memory management and optimization module reconstructs a memory storage engine and an index cache memory of the relational database, rewrites a pointing address of a cache index file in the memory storage engine, adds a persistent memory storage level, and points a storage mode of the cache index file to persistent memory equipment; meanwhile, the memory management and optimization module reasonably distributes the index cache memory on the Namespace of the persistent memory device according to the size of the index file, and finally realizes the persistence of the index file based on the characteristic of the persistent memory device in the APP Direct mode.
The embodiment of the invention provides an Application Programming Interface (API) and a tool related to accessing Namespace. For example, the pmemobj development library contains APIs for transaction object storage, memory allocation, transactions, and general programming tools, the pmemkv development library contains APIs for embedded key-value data storage, and the pmemopol development library contains APIs for creating persistent memory pool files. By utilizing the API, the traditional links such as a file system, page cache, drive and the like can be bypassed in the index persistence stage, and the efficiency of accessing persistent memory equipment is improved to a certain extent, so that the performance of inquiring or retrieving data is improved.
In some embodiments, the method further comprises: in response to the thread not being a thread of a persistent index file type, determining whether the thread is a thread that queries or retrieves a data type; and responding to the thread which inquires or retrieves the data type, and allocating the Dynamic Random Access Memory (DRAM) operation memory resource for the thread.
In some embodiments, the allocating dram run memory resources for the thread comprises: and allocating different types of used memories according to the types of the query or retrieval data statements.
When the operating system judges that the current thread is a thread for inquiring or retrieving the data type, the memory management and optimization module allocates DRAM (dynamic random access memory) operation memory resources for the thread to ensure the normal operation of the task. Specifically, the operating memories of DRAM types, such as a sequential read data cache region using memory, a random read data cache region using memory, a sequential use memory, a connection use memory, a temporary table use memory, and the like, are allocated reasonably according to the type of the query or search data statement.
The invention discloses an index persistence optimization method based on persistent memory, which utilizes persistent memory equipment as a storage medium for index persistence to replace a traditional disk as the index persistence scheme of the storage medium. The advantages of this embodiment are as follows: 1) the persistent memory transmits data through a memory bus, and has higher IO performance and lower delay compared with a common disk by adopting a Load/Store instruction set structure which is the same as that of a DRAM (dynamic random access memory), so that disk overhead is avoided when a persistent index file is read and written, and the performance of inquiring or retrieving data is improved; 2) the persistent memory adopts a byte addressing mode, a data structure in the common memory is also suitable for the persistent memory, the type of the data structure (usually a tree structure) of the index file per se does not need to be changed, and the efficiency of reading and writing the index file is improved.
It should be particularly noted that, the steps in the embodiments of the above-mentioned method for optimizing index persistence may be mutually intersected, replaced, added, or deleted, and therefore, these reasonable permutation and combination transformations should also belong to the scope of the present invention, and should not limit the scope of the present invention to the embodiments.
In view of the above object, according to a second aspect of the embodiments of the present invention, there is provided a system for optimizing index persistence, including: the control module is configured to initialize a persistent memory device to a nonvolatile persistent mode and create a namespace in the persistent memory device; the first judgment module is configured to respond to a received request for applying for memory resources by a thread and judge whether the thread is a thread of a persistent index file type; the second judgment module is configured to respond to the thread being of the persistent index file type, and judge whether the state of the persistent memory device is healthy; and the persistent module is configured to respond to the state health of the persistent memory device, provide persistent memory storage type memory resources to the thread, and allocate the index cache memory in the namespace according to the size of the index file.
In some embodiments, the persistence module is configured to: and reconstructing a memory storage engine and an index cache memory of the relational database.
In some embodiments, the persistence module is configured to: and rewriting the pointing address of the cache index file in the memory storage engine, and adding a hierarchy of the persistent memory storage.
In some embodiments, the persistence module is configured to: and pointing the storage mode of the cache index file to the persistent memory device.
In some embodiments, the system further comprises: the third judging module is configured to respond to the thread which is not of the persistent index file type, and judge whether the thread is a thread for inquiring or retrieving the data type; and responding to the thread which inquires or retrieves the data type, and allocating the Dynamic Random Access Memory (DRAM) operation memory resource for the thread.
In some embodiments, the third determining module is configured to: and allocating different types of used memory according to the types of the query or retrieval data statements.
In some embodiments, the control module is configured to: and acquiring the actual capacity of the persistent memory device, and creating a plurality of namespaces which have the same size and are isolated from each other according to the actual capacity.
In view of the above object, a third aspect of the embodiments of the present invention provides a computer device, including: at least one processor; and a memory storing computer instructions executable on the processor, the instructions being executable by the processor to perform the steps of: s1, initializing the persistent memory device into a nonvolatile persistent mode, and creating a namespace in the persistent memory device; s2, responding to the request of the thread for applying the memory resource, and judging whether the thread is a thread of a persistent index file type; s3, responding to the thread with the type of the persistent index file, and judging whether the state of the persistent memory device is healthy; and S4, responding to the state health of the persistent memory device, providing the persistent memory storage type memory resource for the thread, and allocating the index cache memory in the namespace according to the size of the index file.
In some embodiments, said providing persistent memory storage memory resources to said thread comprises: and reconstructing a memory storage engine and an index cache memory of the relational database.
In some embodiments, said providing persistent memory storage memory resources to said thread comprises: and rewriting the pointing address of the cache index file in the memory storage engine, and adding a hierarchy of the persistent memory storage.
In some embodiments, said providing persistent memory storage memory resources to said thread comprises: and pointing the storage mode of the cache index file to the persistent memory device.
In some embodiments, the steps further comprise: in response to the thread not being a thread of a persistent index file type, determining whether the thread is a thread that queries or retrieves a data type; and responding to the thread which inquires or retrieves the data type, allocating the dynamic random access memory operation memory resource for the thread.
In some embodiments, the allocating dram run memory resources for the thread comprises: and allocating different types of used memory according to the types of the query or retrieval data statements.
In some embodiments, said creating a namespace in said persistent memory device comprises: and acquiring the actual capacity of the persistent memory device, and creating a plurality of namespaces which have the same size and are isolated from each other according to the actual capacity.
Fig. 2 is a schematic hardware structural diagram of an embodiment of the computer device for optimizing index persistence according to the present invention.
Taking the apparatus shown in fig. 2 as an example, the apparatus includes a processor 301 and a memory 302, and may further include: an input device 303 and an output device 304.
The processor 301, the memory 302, the input device 303 and the output device 304 may be connected by a bus or other means, and fig. 2 illustrates the connection by a bus as an example.
The memory 302, which is a non-volatile computer-readable storage medium, may be used for storing non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the method for optimizing index persistence in the embodiments of the present application. The processor 301 executes various functional applications of the server and data processing by running nonvolatile software programs, instructions and modules stored in the memory 302, i.e. implements the method for optimizing index persistence of the above-described method embodiment.
The memory 302 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the method of optimizing index persistence, and the like. Further, the memory 302 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 302 optionally includes memory located remotely from processor 301, which may be connected to a local module via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 303 may receive information such as a user name and a password that are input. The output means 304 may comprise a display device such as a display screen.
Program instructions/modules corresponding to one or more methods of optimizing index persistence are stored in the memory 302 and, when executed by the processor 301, perform the method of optimizing index persistence in any of the method embodiments described above.
Any embodiment of a computer device that performs the above-described method for optimizing index persistence may achieve the same or similar effects as any of the preceding method embodiments corresponding thereto.
The invention also provides a computer readable storage medium storing a computer program which, when executed by a processor, performs the method as above.
Finally, it should be noted that, as one of ordinary skill in the art can appreciate that all or part of the processes of the methods of the above embodiments can be implemented by a computer program to instruct related hardware, and the program of the method for optimizing index persistence can be stored in a computer readable storage medium, and when executed, can include the processes of the embodiments of the methods as described above. The storage medium of the program may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
The foregoing are exemplary embodiments of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (8)
1. A method for optimizing index persistence, comprising the steps of:
initializing persistent memory equipment into a nonvolatile persistent mode, and creating a namespace in the persistent memory equipment;
responding to a request of a thread for applying for memory resources, and judging whether the thread is a thread of a persistent index file type;
responding to the thread of the type of the persistent index file, and judging whether the state of the persistent memory device is healthy or not; and
providing persistent memory storage type memory resources to the thread in response to the state health of the persistent memory device, and allocating index cache memory in the namespace according to the size of an index file,
the method further comprises the following steps:
in response to the thread not being a thread of a persistent index file type, determining whether the thread is a thread that queries or retrieves a data type; and
in response to the thread being a thread that queries or retrieves a data type, allocating Dynamic Random Access Memory (DRAM) run memory resources for the thread,
wherein the creating a namespace in the persistent memory device comprises:
and acquiring the actual capacity of the persistent memory device, and creating a plurality of namespaces which have the same size and are isolated from each other according to the actual capacity.
2. The method of claim 1, wherein providing persistent memory storage memory resources to the thread comprises:
and reconstructing a memory storage engine and an index cache memory of the relational database.
3. The method of claim 1, wherein providing persistent memory storage memory resources to the thread comprises:
and rewriting the pointing address of the cache index file in the memory storage engine, and adding a hierarchy of the persistent memory storage.
4. The method of claim 1, wherein providing persistent memory storage memory resources to the thread comprises:
and pointing the storage mode of the cache index file to the persistent memory device.
5. The method of claim 1, wherein allocating dram run memory resources for the thread comprises:
and allocating different types of used memory according to the types of the query or retrieval data statements.
6. A system for optimizing index persistence, comprising:
the control module is configured to initialize a persistent memory device to a nonvolatile persistent mode and create a namespace in the persistent memory device;
the first judgment module is configured to respond to a received request for applying for memory resources by a thread and judge whether the thread is a thread of a persistent index file type;
the second judgment module is configured to respond to the thread being of the persistent index file type, and judge whether the state of the persistent memory device is healthy; and
a persistent module configured to provide persistent memory storage type memory resources to the thread in response to the state health of the persistent memory device, and allocate an index cache memory in the namespace according to a size of an index file,
the system further comprises a third determination module configured to: in response to the thread not being a thread of a persistent index file type, determining whether the thread is a thread that queries or retrieves a data type; and in response to the thread being a thread that queries or retrieves a data type, allocating Dynamic Random Access Memory (DRAM) operating memory resources for the thread,
wherein the creating a namespace in the persistent memory device comprises:
and acquiring the actual capacity of the persistent memory device, and creating a plurality of namespaces which have the same size and are isolated from each other according to the actual capacity.
7. A computer device, comprising:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method of any one of claims 1 to 5.
8. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011181128.4A CN112434025B (en) | 2020-10-29 | 2020-10-29 | Method, system, device and medium for optimizing index persistence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011181128.4A CN112434025B (en) | 2020-10-29 | 2020-10-29 | Method, system, device and medium for optimizing index persistence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112434025A CN112434025A (en) | 2021-03-02 |
| CN112434025B true CN112434025B (en) | 2022-08-12 |
Family
ID=74696472
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011181128.4A Active CN112434025B (en) | 2020-10-29 | 2020-10-29 | Method, system, device and medium for optimizing index persistence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112434025B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113590536B (en) * | 2021-05-20 | 2023-12-29 | 济南浪潮数据技术有限公司 | Data storage method, system, electronic equipment and storage medium |
| CN113254427B (en) * | 2021-07-15 | 2021-11-16 | 深圳市同富信息技术有限公司 | Database expansion method and device |
| CN113779089A (en) * | 2021-09-14 | 2021-12-10 | 杭州沃趣科技股份有限公司 | Method, device, equipment and medium for maintaining database hotspot data |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160170645A1 (en) * | 2014-12-11 | 2016-06-16 | Intel Corporation | Computing method and apparatus with persistent memory |
| CN109885573A (en) * | 2019-02-22 | 2019-06-14 | 广州荔支网络技术有限公司 | A kind of maintaining method of data-storage system, device and mobile terminal |
-
2020
- 2020-10-29 CN CN202011181128.4A patent/CN112434025B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160170645A1 (en) * | 2014-12-11 | 2016-06-16 | Intel Corporation | Computing method and apparatus with persistent memory |
| CN109885573A (en) * | 2019-02-22 | 2019-06-14 | 广州荔支网络技术有限公司 | A kind of maintaining method of data-storage system, device and mobile terminal |
Non-Patent Citations (2)
| Title |
|---|
| 《A Protected Block Device for Persistent Memory》;Feng Chen 等;《2014 30th Symposium on Mass Storage Systems and Technologies (MSST)》;20141231;全文 * |
| 《一种高效的持久性内存管理系统设计》;余海涛;《无线互联科技》;20191031;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112434025A (en) | 2021-03-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112434025B (en) | Method, system, device and medium for optimizing index persistence | |
| CN104346294A (en) | Data reading/writing method and device and computer system on basis of multi-level Cache | |
| CN106557427B (en) | Memory management method and device for shared memory database | |
| CN111858509B (en) | Distributed computing method and device based on container | |
| TWI764265B (en) | Memory system for binding data to a memory namespace | |
| CN107615243B (en) | Method, device and system for calling operating system library | |
| CN108845958B (en) | System and method for interleaver mapping and dynamic memory management | |
| CN111930643B (en) | Data processing method and related equipment | |
| CN102609305A (en) | Method for sharing internal memory in server cluster | |
| WO2024078429A1 (en) | Memory management method and apparatus, computer device, and storage medium | |
| JPH10333949A (en) | Data storage control system | |
| CN115794669A (en) | Method, device and related equipment for expanding memory | |
| CN115904212A (en) | Data processing method and device, processor and hybrid memory system | |
| CN115543224B (en) | ZNS SSD-based file system control method, device and equipment | |
| US7350210B2 (en) | Generic data persistence application program interface | |
| CN112948336B (en) | Data acceleration method, cache unit, electronic device and storage medium | |
| EP3752924A1 (en) | Central processing unit cache friendly multithreaded allocation | |
| CN115712500A (en) | Memory release method, memory recovery method, memory release device, memory recovery device, computer equipment and storage medium | |
| CN110673791B (en) | Metadata refreshing method, device and equipment and readable storage medium | |
| CN104283909A (en) | Cloud computing method and device compatible with desktop applications | |
| CN107066208B (en) | Asymmetric read-write method of external memory device and NVM (non-volatile memory) external memory device | |
| US8756193B2 (en) | System and method for persisting object pointers | |
| KR102565873B1 (en) | Method for allocating memory bus connected storage in numa system | |
| US20230140158A1 (en) | Integrated circuit and configuration method thereof | |
| EP4322001A1 (en) | Memory expansion method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |