CN116126208A

CN116126208A - Storage object index storage method and device, medium and electronic equipment

Info

Publication number: CN116126208A
Application number: CN202111341163.2A
Authority: CN
Inventors: 胡杰; 张明; 肖凯西; 张瑾瑜; 韩建勋; 解文斌; 梁天健; 王悦; 李华
Original assignee: China Telecom Corp Ltd
Current assignee: China Telecom Corp Ltd
Priority date: 2021-11-12
Filing date: 2021-11-12
Publication date: 2023-05-16

Abstract

The disclosure provides an index storage method, device, medium and equipment for storage objects, and relates to the technical field of computer storage. The index storage method of the storage object comprises the following steps: acquiring parameters of an index positioned at a first storage position; storing the index of the first storage position meeting the preset parameter condition into a second storage position according to the parameter of the index of the first storage position; when the second storage position is in a triggering state, acquiring parameters of an index positioned in the second storage position; and returning the index of the second storage position meeting the preset elimination condition to the first storage position according to the parameter of the index of the second storage position. The method and the device can realize accurate classified storage of the index and improve index retrieval efficiency.

Description

Storage object index storage method and device, medium and electronic equipment

Technical Field

The present disclosure relates to the field of computer storage technologies, and in particular, to a storage object index storage method, a storage object index storage device, a computer readable medium, and an electronic device.

Background

With the rapid increase of data volume in the big data age, huge data cannot be stored in the jurisdiction of one operating system, and the data volume needs to be distributed to more disks managed by the operating system.

The currently adopted object storage is a cloud storage service based on a large-scale distributed high concurrency storage framework, and indexes are stored in a metadata pool. When the data amount of the accompanying storage object increases, the index access time of each time is relatively long, so that the index data query efficiency is low.

It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Disclosure of Invention

The embodiment of the disclosure aims to provide an index storage method, device, medium and electronic equipment for storage objects, which can realize accurate classified storage of indexes and improve index retrieval efficiency.

A first aspect of an embodiment of the present disclosure provides an index storing method for a storage object, including:

acquiring parameters of an index positioned at a first storage position;

storing the index of the first storage position meeting the preset parameter condition into a second storage position according to the parameter of the index of the first storage position;

when the second storage position is in a triggering state, acquiring parameters of an index positioned in the second storage position;

And returning the index of the second storage position meeting the preset elimination condition to the first storage position according to the parameter of the index of the second storage position.

In an exemplary embodiment of the present disclosure, the parameters of the index include: access parameters, weight parameters.

In an exemplary embodiment of the disclosure, the storing the index of the first storage location meeting the preset parameter condition to the second storage location according to the parameter of the index of the first storage location includes:

comparing the index parameter of the first storage location with a plurality of preset sub-parameter conditions, the plurality of preset sub-parameter conditions comprising: access conditions, weight conditions;

and storing the index of the sub-parameter condition meeting the access condition and/or the weight condition in a second storage position.

In an exemplary embodiment of the disclosure, the returning, according to the parameter of the index of the second storage location, the index of the second storage location meeting the preset elimination condition to the first storage location includes:

comparing the parameter of the index of the second storage location with a preset elimination condition, wherein the preset elimination condition comprises: eliminating access conditions and eliminating weight conditions;

And returning the index meeting the elimination access condition and/or the elimination weight condition to the first storage position.

In an exemplary embodiment of the present disclosure, after the step of storing the index of the first storage location meeting the preset parameter condition in the second storage location according to the parameter of the index of the first storage location, the method further includes:

setting a first storage table and a second storage table in the second storage position;

an active index is stored in the first storage table and an inactive index is stored in the second storage table.

In an exemplary embodiment of the present disclosure, the storing the active index in the first storage table and the inactive index in the second storage table includes:

determining the index as an active index or an inactive index according to the index parameters;

and storing the active index into the first storage table, and storing the inactive index into the second storage table.

In an exemplary embodiment of the present disclosure, after the step of storing the index into the corresponding storage table, the method further includes:

acquiring an update parameter of the index after a period of time, and re-determining the index as an active index or an inactive index according to the update parameter;

And storing the active index or the inactive index update into a corresponding storage table.

when the second storage table of the second storage position is empty, randomly eliminating the index of the first storage table; otherwise the first set of parameters is selected,

and eliminating the index in the second storage table of the second storage position based on a preset elimination condition.

In an exemplary embodiment of the present disclosure, after the step of returning the index of the second storage location meeting the preset elimination condition to the first storage location according to the parameter of the index of the second storage location, the method further includes:

and detecting whether the second storage position is in a trigger state, if so, continuously acquiring parameters of the index positioned in the second storage position, and returning the index of the second storage position meeting the preset elimination condition to the first storage position.

According to a second aspect of embodiments of the present disclosure, there is provided an index storage device storing objects, including:

the first parameter acquisition module is used for acquiring parameters of the index positioned at the first storage position;

The index storage module is used for storing the index of the first storage position meeting the preset parameter condition to the second storage position according to the parameter of the index of the first storage position;

the second parameter acquisition module is used for acquiring parameters of the index positioned at the second storage position when the second storage position is in a trigger state;

and the index returning module is used for returning the index of the second storage position meeting the preset elimination condition to the first storage position according to the parameter of the index of the second storage position.

In an exemplary embodiment of the present disclosure, the index storing module is configured to store, according to a parameter of the index of the first storage location, the index of the first storage location meeting a preset parameter condition to a second storage location, including:

In an exemplary embodiment of the present disclosure, the returning module is configured to return, according to a parameter of the index of the second storage location, the index of the second storage location meeting a preset elimination condition to the first storage location, including:

In one exemplary embodiment of the present disclosure, the index storage device storing the object further includes: and the sub-table storage module. The sub-table storage module is used for setting a first storage table and a second storage table in the second storage position;

In an exemplary embodiment of the present disclosure, the sub-table storage module is configured to store an active index in the first storage table and an inactive index in the second storage table, including:

In an exemplary embodiment of the present disclosure, the apparatus further comprises: the sub-table storage module is used for storing the index into a corresponding storage table, and further comprises:

In an exemplary embodiment of the present disclosure, the index returning module is configured to return, according to a parameter of the index of the second storage location, the index of the second storage location meeting a preset elimination condition to the first storage location, including:

In one exemplary embodiment of the present disclosure, the apparatus further comprises: and (5) circularly eliminating the module. The loop elimination module is used for returning the index of the second storage position meeting the preset elimination condition to the first storage position according to the parameter of the index of the second storage position, and further comprises:

According to a third aspect of the embodiments of the present disclosure, there is provided a computer readable medium having stored thereon a computer program, which when executed by a processor implements the index storage method of a storage object as described in the first aspect of the above embodiments.

According to a fourth aspect of embodiments of the present disclosure, there is provided an electronic device, comprising: one or more processors; and a storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the method for storing an index of a storage object according to the first aspect of the embodiment.

The technical scheme provided by the embodiment of the disclosure can comprise the following beneficial effects:

in the technical solutions provided in some embodiments of the present disclosure, parameters of the first storage location index may be obtained, and whether parameter conditions are met is determined to change the storage location to the second storage location; and when the second storage position is in the triggering state, acquiring the parameters of the second storage position, and returning the index of the second storage position meeting the preset elimination condition to the first storage position. In one aspect, embodiments of the present disclosure are implemented to accurately store an index to a corresponding location by setting different conditions to change the storage location of the corresponding index. On the other hand, after the storage locations of the indexes are distinguished, the efficiency of index retrieval can be improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort.

FIG. 1 schematically illustrates an exemplary scenario of an index storage method and apparatus for storing objects to which embodiments of the present disclosure may be applied;

FIG. 2 schematically illustrates a flow diagram of a method of index-keeping of a storage object according to one embodiment of the disclosure;

FIG. 3 schematically illustrates a flow diagram of a method of index-keeping of a storage object according to one embodiment of the disclosure;

FIG. 4 schematically illustrates a flow diagram of a method of index-keeping of a storage object according to one embodiment of the disclosure;

FIG. 5 schematically illustrates a flow diagram of a method of index-keeping of a storage object according to one embodiment of the disclosure;

FIG. 6 schematically illustrates a flow diagram of a method of index-keeping of a storage object according to one embodiment of the disclosure;

FIG. 7 schematically illustrates a flow diagram of a process for eliminating a second storage location index in one embodiment in accordance with the disclosure;

FIG. 8 schematically illustrates a block diagram of an index store storing objects in accordance with one embodiment of the present disclosure

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present disclosure. One skilled in the relevant art will recognize, however, that the aspects of the disclosure may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.

FIG. 1 schematically illustrates a schematic diagram of an exemplary system architecture of a storage object index storage method and apparatus to which embodiments of the present disclosure may be applied.

As shown in fig. 1, the system architecture may include an object storage cluster 101, a Solid State Disk 102 (Solid State Disk or Solid State Drive, abbreviated as SSD), and a dynamic random access memory 103 (Dynamic Random Access Memory, abbreviated as DRAM), i.e., a memory 103. The Object storage cluster 101 is an abstract storage cluster defined in a Ceph distributed file system to serve as the bottom layer of the Ceph, the Object storage cluster 101 is a storage cluster formed by a plurality of host organizations, and each host may have a plurality of Object storage devices (objects-based Storage Device, abbreviated as OSD), and a single OSD is used as a disk for storing data. The solid state disk 102 is a hard disk made of a solid state electronic memory chip array, and includes a solid state disk using FLASH memory (FLASH chip) or DRAM as a storage medium, and a solid state disk based on XPoint particles, and a metadata resource pool is formed by using the hard disk (SSD). The dynamic random access memory 103 can be read and written at any time at a relatively high speed, and can be used as a temporary data storage medium for an operating system or other running programs.

In particular, according to embodiments of the present disclosure, the processes described below with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via a communication portion, and/or installed from a removable medium. The computer program, when executed by a Central Processing Unit (CPU), performs the various functions defined in the methods and apparatus of the present disclosure.

It should be noted that the computer readable medium shown in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware, and the described units may also be provided in a processor. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.

To facilitate an understanding of the disclosed embodiments, the following explains some of the scenarios to which the present disclosure pertains:

the object storage is a cloud storage service based on a large-scale distributed high concurrency storage framework, namely key value storage. Unlike conventional file storage, the object storage is to separate Metadata, and its control node is called Metadata Server (MDS) and is mainly responsible for attribute management of the stored objects. The portion of the object store that is mainly responsible for storing file data is referred to as an object store (Object Storage Device, abbreviated OSD). When the user accesses the object content, the MDS metadata server is accessed to acquire related information, such as the requested object is stored in which OSD, and further the user accesses the specific OSD to acquire the read-write operation of the object content.

Referring to fig. 2, fig. 2 schematically illustrates a flowchart of a method of index-keeping of a storage object according to one embodiment of the present disclosure. As shown in fig. 2, the method for storing the index of the storage object may include:

s210, acquiring parameters of an index located in a first storage position.

S220, storing the index of the first storage position meeting the preset parameter condition into a second storage position according to the parameter of the index of the first storage position.

S230, when the second storage position is in the trigger state, acquiring parameters of the index located in the second storage position.

S240, returning the index of the second storage position meeting the preset elimination condition to the first storage position according to the parameter of the index of the second storage position.

By implementing the storage object index storage method shown in fig. 2, parameters of the first storage position index can be acquired, and whether parameter conditions are met or not is judged so as to change the storage position to the second storage position; and when the second storage position is in the triggering state, acquiring the parameters of the second storage position, and returning the index of the second storage position meeting the preset elimination condition to the first storage position. By implementing the embodiment of the disclosure, on one hand, the storage position of the corresponding index is changed by setting different conditions, so that the index storage can be more accurately stored in the corresponding storage position. On the other hand, after the storage locations of the indexes are distinguished, the efficiency of index retrieval can be improved.

The above steps for the present exemplary embodiment are described in more detail below.

In step S210, parameters of an index located at a first storage location are acquired.

In the embodiment of the disclosure, the first storage location may be a memory, further may be a memory, also referred to as a main memory, for example, the first storage location may be a random access memory RAM and a read only memory ROM; the first storage location may be a hard disk, an optical disc, a commonly used USB (Universal Serial Bus, abbreviated as USB), or the like. The memory capacity of the external memory is large, the memory capacity of the internal memory is limited and is much smaller than the external memory, but the information in the external memory must be read into the internal memory before being called or processed by the central processing unit (central processing unit, CPU for short). An index is another way to represent data, with the index providing an order of data that is different from the physical storage order of the data on disk. The special purpose of the index is to rearrange the physical location of the records within the table, which can speed up the search for data.

Optionally, the first storage location adopts a solid state disk to store Index data, and utilizes a plurality of high-performance Solid State Disks (SSDs) to form a metadata resource pool, so that Index data (Bucket Index) is migrated from the SATA hard disk with low access read-write speed to the high-performance solid state disk. The storage speed of SSD, NMVE and other solid state disks is very fast, the access read-write speed of index can be accelerated, and the solid state disk is a nonvolatile memory, so even if power is off, data cached on the nonvolatile memory can not be lost. While also maintaining lower costs.

In the embodiment of the disclosure, the parameters of the index located in the first storage location are obtained, and the parameters may include access conditions of the index, such as the number of accessed times, access time, access frequency, and index weight value. According to different application scenarios, the content of the index parameter can be configured by itself, and the embodiment of the disclosure is not limited herein.

For example, in the process of obtaining the access times and access times of the indexes, a very short time (close to 0) can be set for the slow log of the indexes, then all search operations are recorded by the slow log, then the information of what time each index is accessed can be obtained by analyzing the slow log, and the access times can be counted. The disclosed embodiments herein do not impose particular restrictions on how the index obtains access.

In step S220, the index of the first storage location meeting the preset parameter condition is stored in the second storage location according to the parameter of the index of the first storage location.

In the embodiment of the present disclosure, the selection of the second storage location memory may be determined within the range of the first storage location, which is not described herein. For example, the second memory location may be selected as a dynamic random access memory DRAM, so that DRAM performance can be fully and reasonably utilized. The memory transfer rate of the second memory location is higher than the transfer rate of the first memory location memory.

Optionally, the parameters of the index may include access time, access times, access frequency and weight value, and the corresponding preset condition is a parameter-related condition. For example, the preset parameter condition may be that the access time is within one month from the current time to the previous time, the number of accesses is greater than ten, the access frequency is at least once every two days, and the weight value is at least five. And storing the index meeting the preset parameter condition in a second storage position. The embodiments of the present disclosure are not specifically limited herein to specific preset parameter conditions.

In step S230, when the second storage location is in the triggered state, the parameter of the index located in the second storage location is acquired.

The second storage location is used as a memory, the storage capacity of the second storage location is limited, and when the capacity utilization ratio approaches to the upper limit, the index which meets the condition and is positioned at the first storage location cannot be stored in the second storage location. In addition, a large number of indexes are stored in the first storage position and the second storage position, and the indexes stored in the second storage position can not always meet the preset parameter condition. When the index of the second storage position does not meet the preset parameter condition, the index is considered to occupy the storage space of the second storage position and needs to be moved out.

In the embodiment of the disclosure, the second storage position is in a trigger state, and a threshold value can be set for the capacity use proportion of the second storage position, and the threshold value can be manually set by people; or may be automatically set according to the specific capacity of the second storage location. The threshold may be a fixed value or may be a value that changes dynamically during use. In addition, the second storage location is in a trigger state, or may be in a trigger state directly caused by manual operation. The disclosed embodiments herein do not impose particular limitations on the trigger state.

In step S240, the index of the second storage location meeting the preset elimination condition is returned to the first storage location according to the parameter of the index of the second storage location.

In the embodiment of the present disclosure, the parameters of the index may include access time, access times, access frequency and weight value, and the corresponding preset condition is a parameter-related condition. The access time, access times, and access frequency may be recorded by adding a field to each index data. The weight value may be determined by the user and set for the object according to the type of the object and the importance level when the user uploads the data, or may be automatically set by the system identifying the type of the object and determining the importance level based on the content of the data.

For example, the preset elimination condition may be that the access time is not accessed within one month before the current time, the access times are less than ten times, the access frequency is less than once a day, and the weight value is less than five. And returning the index meeting the preset elimination condition to the first storage position. The disclosed embodiments are not particularly limited herein to specific preset elimination conditions.

And returning the index of the second storage position meeting the preset elimination condition to the first storage position. Alternatively, a first-in first-out algorithm (First in First out, FIFO) may be employed, a least recently used algorithm (Least Frequently Used, LFU) and a least recently unused algorithm (The Least Recently Used, LRU). For example, when the LRU algorithm is used, a data structure is designed for the LRU Cache, which supports two operations: (1) get (key): if the key is in the cache, returning a corresponding value, otherwise returning-1, (2) set (key, value), if the key is not in the cache, inserting the (key, value) into the cache; if the key is in the cache, the value of value is reset. The data structure adopted by the LRU algorithm may be to store data by using an array, mark each data item with an access time stamp, and each time a new data item is inserted, first self-increment the time stamp of the data item existing in the array, and set the time stamp of the new data item to 0 and insert the time stamp into the array. Each time a data item in the array is accessed, the timestamp of the accessed data item is set to 0. When the array space is full, the data item with the largest time stamp is eliminated. Or a linked list and a hash map may be used. When a new data item needs to be inserted, set (value): if the key exists in the hash map, firstly resetting a corresponding value, then acquiring a corresponding node cur, deleting the cur node from the linked list, and moving the node to the head of the linked list; if the key does not exist in the hash map, a node is newly built and put in the head of the linked list. When the Cache is full, the last node of the linked list is deleted. When accessing data, if the data item exists in the linked list, the node is moved to the head of the linked list, otherwise, -1 is returned. For different usage scenarios and conditions, different algorithms may be employed to return the relevant index to the first storage location, and embodiments of the present disclosure are not particularly limited herein as to how to return the index of the second storage location to the first storage location.

The index of the proper first storage position is accurately stored in the second storage position, which is necessary for distinguishing and storing the index, otherwise, the index retrieval efficiency is affected. Whether the preset parameter condition is met or not is taken as an important link for selecting the index of the proper first storage position, and the setting of the preset parameter condition also has influence on the actual effect of index retrieval efficiency.

Based on one or more of the above problems, in an embodiment of the present disclosure, a new method for storing an index of a storage object is provided. Referring to fig. 3, in the embodiment of the disclosure, according to the parameter of the index of the first storage location, the index of the first storage location meeting the preset parameter condition is stored in the second storage location, which includes the following steps:

s310, comparing the index parameter of the first storage location with a plurality of preset sub-parameter conditions, wherein the plurality of preset sub-parameter conditions comprise: access conditions, weight conditions.

In the embodiment of the present disclosure, the access condition may be a data condition related to the access index as to whether to access, access time, access number, and the like, and the weight condition may be a condition as to whether to have data related to the weight as to a weight value, a size of the weight value, and the like.

S320, storing the index of the sub-parameter condition meeting the access condition and/or the weight condition in a second storage position.

For example, the missed index of an access may be added to the second storage location after the missed index is accessed, because if the index is accessed, then the likelihood that the index is accessed again is high. The index which has the weight value but is not stored in the second storage position can be stored in the second storage position, and the weight value can be used as a consideration factor of an index data replacement strategy when the storage capacity is full by setting the weight of the object file according to experience or special requirements by a user, so that important files and special files are prevented from being replaced immediately under the condition of full cache, and the time of the important files and the special files in the second storage position is prolonged to improve the retrieval performance of the system. A period of time, which is divided into cold index data and hot index data, may be set, and the access frequency of the index data in the SSD is periodically scanned. The period is determined based on the use condition of the index data, if the index data is used frequently, the change of the hot index and the cold index is obvious, and a shorter period can be set; if the index data is used less, a shorter period is set accordingly for scanning. And simultaneously setting a threshold value of access frequency, wherein index data of the access frequency exceeds the threshold value and is set as hot index data, and the rest index data are set as cold index data. The hot index data and the index data that has not been accessed recently are placed in a second storage location.

According to the embodiment of the disclosure, the specific preset parameter conditions can be set, the relevant indexes can be accurately selected to the second storage position, and the index data meeting the preset parameter conditions can be concentrated to the second storage position, so that the indexes meeting the parameter conditions can be conveniently searched, and the searching efficiency is improved.

After the index is distinguished from the storage location according to different characteristics thereof, convenient retrieval meeting parameter conditions can be realized. However, since the parameters of the index of the second storage location are constantly changing, and there is still a large difference between the indexes of the second storage location, a further distinction is required. In addition, since the new index data is always stored in the second storage location, the capacity problem of the second storage location is caused, and then part of the index is also required to be recovered to the first storage location.

Based on one or more of the above problems, in the present exemplary embodiment, a new storage object index storing method is proposed, that is, a first storage table and a second storage table are set in the second storage position; an active index is stored in the first storage table and an inactive index is stored in the second storage table. Referring to fig. 4, in this exemplary embodiment, a new storage object index storing method may further include the steps of:

S410, setting a first storage table and a second storage table in the second storage position.

In the embodiment of the disclosure, the first storage table and the second storage table may be linked lists. A linked list is a non-contiguous, non-sequential storage structure on physical storage elements, and the logical order of data elements is achieved by the order of pointer links in the linked list. The linked list is made up of a series of nodes (each element in the linked list is called a node) that can be dynamically generated at runtime. Each node comprises two parts: one is a data field for storing data elements, the other is a pointer field for storing the address of the next node, and the linked list structure can fully utilize the memory space of the computer to realize flexible memory dynamic management. Two special linked lists are set in a second storage position, such as a DRAM memory resource, and are used for storing index data with different characteristics, namely an active linked list and an inactive linked list. The active linked list is used as a first storage table, and the inactive linked list is used as a second storage table.

S420, storing the active index in the first storage table, and storing the inactive index in the second storage table.

It will be appreciated that there are many ways to identify active indexes, and based on the characteristics of the indexes, many ways to determine active indexes and inactive indexes can be derived.

For example, the case where the index is stored in the first storage table may be: if the missed index has recently had an access record, the missed index of the access is added to the active linked list of the second storage location, because if the index is accessed, then the index is more likely to be accessed again. An index having a weight value but not stored in the second storage location may be deposited into an active linked list of the second storage location. It may be that when the index data in the inactive linked list is accessed again, it is added to the active linked list.

The case where the index is stored in the second storage table may be: when the index data of the active linked list is not accessed for a period of time, the index data is added to the inactive linked list. And placing index data which have weight and are not accessed recently into a memory, and adding the index data into an inactive index data linked list. And placing the index data which are hot and are not accessed recently into a memory, and adding the index data into an inactive index data linked list. The embodiments of the present disclosure are not particularly limited herein to the specific rules for storing tables.

In the embodiment of the disclosure, the index may be stored in the corresponding storage table by determining the index as an active index or an inactive index according to a parameter of the index. The indexes meeting the parameter conditions can be further distinguished, convenient management is carried out in different storage tables, and when the capacity of the second storage position is insufficient, part of the indexes can be directly recovered from the second storage table, so that the storage performance of the second storage position is improved, and the retrieval efficiency is improved.

The index data is used as an object, and parameters of the index are changed under different use conditions. Then, even if the first storage location and the second storage location are distinguished from the index and further, a distinction is made between the first storage table and the second storage table in the second storage location. The distinction that has been made may not be adapted to the changed state because of the change of the index parameter. If the index parameter is changing, but the storage position of the index is not changing, the original effect of distinguishing storage of the index is lost.

Based on one or more of the above problems, in the present exemplary embodiment, a new index storage method of a storage object is proposed. Referring to fig. 5, in this exemplary embodiment, after the step of storing the index into the corresponding storage table, the following steps may be further included:

s510, acquiring update parameters of the index after a period of time, and re-determining the index as an active index or an inactive index according to the update parameters.

In the embodiment of the present disclosure, the length of the "period of time" may be set to be one day, one week or other time length, and may be freely configured according to different use cases. Based on the above-mentioned judging manner of the active index and the inactive index, after the update parameter of the index after a period of time is obtained, the index is redetermined to be the active index or the inactive index according to the update parameter, so that the embodiments of the disclosure are not described herein again.

It will be appreciated that if the index type is redetermined, the index type is not changed from before, and the memory table need not be changed; if the index parameter changes to result in different index type judgment results, the index needs to be changed to other storage tables.

S520, storing the index update into a corresponding storage table.

In the embodiment of the disclosure, the index can be redetermined to be an active index or an inactive index according to update parameters of the index after a period of time by acquiring the update parameters of the index; and storing the index update into a corresponding storage table. The index type can be judged newly after a period of time, and the index storage table is correspondingly changed according to whether the index type is changed, so that the accuracy of index storage can be ensured, and the index retrieval efficiency can be ensured.

In an exemplary embodiment of the present disclosure, a new index storage method of a storage object is also presented. Referring to fig. 6, the step of returning the index of the second storage location, which meets the preset elimination condition, to the first storage location according to the parameter of the index of the second storage location includes the steps of:

S610, when the second storage table of the second storage position is empty, randomly eliminating the index of the first storage table.

It will be appreciated that the inactive index stored in the second memory table should first be eliminated from the inactive index to the first memory location when the capacity of the second memory location is too high and the retrieval performance is degraded. However, when the second storage table of the second storage location is empty, it indicates that there is no inactive index, and at this time, the active index stored in the first storage table can only be randomly eliminated to ensure the capacity of the second storage location.

S620, eliminating the index in the second storage table of the second storage position based on a preset elimination condition.

Referring to fig. 7, fig. 7 is a schematic diagram of a preset elimination condition in an embodiment of the disclosure:

after the start 701, in a condition 702, it is determined whether the inactive linked list is empty, and if the inactive linked list is empty, a condition 703 is executed. Otherwise conditional 704 is performed.

At condition 703, when the inactive linked list is empty, the most recently high load operation is displayed, and the index data in the active linked list will be randomly retired. Since there is no inactive linked list itself, the active index is a valuable index to the user, simply because too many active indexes result in an excessively high proportion of capacity usage. Therefore, it is necessary to indicate to the user that the memory resources are insufficient and that the memory resources are increased, and then condition 709 is performed.

When the inactive linked list is not empty, at condition 704, it is determined whether there is normal index data that does not contain a weight and is not a hot index, and if so, condition 705 is performed. Otherwise, condition 106 is performed.

At condition 705, index data that does not contain a weight and is not a hot index is found, a portion of such index data is preferentially randomly eliminated, and then condition 709 is performed.

At condition 706, a determination is made as to whether there is thermal index data that does not contain a weight, and if so, condition 707 is performed. Otherwise, conditional 708 is performed.

At condition 707, hot index data is found that does not contain weights, and portions of such index data will be preferentially randomly eliminated, followed by execution of condition 709.

In condition 708, index data meeting the requirements is not found, the index data with lower partial weight is preferentially eliminated, and then condition 709 is executed.

In an exemplary embodiment of the present disclosure, a condition 709 is further included to determine, after performing the elimination of the memory index data, whether the memory space is smaller than the set threshold again. If less, then the second storage location is indicated as still in the triggered state, then when the execution condition 710 ends. Otherwise, the execution of the condition 702 is continued, and the second storage location is guaranteed to be no longer in the triggered state by loop determination and elimination, so that the second storage location has sufficient remaining capacity.

In the embodiment of the disclosure, on the basis of the distinguishing management of the first storage table and the second storage table, how to reasonably eliminate the index data under the condition that the second storage position is in the trigger condition can be further refined. The method can ensure that the index stored in the first storage position is high in rationality, and is beneficial to improving the retrieval efficiency.

It should be noted that although the steps of the methods in the present disclosure are depicted in the accompanying drawings in a particular order, this does not require or imply that the steps must be performed in that particular order, or that all illustrated steps be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.

Further, in this example embodiment, an index storage device for storing objects is also provided. Referring to fig. 8, the apparatus 800 includes:

a first parameter obtaining module 810, configured to obtain a parameter of an index located in a first storage location;

an index storage module 820, configured to store, according to the parameter of the index of the first storage location, the index of the first storage location that meets a preset parameter condition to a second storage location;

A second parameter obtaining module 830, configured to obtain, when the second storage location is in the triggered state, a parameter of an index located in the second storage location;

the index returning module 840 is configured to return the index of the second storage location meeting the preset elimination condition to the first storage location according to the parameter of the index of the second storage location.

In an exemplary embodiment of the present disclosure, the index storing module 820 is configured to store, according to a parameter of the index of the first storage location, the index of the first storage location meeting a preset parameter condition to the second storage location, including:

In an exemplary embodiment of the present disclosure, the index returning module 840 is configured to return, according to a parameter of the index of the second storage location, the index of the second storage location meeting a preset elimination condition to the first storage location, including:

In one exemplary embodiment of the present disclosure, the index storage device storing the object further includes: a sub-table storage module (not shown). The sub-table storage module is used for setting a first storage table and a second storage table in the second storage position;

In an exemplary embodiment of the present disclosure, after the step of storing the index into the corresponding storage table, the sub-table storage module further includes:

In an exemplary embodiment of the present disclosure, the index returning module 820 is configured to return, according to a parameter of the index of the second storage location, the index of the second storage location meeting a preset elimination condition to the first storage location, including:

In one exemplary embodiment of the present disclosure, the apparatus further comprises: a recycle elimination module (not shown). The loop elimination module is used for returning the index of the second storage position meeting the preset elimination condition to the first storage position according to the parameter of the index of the second storage position, and further comprises:

The foregoing apparatus is used for executing the method provided in the foregoing embodiment, and its implementation principle and technical effects are similar, and are not described herein again.

The above modules may be one or more integrated circuits configured to implement the above methods, for example: one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), or one or more microprocessors (Digital Singnal Processor, abbreviated as DSP), or one or more field programmable gate arrays (Field Programmable Gate Array, abbreviated as FPGA), or the like. For another example, when a module above is implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU) or other processor that may invoke the program code. For another example, the modules may be integrated together and implemented in the form of a System-on-a-chip (SOC).

Optionally, the present invention also provides a program product, such as a computer readable storage medium, comprising a program for performing the above-described method embodiments when being executed by a processor.

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in hardware plus software functional units.

The integrated units implemented in the form of software functional units described above may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (english: processor) to perform some of the steps of the methods according to the embodiments of the invention. And the aforementioned storage medium includes: u disk, mobile hard disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.

The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes or substitutions are covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An index storage method of a storage object, comprising:

acquiring parameters of an index positioned at a first storage position;

2. The method according to claim 1, wherein storing the index of the first storage location meeting the preset parameter condition in the second storage location according to the parameter of the index of the first storage location comprises:

and storing the index meeting the access condition and/or the weight condition in a second storage position.

3. The method of claim 1, wherein returning the index of the second storage location meeting the preset elimination condition to the first storage location according to the parameter of the index of the second storage location comprises:

4. The method according to claim 1, further comprising, after the step of storing the index of the first storage location meeting the preset parameter condition to the second storage location according to the parameter of the index of the first storage location:

5. The method of claim 4, wherein storing the active index in the first memory table and the inactive index in the second memory table comprises:

6. The method of claim 4, wherein returning the index of the second storage location meeting the preset elimination condition to the first storage location according to the parameter of the index of the second storage location comprises:

7. The method according to claim 1, further comprising, after the step of returning the index of the second storage location meeting the preset elimination condition to the first storage location according to the parameter of the index of the second storage location:

And detecting whether the second storage position is in a trigger state, if so, continuing to acquire parameters of the index positioned at the second storage position, and returning the index of the second storage position meeting the preset elimination condition to the first storage position.

8. An index storage device for storing objects, comprising:

9. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1-7.

10. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs which when executed by the one or more processors cause the one or more processors to implement the method of any of claims 1-7.