CN114090637A

CN114090637A - Data access method, device, equipment and storage medium

Info

Publication number: CN114090637A
Application number: CN202111434099.2A
Authority: CN
Inventors: 郑庆
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2021-11-29
Filing date: 2021-11-29
Publication date: 2022-02-25

Abstract

The application relates to the technical field of artificial intelligence, and discloses a data access method, a device, equipment and a storage medium, wherein the method comprises the following steps: dividing the disk and the memory into partitions with preset values according to the partition instruction; when a data storage instruction is received, acquiring a storage key word and a storage key value from the instruction; obtaining a target partition sequence according to the hash value of the storage keyword; acquiring a target disk partition and a target cache partition according to the target partition sequence, determining a management area, a data area and a recovery area according to the target disk partition, and determining a temporary cache area and a permanent cache area according to the target cache partition; when the storage data amount corresponding to the storage key value is judged to be smaller than or equal to a first numerical value, storing the storage key word and the storage key value into a temporary cache region; otherwise, storing the storage key words and the storage key values into the target data area, correspondingly generating management information, and storing the management information into the cache resident area and the management area.

Description

Data access method, device, equipment and storage medium

Technical Field

The present application relates to the field of artificial intelligence technologies, and in particular, to a data access method, apparatus, device, and storage medium.

Background

Most backend services will put frequently used and infrequently changed data into cache in order to improve performance and improve concurrency. Each retrieval of data from a cache is much faster than the retrieval of data from a relational database. Currently, common caching tools in the market include redis, leveldb and the like. A redis database is a memory-only database that puts all data into memory. However, computer memory is expensive, and memory space is small compared to disks, and additionally, the utilization rate of cpu on a computer is not high because a redis database uses a single thread to process requests. And the level db database is a google database, and adopts a layered design, and the latest data is placed at the top layer. Reading data requires querying in each layer, has poor performance, and is suitable for scenes with more writes and less reads. They all have different disadvantages and do not give good compromise between performance and cost.

Disclosure of Invention

The present application mainly aims to provide a data access method, device, apparatus and storage medium, which aim to reduce hardware cost while ensuring database performance.

In a first aspect, the present application provides a data access method, including:

when a partition instruction is received, dividing a disk and a cache into a plurality of partitions with preset values, wherein each cache partition comprises a temporary cache area and a permanent cache area, and each disk partition comprises a management area, a data area and a recovery area;

when a data storage instruction is received, acquiring a storage keyword and a storage key value corresponding to the storage keyword according to the data storage instruction;

determining a hash value corresponding to the storage keyword as a target hash value, and obtaining a target partition sequence by taking the remainder of the preset numerical value according to the target hash value;

acquiring a target disk partition and a target cache partition according to the target partition sequence, determining a target management area, a target data area and a target recovery area according to the target disk partition, and determining a target cache temporary area and a target cache permanent area according to the target cache partition;

judging whether the storage data amount corresponding to the storage key value is larger than a first numerical value or not;

if the storage data amount is smaller than or equal to the first numerical value, storing the storage key word and the storage key value into the target cache temporary area;

if the storage data volume is larger than the first numerical value, storing the storage key word and the storage key value into the target data area, generating management information according to the storage key word and the storage key value, and storing the management information into the target cache resident area and the target management area.

In a second aspect, the present application further provides a data access device, comprising:

the system comprises a partitioning module, a cache module and a recovery module, wherein the partitioning module is used for partitioning a disk and a cache into partitions with preset numerical values when a partitioning instruction is received, each cache partition comprises a temporary cache area and a permanent cache area, and each disk partition comprises a management area, a data area and a recovery area;

the storage instruction receiving module is used for acquiring a storage keyword and a storage key value corresponding to the storage keyword according to a data storage instruction when the data storage instruction is received;

the partition sequence acquisition module is used for determining the hash value corresponding to the storage keyword as a target hash value and obtaining a target partition sequence by taking the remainder of the preset numerical value according to the target hash value;

the partition acquisition module is used for acquiring a target disk partition and a target cache partition according to the target partition sequence, determining a target management area, a target data area and a target recovery area according to the target disk partition, and determining a target cache temporary area and a target cache permanent area according to the target cache partition;

the judging module is used for judging whether the storage data volume corresponding to the storage key value is larger than a first numerical value or not;

a first storage module, configured to store the storage key and the storage key value into the target cache temporary area if the storage data amount is less than or equal to the first numerical value;

and the second storage module is used for storing the storage key word and the storage key value into the target data area if the storage data volume is larger than the first numerical value, generating management information according to the storage key word and the storage key value, and storing the management information into the target cache resident area and the target management area.

In a third aspect, the present application also provides a computer device comprising a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the data access method as described above.

In a fourth aspect, the present application also provides a storage medium having a computer program stored thereon, where the computer program is executed by a processor to implement the steps of the data access method as described above.

The application provides a data access method, a device, equipment and a storage medium, wherein when a partition instruction is received, a disk and a cache are divided into partitions with preset numerical values; when a data storage instruction is received, acquiring a target disk partition and a target cache partition according to the hash value corresponding to the storage keyword; if the storage data volume corresponding to the storage key value is smaller than or equal to the first numerical value, storing the storage key word and the storage key value into a target cache temporary region corresponding to the target cache partition; otherwise, storing the storage key words and the storage key values into the target data area corresponding to the target disk partition, and simultaneously generating and storing corresponding management information. Through the technical scheme provided by the application, the performance of the database is ensured, and meanwhile, the hardware cost is reduced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flowchart illustrating steps of a data access method according to an embodiment of the present application;

FIG. 2 is a schematic block diagram of a data access device according to an embodiment of the present application;

fig. 3 is a schematic block diagram of a structure of a computer device according to an embodiment of the present disclosure.

The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The flow diagrams depicted in the figures are merely illustrative and do not necessarily include all of the elements and operations/steps, nor do they necessarily have to be performed in the order depicted. For example, some operations/steps may be decomposed, combined or partially combined, so that the actual execution sequence may be changed according to the actual situation. In addition, although the division of the functional blocks is made in the device diagram, in some cases, it may be divided in blocks different from those in the device diagram.

The embodiment of the application provides a data access method, a data access device, data access equipment and a storage medium. The data access method can be applied to terminal equipment or a server, and the terminal equipment can be electronic equipment such as a mobile phone, a tablet computer, a notebook computer, a desktop computer, a personal digital assistant and wearable equipment; the server may be a single server or a server cluster including a plurality of servers. The following explanation takes the application of the data access method to a server as an example.

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.

Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a data access method according to an embodiment of the present disclosure.

As shown in fig. 1, the data access method includes steps S10 to S16.

Step S10, when the partition instruction is received, the disk and the cache are divided into partitions of a preset number, where each cache partition includes a temporary cache area and a permanent cache area, and each disk partition includes a management area, a data area, and a recovery area.

It can be understood that, if the main body executing the present application is a server, when a partition instruction is received, the server is triggered to divide the disk space into the partitions with the preset values, and the cache space is also divided into the partitions with the preset values. Each disk partition has a cache partition to work with. In some embodiments, the cache referred to herein is a cache in a memory bank.

The inside of each cache partition is divided into two sub-areas, namely a cache temporary area and a cache permanent area. The temporary cache area is a cache partition and is used for storing temporary data. And the cache resident area is a cache partition used for storing resident data. Specifically, the cache resident area is used for storing management information.

The interior of each disk partition is divided into three sub-areas, namely a management area, a data area and a recovery area. The data stored in the management area is the same as the data stored in the cache resident area, and the difference is that the data in the management area is stored in a disk, and the data in the cache resident area is stored in a cache. When the data amount corresponding to the storage data meets a certain condition, for example, the storage space is greater than 1MB, the storage data can be stored in the storage file of the data area. The reclamation area is used for recording the storage space size occupied by the old data of the recyclable storage space in each storage file of the data area.

In some embodiments, the preset value is an exponential power of 2, and may be, for example, 4, 8, 16, 32, etc., without limitation. It can be understood that setting the preset value to the power of 2, the data access efficiency can be improved, and the capacity expansion operation of the later partition is facilitated.

And step S11, when a data storage instruction is received, acquiring a storage key word and a storage key value corresponding to the storage key word according to the data storage instruction.

It can be understood that the data storage instruction is a data write instruction, and the data storage instruction includes a storage key and a storage key value, where the storage key is a key, and the storage key value is a value corresponding to the key.

Illustratively, assume that the following data information is included in the data storage instruction: { "name": ABC "}. Then in the data store instruction, "name" is key and "ABC" is value.

And step S12, determining the hash value corresponding to the storage keyword as a target hash value, and obtaining a target partition sequence by taking the remainder of the preset numerical value according to the target hash value.

It is to be understood that the target partition sequence is a value obtained by adding up the preset value to the target hash value, and is used for matching the storage partition. And the storage partitions are matched according to the target hash value, so that the storage data can be more uniformly distributed into the storage partitions.

For example, assuming that the preset value is 32 and the hash value corresponding to the storage key is 51, the storage key is used to balance 32, and the obtained target partition sequence is 19.

Step S13, obtaining a target disk partition and a target cache partition according to the target partition sequence, determining a target management area, a target data area, and a target recovery area according to the target disk partition, and determining a target cache temporary area and a target cache permanent area according to the target cache partition.

It is understood that, through step S10, the disk is divided into the disk partitions with the preset number, and the cache is divided into the cache partitions with the preset number. And correspondingly, obtaining the cache partition according to the target partition sequence, namely obtaining the target cache partition.

The management area in the target disk partition is a target management area, the data area in the target disk partition is a target data area, and the recovery area in the target disk partition is a target recovery area. Similarly, the temporary cache area in the target cache partition is the target temporary cache area, and the permanent cache area in the target cache partition is the target permanent cache area.

For example, assuming the preset value is 32, the target partition sequence is 19. Then the 19 th disk partition is obtained from the 32 disk partitions as the target storage partition, and the 19 th cache partition is obtained from the 32 cache partitions as the target cache partition.

And step S14, judging whether the storage data amount corresponding to the storage key value is larger than a first numerical value.

It can be understood that the amount of data to be stored, i.e. the storage space required to store the key value when storing the key value.

In some embodiments, the first value is set to 1MB, may also be set to 2MB, 3MB, etc., and may also be set by itself as needed, which is not limited herein.

Step S15, if the storage data amount is smaller than or equal to the first value, storing the storage key and the storage key value into the target cache temporary area.

It can be understood that the access efficiency of data in the cache is higher than that of the disk, and accordingly, the hardware providing the cache space is the memory bank, the hardware providing the disk space is the disk, and the hardware cost of the memory bank is higher than that of the disk under the same storage space.

It is understood that in a certain storage space, more data with a small data amount can be stored than data with a large data amount. If the data size corresponding to the storage key value is not larger than the first numerical value, the storage key value can be regarded as data with small data size. At this time, the storage key and the storage key value are stored into the target cache temporary area.

By the implementation method, the target cache temporary area can store more small data, so that frequent data writing to a disk is avoided, and the data storage efficiency can be improved.

Step S16, if the storage data size is greater than the first value, store the storage key and the storage key value in the target data area, generate management information according to the storage key and the storage key value, and store the management information in the target cache resident area and the target management area.

It can be understood that, if the data size corresponding to the storage key value is larger than the first value, the storage key value may be considered as data with a large data size, and at this time, the storage key and the storage key value need to be stored in the target data area.

The management information is stored in the target cache resident area and the target management area at the same time, and when the data in the target cache resident area is lost due to the problems of power failure, downtime and the like of equipment, the target cache resident area can also perform data recovery through the target management area. In addition, when data needs to be queried, the management information can be acquired from the target cache resident area, and corresponding data can be acquired from the disk through the management information. By the technical scheme provided by the embodiment, the data query efficiency can be improved.

In some embodiments, the storing the management information in the target cache resident area and the target management area includes:

determining the storage capacity of the target cache resident area as a target resident capacity;

surplus is taken out of the target resident capacity according to the target hash value, and a target storage address is obtained;

and respectively storing the management information into the target cache resident area and the storage position of the target management area corresponding to the target storage address.

It is understood that the storage capacity of the target cache resident area, i.e. the size of the storage space of the target cache resident area, is the sum of the used storage space and the unused storage space of the target cache resident area.

And (4) obtaining the target storage address by taking the surplus of the target resident capacity according to the hash value corresponding to the storage keyword. And respectively storing the management information into the storage positions of the target cache resident area and the target storage address corresponding to the target management area. When the data needs to be queried, the query address can be matched according to the hash value corresponding to the query keyword, and the data is queried from the position corresponding to the target cache resident area according to the query address. Through the technical scheme corresponding to the embodiment, when data is queried, the data does not need to be queried from the beginning in a traversal mode in the target cache resident area, and the data query efficiency can be improved.

In some embodiments, if the target cache resident area and the area of the target management area corresponding to the target storage address already store data, determining whether the next storage address of the target storage address stores data, and repeating the steps until an address without data is matched and management information is stored at the address.

In some embodiments, the method further comprises:

determining the residual storage capacity of the target cache temporary area as the residual temporary capacity;

when the residual temporary capacity is smaller than a second numerical value, selecting a temporary key word and a temporary key value corresponding to the temporary key word from temporary data of the target cache temporary area;

splicing the temporary key words and the temporary key values to obtain a first recording result, and determining the storage data volume corresponding to the first recording result as a first data volume;

configuring the temporary keywords and the storage file of the temporary keyword value according to the storage state information of the target data area, and determining a storage path corresponding to the storage file as a matching storage path;

acquiring a temporary check value corresponding to the temporary key value according to a preset algorithm, wherein the temporary check value is used for checking the temporary key value;

splicing the temporary keyword, the matched storage path, the temporary check value and the first data volume to obtain a second recording result, and determining that the storage data volume corresponding to the second recording result is a second data volume;

when the first data volume is smaller than or equal to the second data volume, determining that the first recording result is the temporary key and the management information of the temporary key value;

and storing the first record result into the target cache resident area and the target management area.

It is understood that the remaining storage space of the target cache temporary area is the remaining temporary capacity. When the remaining temporary capacity of the target cache temporary area is smaller than the second value, the data stored in the target cache temporary area needs to be stored in the target cache permanent area and the target disk partition, and the data in the target cache temporary area needs to be cleared.

In some embodiments, the second value may be set to 1MB or 0.5MB, or may be set to other values according to the capacity of the temporary area of the target cache, which is not limited herein.

Specifically, keywords and key values are selected one by one from temporary data in the temporary area of the target cache for processing, where the currently selected keyword to be processed is the temporary keyword, and the key value corresponding to the temporary keyword is the temporary key value.

And splicing the temporary key words and the temporary key values to obtain a splicing result, namely a first recording result, and storing a storage space required to be occupied by the first recording result, namely a first data volume.

It can be understood that the target data area stores the key and the key value by constructing the storage files, and the size of the storage space allocated to each storage file by the target data area is the same, and only when the storage space of the last storage file located in the target data area cannot store new data continuously because a large amount of data is already stored, a new storage file continues to be created to store the key and the key value. For example, assuming that there are A, B two storage files in the target data area, and the storage space allocated to each storage file in the target data area is 10MB, the B file is created after the a file is full of data.

In addition, the target data area is provided with a corresponding recording file, and the recording file records storage state information of the target data area, such as how much storage space remains in the last storage file. And judging whether the last storage file of the target data area can sufficiently contain the target key word and the target key value or not by storing the state information, if not, creating a new storage file, and taking the storage file as the last storage file of the target data area. And storing the target key and the file path corresponding to the storage file of the target key value, namely, matching the storage path.

In some embodiments, the preset algorithm may be an md5 encryption algorithm, or may be other algorithms, and the check value corresponding to the temporary key value, that is, the temporary check value, may be obtained through the preset algorithm. Through the temporary check value, whether the temporary key value corresponding to the temporary key is changed or not can be checked.

The second recording result records a temporary keyword, a matching storage path, a temporary check value and a first data volume, and stores a storage space required to be occupied by the second recording result, namely a second data volume. It will be appreciated that if the amount of data for the target key value is small enough, the first amount of data will be smaller than the second amount of data. At this time, the first recording result is determined to be the management information of the temporary key and the temporary key value, and the first recording result, that is, the management information of the temporary key and the temporary key value is stored in the target cache resident area and the target management area.

By the technical scheme provided by the embodiment, if the data volume of the target key value is small enough, when the management information is acquired in a matching manner during data query, the corresponding key value can be directly acquired from the management information.

In some embodiments, the method further comprises:

when the first data volume is larger than the second data volume, determining that the second recording result is the temporary key and the management information of the temporary key value;

storing the first recording result into the target data area according to the matching storage path;

and storing the second recording result into the target cache resident area and the target management area.

It is understood that if the amount of data for the target key value is too large, the first amount of data may be larger than the second amount of data. At this time, the second recording result is determined to be the management information of the temporary key and the temporary key value, and the second recording result, that is, the management information of the temporary key and the temporary key value is stored in the target cache persistent area and the target management area. Meanwhile, storing the first recording result into a position corresponding to the target data area according to the matched storage path;

by the technical scheme provided by the embodiment, when data is queried, after the corresponding second recording result is obtained according to the query keyword, the corresponding storage file can be obtained according to the matching storage path in the second recording result, so that the first recording result corresponding to the query keyword is obtained from the storage file, and the key value corresponding to the query keyword is obtained from the first recording result.

It is understood that the corresponding data amount is very small regardless of the first recording result or the second recording result, and thus the target cache resident area can store a large amount of management information.

In some embodiments, the method further comprises:

when a data query instruction is received, acquiring query keywords from the data query instruction;

determining a hash value corresponding to the query keyword as a query hash value, and obtaining a query partition sequence by taking the remainder of the preset numerical value according to the query hash value;

acquiring a query disk partition and a query cache partition according to the query partition sequence;

determining a query management area, a query data area and a query recovery area according to the query disk partition, and determining a query cache temporary area and a query cache permanent area according to the query cache partition;

judging whether the query cache temporary area stores a query key value corresponding to the query key word;

if the query key value is stored in the temporary region of the query cache, returning the query key value;

if the query key value is not stored in the query cache temporary area, acquiring query management information matched with the query key word from the query cache permanent area, and acquiring the query key value according to the query management information.

It can be understood that the data query instruction is provided with a query keyword, and after the query keyword is obtained, the corresponding disk partition and the cache partition may be matched according to the hash value of the query keyword, that is, the disk partition and the cache partition are queried. The management area in the disk partition is inquired, namely the inquiry management area, the data area in the disk partition is inquired, namely the inquiry data area, and the recovery area in the disk partition is inquired, namely the inquiry recovery area. Similarly, querying the temporary cache area in the cache partition is referred to as querying the temporary cache area, and querying the permanent cache area in the cache partition is referred to as querying the permanent cache area.

In addition, when the key and the key value are stored, if the data size corresponding to the key value is smaller than or equal to the first numerical value, the key and the key value are stored in the temporary buffer area. That is, when a data query instruction is received, the query key and the query key value may also be stored in the query cache temporary area. The query data in the temporary area of the cache is queried first. If the query is not available, the management information matched with the query keyword is obtained in the query cache resident area, namely the management information is queried. By querying the management information, a query key value corresponding to the query key can be obtained.

It can be understood that the speed of reading data from the cache is higher than the efficiency of reading data from the disk, and in this embodiment, the data is read from the cache regardless of whether the query key value is directly obtained from the temporary area of the query cache or the query management information is obtained from the permanent area of the query cache. By the technical scheme provided by the embodiment, the efficiency of data query can be improved.

In some embodiments, the method further comprises:

when a data updating instruction is received, acquiring an updating keyword and an updating key value corresponding to the updating keyword from the data updating instruction;

determining a hash value corresponding to the update keyword as an update hash value, and taking the remainder of the preset numerical value according to the update hash value to obtain an update partition sequence;

acquiring an updated disk partition and an updated cache partition according to the updated partition sequence;

determining an update management area, an update data area and an update recovery area according to the update disk partition, and determining an update cache temporary area and an update cache permanent area according to the update cache partition;

updating the data information of the update temporary cache region corresponding to the update key word according to the update key value;

when the data information corresponding to the updating key word is not stored in the temporary area of the updating cache, acquiring first management information matched with the updating key word from the permanent area of the updating cache;

updating the update recovery area according to the first management information, and removing the first management information from the update management area and the update cache resident area;

judging whether the storage data volume corresponding to the updating key value is larger than the first numerical value or not;

if the storage data volume corresponding to the update key value is smaller than or equal to the first numerical value, storing the update key and the update key value into the update cache temporary area;

if the storage data volume corresponding to the update key value is larger than the first numerical value, the update key and the update key value are stored in the update data area, second management information is generated according to the update key and the update key value, and the second management information is stored in the update cache resident area and the update management area.

It can be understood that the data update instruction is provided with an update key and an update key value, and after the update key is obtained, the corresponding disk partition and the cache partition may be matched according to the hash value of the update key, that is, the disk partition is updated and the cache partition is updated. The management area in the disk partition is updated, namely the update management area, the data area in the disk partition is updated, namely the update data area, and the recovery area in the disk partition is updated, namely the update recovery area. Similarly, the temporary cache area in the cache partition is updated, that is, the temporary cache area is updated, and the permanent cache area in the cache partition is updated, that is, the permanent cache area is updated.

It can be understood that if the data information corresponding to the update key is stored in the temporary area of the update cache, the data update can be completed by setting the key value of the data information as the update key value.

In addition, if the data information corresponding to the update key is not stored in the update cache temporary area, the previously stored management information corresponding to the update key, i.e., the first management information, is acquired from the update cache permanent area. And if the first management information is provided with a matching storage path and a first data volume corresponding to the original keyword and the key value, recording the matching storage path and the first data volume into an update recovery area. After the record in the update recovery area, the first management information is useless, and at this time, the first management information needs to be removed from the update management area and the update cache resident area to release the storage space of the update management area and the update cache resident area. After removing the first management information, the process of updating the data is the same as the process of storing the data, and reference may be made to steps S14 to S16.

It can be understood that, when a data update instruction is received, the old data of the storage file in the update data area is not directly removed, but the recoverable space of each storage file is recorded in the update recovery area, and when the storage file reaches a certain condition, the old data is uniformly removed.

In some embodiments, the method further comprises:

when a data deleting instruction is received, acquiring a deleting keyword from the data deleting instruction;

determining the hash value corresponding to the deletion keyword as a deletion hash value, and taking the remainder of the preset numerical value according to the deletion hash value to obtain a deletion partition sequence;

acquiring a deleted disk partition and a deleted cache partition according to the deleted partition sequence;

determining a deletion management area, a deletion data area and a deletion recovery area according to the deleted disk partition, and determining a deletion cache temporary area and a deletion cache permanent area according to the deletion cache partition;

removing data information corresponding to the deletion key from the deletion cache temporary area;

when the temporary area of the delete cache does not store the data information corresponding to the delete key words, obtaining delete management information matched with the delete key words from the permanent area of the delete cache;

updating the deletion recovery area according to the deletion management information, and removing the deletion management information from the deletion management area and the deletion cache resident area;

it can be understood that the processing procedure for the data deletion instruction is substantially the same as the processing procedure for the first half of the data update instruction, and therefore, redundant description is not repeated here.

In some embodiments, the method further comprises:

periodically checking the recovery areas corresponding to the disk partitions, and screening the recovery areas with recovery record information meeting preset conditions from the recovery areas as execution recovery areas;

determining the disk partition to which the execution recovery area belongs as a recovery disk partition, and determining the cache partition corresponding to the recovery disk partition as a recovery cache partition;

setting the recovery disk partition and the recovery cache partition to be in a locking state;

acquiring the data area corresponding to the recovery disk partition as a recovery data area;

merging the stored data in the recovery data area according to the recovery record information of the execution recovery area;

and when the merging processing is finished, setting the recovery disk partition and the recovery cache partition to be in an unlocking state.

It can be understood that, when the data update instruction and the data deletion instruction are received, the old data stored in the storage file of the data area is not directly deleted. But the storage path corresponding to the storage file storing the old data and the data amount of the old data are recorded in the recycling record information of the recycling area.

And when the recovery area corresponding to each disk partition is periodically checked, if the recovery record information of the recovery area is found to meet the preset condition, confirming the recovery area as an execution recovery area. Correspondingly, the disk partition in which the recovery execution area is accommodated is the recovery disk partition. And the cache partition corresponding to the recovery disk partition is a recovery cache partition, the data area of the recovery disk partition is a recovery data area, and the management area of the recovery disk partition is a recovery management area.

Before merging the stored data in the recovery data area according to the recovery record information of the execution recovery area, the recovery disk partition and the recovery cache partition need to be set to be in a locked state, and at this time, the recovery disk partition and the recovery cache partition cannot read and write the data. And when the merging processing is finished, unlocking the recovery disk partition and the recovery cache partition. By the method, data can be prevented from being read and written in the recovery process, so that the data recovery process is disturbed, and even data errors are caused.

When merging processing is carried out, screening storage files for executing recycling operation according to recycling record information of the recycling area, checking the storage files with corresponding management information in the recycling cache partition through check values corresponding to all key values of the storage files, and when the check values are not passed, indicating that the key values are old data, and deleting the old data.

In the process of deleting old data in the storage file, the storage space of the corresponding position of the storage file is released, at the moment, the storage files which execute the recovery operation are merged, the storage space is reallocated to the storage data in the storage file, and a plurality of storage files can be merged into a small number of storage files through the merging processing.

Meanwhile, in the merging process, the storage location corresponding to the storage data in the storage file is changed, and the management information of the recovery cache partition and the recovery management partition needs to be updated.

In some embodiments, the preset condition may be: there is more than 5 stored files in the data area corresponding to more than twenty percent of the recoverable space, in which case at least 5 stored files may be merged into 4 stored files. It can be understood that, in the embodiment, the recovery record information in the recovery area triggers the data area to perform data merging processing only when the recovery record information reaches the preset condition.

In the application, when a partitioning instruction is received, a disk and a cache are divided into partitions with preset numerical values; when a data storage instruction is received, acquiring a target disk partition and a target cache partition according to the hash value corresponding to the storage keyword; if the storage data volume corresponding to the storage key value is smaller than or equal to the first numerical value, storing the storage key word and the storage key value into a target cache temporary region corresponding to the target cache partition; otherwise, storing the storage key words and the storage key values into the target data area corresponding to the target disk partition, and simultaneously generating and storing corresponding management information. Through the technical scheme provided by the application, the performance of the database is ensured, and meanwhile, the hardware cost is reduced.

Referring to fig. 2, fig. 2 is a schematic block diagram of a data access device according to an embodiment of the present disclosure.

As shown in fig. 2, the data access apparatus 201 includes:

the partition module 2011 is configured to, when receiving a partition instruction, divide the disk and the cache into partitions with preset values, where each cache partition includes a temporary cache area and a permanent cache area, and each disk partition includes a management area, a data area, and a recovery area;

a storage instruction receiving module 2012, configured to, when receiving a data storage instruction, obtain a storage keyword and a storage key value corresponding to the storage keyword according to the data storage instruction;

a partition sequence obtaining module 2013, configured to determine that the hash value corresponding to the storage keyword is a target hash value, and obtain a target partition sequence by taking the remainder of the preset value according to the target hash value;

a partition obtaining module 2014, configured to obtain a target disk partition and a target cache partition according to the target partition sequence, determine a target management area, a target data area, and a target recovery area according to the target disk partition, and determine a target temporary cache area and a target permanent cache area according to the target cache partition;

a judging module 2015, configured to judge whether a storage data amount corresponding to the storage key value is greater than a first numerical value;

a first storing module 2016, configured to store the storage key and the storage key value into the target cache temporary region if the storage data amount is less than or equal to the first numerical value;

a second storing module 2017, configured to, if the amount of the stored data is greater than the first numerical value, store the storage key and the storage key value in the target data area, generate management information according to the storage key and the storage key value, and store the management information in the target cache resident area and the target management area.

In some embodiments, the second storing module 2017, when storing the management information in the target cache resident area and the target management area, includes:

In some embodiments, the data access device 201 further comprises a first temporary data module 2018, configured to determine the remaining storage capacity of the target cache temporary region as a remaining temporary capacity;

In some embodiments, the data access apparatus 201 further includes a second temporary data module 2019, configured to determine that the second recording result is the temporary key and management information of the temporary key value when the first data amount is greater than the second data amount;

In some embodiments, the data access apparatus 201 further includes a query module 2020, configured to, when receiving a data query instruction, obtain a query keyword from the data query instruction;

In some embodiments, the data access apparatus 201 further includes an update module 2021, configured to, when a data update instruction is received, obtain an update key and an update key value corresponding to the update key from the data update instruction;

In some embodiments, the data access apparatus 201 further includes a data recovery module 2022, configured to periodically check the recovery area corresponding to each disk partition, and screen the recovery area with recovery record information meeting a preset condition from each recovery area as an execution recovery area;

It should be noted that, as will be clear to those skilled in the art, for convenience and brevity of description, the specific working processes of the apparatus and the modules and units described above may refer to the corresponding processes in the foregoing data access method embodiments, and are not described herein again.

The apparatus provided by the above embodiments may be implemented in the form of a computer program, which can be run on a computer device as shown in fig. 3.

Referring to fig. 3, fig. 3 is a schematic block diagram of a computer device according to an embodiment of the present disclosure. The computer device includes, but is not limited to, a terminal device.

As shown in fig. 3, the computer device 301 includes a processor 3011, a memory and a network interface connected through a system bus, where the memory may include a storage medium 3012 and an internal memory 3015, and the storage medium 3012 may be non-volatile or volatile.

The storage medium 3012 may store an operating system and computer programs. The computer program comprises program instructions that, when executed, cause the processor 3011 to perform any of the data access methods.

Processor 3011 is used to provide computing and control capabilities, supporting the operation of the overall computer device.

The internal memory 3015 provides an environment for running a computer program on the storage medium 3012, and the computer program, when executed by the processor 3011, may cause the processor 3011 to execute any data access method.

The network interface is used for network communication, such as sending assigned tasks and the like. Those skilled in the art will appreciate that the architecture shown in fig. 3 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

It should be understood that Processor 3011 may be a Central Processing Unit (CPU), and that Processor 3011 may also be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

In some embodiments, the processor 3011 is configured to run a computer program stored in the memory to implement the following steps:

In some embodiments, the processor 3011, when storing the management information in the target cache resident area and the target management area, is configured to:

In some embodiments, the processor 3011 is further configured to implement:

It should be noted that, as is clear to those skilled in the art, for convenience and brevity of description, the specific working process of the computer device described above may refer to the corresponding process in the foregoing data access method embodiment, and details are not described herein again.

The present application also provides a storage medium, which is a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, where the computer program includes program instructions, and a method implemented when the program instructions are executed may refer to various embodiments of the data access method in the present application.

The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, for example, a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device.

It is to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items. It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments. While the invention has been described with reference to specific embodiments, the scope of the invention is not limited thereto, and those skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for accessing data, comprising:

2. The method of claim 1, wherein storing the management information in the target cache resident area and the target management area comprises:

3. The method of claim 2, further comprising:

4. The method of claim 3, further comprising:

5. The method of claim 4, further comprising:

6. The method according to any one of claims 1-5, further comprising:

7. The method of claim 6, further comprising:

8. A data access device, comprising:

9. A computer arrangement comprising a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the data access method of any one of claims 1 to 7.

10. A computer-readable storage medium, having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the steps of the data access method of any one of claims 1 to 7.