CN112241376A - Cache data updating method and device, storage medium and computer equipment - Google Patents

Abstract

According to the cache data updating method, the cache data updating device, the cache data updating storage medium and the computer equipment, the data volume of the file blocks in the second type memory, which are the same as the sequence numbers of the file blocks in the first type memory and the sequence numbers of the file block groups, can be determined according to the sequence numbers of the file blocks and the sequence numbers of the file block groups, and if the data volume of the file blocks in the first type memory is different from the data volume of the file blocks in the second type memory, the file blocks in the second type memory are updated to the file blocks in the first type memory. The technical means that the data quantity of the file blocks with the same sequence numbers of the file blocks and the same sequence numbers of the file block groups stored in the first type memory and the second type memory are kept consistent is adopted, so that the technical problem that the cache memory cannot be updated is solved, and the technical effect of keeping the data in the cache memory consistent with the data in the non-cache memory is achieved.

Description

Cache data updating method and device, storage medium and computer equipment

Technical Field

The present invention relates to the field of data storage, and in particular, to a method and an apparatus for updating cache data, a storage medium, and a computer device.

Background

To facilitate data retrieval by a data requestor, a data provider typically places data requested by the data requestor from non-cache memory into cache memory. For example, the data provider may place hotspot segment data in a video from a non-cache memory into a cache memory, so that the data provider can quickly send the hotspot segment data to the data requester after the data requester requests the hotspot segment data.

However, after the data in the non-cached memory is placed in the cached memory, if the data in the non-cached memory is increased, the data in the cached memory may be inconsistent with the data in the non-cached memory. For example, after a data provider places hotspot fragment data in a video from a non-cache memory into a cache memory, as the hotspot fragment data of the video in the non-cache memory increases, the hotspot fragment data in the cache memory does not increase, and a problem arises in that the hotspot fragment data in the cache memory is inconsistent with the hotspot fragment data in the non-cache memory.

Disclosure of Invention

In view of the foregoing problems, the present invention provides a method, an apparatus, a storage medium, and a computer device for updating cache data, which overcome the foregoing problems or at least partially solve the foregoing problems, and the technical solutions are as follows:

a cache data updating method comprises the following steps:

acquiring a first data request, wherein the first data request carries a file identifier, a starting position and an offset of a first file;

determining the sequence number of at least one file block which is stored in a first type memory and is matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request;

determining the data amount of each file block in the first type memory;

determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block;

for any file block group with determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory.

Optionally, the determining, according to the sequence number of the file block, the sequence number of the file block group in which the data requested by the first data request is located includes:

for any file block of the at least one file block: and determining the sequence number of the file block group to which the file block belongs according to the sequence number of the file block, and determining the sequence number of the file block group as the sequence number of the file block group to which part or all of the data requested by the first data request belongs.

Optionally, the updating the file block stored in the second type memory to the file block stored in the first type memory includes:

and copying the data corresponding to the file block stored in the first type memory to the file block stored in the second type memory.

Optionally, the preset data size is a sum of maximum data sizes that all file blocks in the file block group can correspond to.

Optionally, the determining, according to the sequence number of the file block, the sequence number of the file block group in which the data requested by the first data request is located includes:

for each of the file blocks: and dividing the sequence number of the file block minus the initial sequence number of the file block by the number of the file blocks contained in the file block group to obtain a quotient, rounding the quotient downwards, and determining the rounded result as the sequence number of the file block group to which the file block belongs.

A cache data update apparatus, comprising: a data request obtaining unit, a file block number determining unit, a file block data amount determining unit, a file block group number determining unit, and a file block updating unit,

the data request obtaining unit is used for obtaining a first data request, wherein the first data request carries a file identifier, a starting position and an offset of a first file;

the file block sequence number determining unit is used for determining the sequence number of at least one file block which is stored in a first type memory and matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request;

the file block data amount determining unit is used for determining the data amount of each file block in the first type memory;

the file block group sequence number determining unit is used for determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block;

the file block updating unit is configured to, for any file block group with a determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory.

Optionally, the file block group number determining unit is specifically configured to, for any file block in the at least one file block: and determining the sequence number of the file block group to which the file block belongs according to the sequence number of the file block, and determining the sequence number of the file block group as the sequence number of the file block group to which part or all of the data requested by the first data request belongs.

Optionally, the file block updating unit is specifically configured to copy the data corresponding to the file block stored in the first type memory to the file block stored in the second type memory.

A storage medium having stored therein computer-executable instructions that, when loaded and executed by a processor, carry out a method of updating cached data as claimed in any one of the preceding claims.

A computer device comprising a processor, a memory and a program stored on the memory and executable on the processor, the processor when executing the program implementing at least the steps of:

acquiring a first data request, wherein the first data request carries a file identifier, a starting position and an offset of a first file;

determining the sequence number of at least one file block which is stored in a first type memory and is matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request;

determining the data amount of each file block in the first type memory;

determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block;

for any file block group with determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory.

By means of the technical scheme, the cache data updating method, the cache data updating device, the cache data updating storage medium and the computer equipment can obtain the first data request, wherein the first data request carries the file identifier, the initial position and the offset of the first file; determining the sequence number of at least one file block which is stored in a first type memory and is matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request; determining the data amount of each file block in the first type memory; determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block; for any file block group with determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory. The technical means that the data quantity of the file blocks with the same sequence numbers of the file blocks and the same sequence numbers of the file block groups stored in the first type memory and the second type memory are kept consistent is adopted, so that the technical problem that the cache memory cannot be updated is solved, and the technical effect of keeping the data in the cache memory consistent with the data in the non-cache memory is achieved.

The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

fig. 1 is a schematic flowchart illustrating a cache data updating method according to an embodiment of the present invention;

fig. 2 is a schematic diagram illustrating a manner of allocating sequence numbers of file blocks according to an embodiment of the present invention;

fig. 3 is a schematic diagram illustrating a manner of assigning sequence numbers of file block groups according to an embodiment of the present invention;

fig. 4 is a schematic flowchart illustrating another cache data updating method according to an embodiment of the present invention;

FIG. 5 is a schematic diagram illustrating the data size of the file block groups in the first type of memory and the data size of the file block groups in the second type of memory according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram illustrating a cache data updating apparatus according to an embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

As shown in fig. 1, a cache data updating method provided in an embodiment of the present invention may include:

s100, obtaining a first data request, wherein the first data request carries a file identifier, a starting position and an offset of a first file;

specifically, the identifier of the first file may be a feature identifier for distinguishing the first file from other files. For example, the identity of the first file may be a characteristic identity a, by means of which the storage location of the first file may be determined in a memory holding a plurality of files. The data requested by the first data request may be determined by a start position and an offset of the first file, for example, assuming that the first file may be a video file, it may be determined by the start position and the offset that the first data request requests video data within a first time period in the video file.

S200, determining the sequence number of at least one file block which is stored in a first type memory and matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request;

specifically, in the embodiment of the present invention, the first file may be divided into a plurality of file blocks, and the maximum data size that each file block may correspond to is the same, for example, the data size of the first file is 50MB, and the maximum data size that each file block may correspond to is 4MB, then the first file may be divided into 13 file blocks, in the 13 file blocks, the data size that 12 file blocks correspond to is 4MB, and the data size that 1 file block corresponds to is 2 MB. The maximum data amount that each file block can correspond to can be set according to actual requirements, and the invention is not further limited herein. The embodiment of the invention can allocate the sequence numbers to the file blocks according to the preset first sequence. The result of assigning sequence numbers to the file blocks divided by the first file according to the present invention may be as shown in fig. 2, where the first file is divided into 16 file blocks, and the 16 file blocks may be assigned according to sequence numbers 0, 1, 2, and 15, respectively, in the embodiment of the present invention. In this case, if the first file in the first data request has data corresponding to the file block with sequence number 5 at the start position and data corresponding to the file block with sequence number 10 at the position with the offset added to the start position, the data in the file blocks with sequence numbers 5 to 10 requested by the first data request can be determined (for convenience of description, in the following description of the present invention, the file block with the sequence number is represented in the form of file block + sequence number, for example, file block 5 is a file block with sequence number 5 representing a file block).

The first type of storage may include a plurality of first type storage disks, each of the first type storage disks may have one or more disk blocks, and storage spaces of the disk blocks in the first type storage disks may be the same, where the storage spaces of the disk blocks in the first type storage disks are the same as a maximum data amount to which the file blocks may correspond. Each disk block in the first type of storage may hold a file block. The first type of memory may be a non-cache memory, and the non-cache memory may be a mechanical hard disk, a hybrid hard disk, or the like, and the embodiment of the present invention is not limited herein. The first type memory may be communicatively coupled to an index server, and embodiments of the present invention may record in the index server a sequence number of at least one file block that matches data requested by the first data request. In the subsequent use process, the embodiment of the invention can determine the sequence number of at least one file block matched with the data requested by the first data request in the index server.

S300, determining the data volume of each file block in the first type memory;

specifically, the data amount of each file block corresponding to the first file may be recorded in an index server communicatively connected to the first type storage. Although the maximum data amount corresponding to each file block may be the same, the data amount corresponding to each file block may be different, for example, the maximum data amount corresponding to each file block may be 8MB, where the data amount corresponding to file block a in each file block may be 8MB, and the data amount corresponding to file block B in each file block may be 4 MB. By recording the data volume actually corresponding to the file blocks in the first-type storage in the index server, the data volume of each file block in the first-type storage can be directly determined in the index server.

S400, determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block;

the embodiment of the invention can group a plurality of file blocks obtained by dividing the first file. Specifically, in the embodiment of the present invention, the plurality of file blocks may be divided into one or more file block groups according to a preset number, and a relationship between the file blocks included in the file block group and the file block group is recorded. Meanwhile, the embodiment of the invention can allocate the sequence numbers to the file block groups according to a preset second sequence. For ease of understanding, this is illustrated here in connection with fig. 3 on the basis of fig. 2: if every 4 file blocks are a file block group, the file block groups are distributed according to the sequence numbers 0, 0. If it is determined that the first data request requests data in file blocks 5 to 10, the sequence numbers of the file block groups in which the data requested by the first data request is located are determined to be 1 and 2 (for convenience of description, in the following description of the present invention, the file block group of the sequence number is represented in the form of file block group + sequence number, for example, file block group 0 is a file block group representing a file block group with a sequence number of 0).

Meanwhile, the embodiment of the invention can record the sequence numbers of all the file block groups in the index server. In the subsequent use process, the embodiment of the present invention may determine, in the index server, the sequence number of the file block group in which the data requested by the first data request is located according to the sequence number of the file block.

Optionally, as shown in fig. 4, in another cache data updating method provided in the embodiment of the present invention, step S400 may specifically be:

s410, for any file block in the at least one file block: and determining the sequence number of the file block group to which the file block belongs according to the sequence number of the file block, and determining the sequence number of the file block group as the sequence number of the file block group to which part or all of the data requested by the first data request belongs.

For example, if the file block group 0 includes file block 0, file block 1, and file block 2, the file block group 1 includes file block 3 and file block 4, and the file block group 2 includes file block 5. If the first data request requests data in file block 0 to file block 4, it may be determined that the data requested by the first data request is in file block group 0 and file block group 1; if the first data request requests data in file 2 to file block 5, it may be determined that the data requested by the first data request is in file block group 0, file block group 1, and file block group 2.

Optionally, in another cache data updating method provided in the embodiment of the present invention, the number of file blocks included in each file block group may be the same, and step S400 may specifically be: for each of the file blocks: and dividing the sequence number of the file block minus the initial sequence number of the file block by the number of the file blocks contained in the file block group to obtain a quotient, rounding the quotient downwards, and determining the rounded result as the sequence number of the file block group to which the file block belongs.

For ease of understanding, the description herein is made in conjunction with FIG. 3: if it is determined that the first data request requests data in file blocks 5 to 10, the value obtained by subtracting the sequence number of file block 0 from the sequence number of file block 5 is 5, the number of file blocks included in the file block group is 4, the quotient obtained by dividing the value obtained by subtracting the sequence number of file block 0 from the sequence number of file block 5 by the number of file blocks 4 included in the file block group is rounded down to 1, and the file block group to which file block 5 belongs is file block group 1. Similarly, the file block group to which the file block 6 and the file block 7 belong is the file block group 1, and the file block group to which the file block 8, the file block 9 and the file block 10 belong is the file block group 2.

S500, for any file block group with a determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory.

Specifically, by file identification, the embodiment of the present invention may determine the storage location of the first file in the second type memory, and then determine the data size of the file block group in the second type memory according to the file block group with the determined sequence number. It should be noted that the data amount of the file block group with the same sequence number in the first type memory and the data amount in the second type memory may be the same or different. For ease of understanding, this is illustrated herein in connection with FIG. 5: in fig. 5, the shaded portion represents data corresponding to the file block in the file block group, and the blank portion is a storage space where the file block in the file block group can also correspond to data. The data volume of the file block group 1 in the first type memory and the data volume of the file block group 2 in the second type memory are both 4MB, but the data volume of the file block group 2 in the first type memory is 3MB, and the data volume of the file block group in the second type memory is 2MB, because the data added by the first file is firstly added in the file block group 2 in the first type memory, the data volume of the file block group 2 in the first type memory is the data volume after the data is added by the first file, and the file block group 2 in the second type memory is not updated, the data volume of the file block group 2 in the second type memory is still the data volume before the data is added by the first file, so that the data volume of the file block group with the same sequence number in the first type memory is different from the data volume of the file block group in the second type memory. Therefore, if the data is directly obtained from the second type memory according to the first data request, the obtained data is not the latest data, so that the embodiment of the present invention can determine whether the data of the file block group in the second type memory is the latest data by comparing the data amount of the file block group with the same sequence number in the second type memory with the data amount in the second type memory, so that the data obtained from the second type memory is the latest data.

The second type of storage may include a plurality of second type storage disks, each of the second type storage disks may have one or more disk blocks, and storage space of each disk block in the second type storage disks may be the same, where the storage space of the disk block in the second type storage disk is the same as the maximum data amount to which the file block may correspond. Each disk block in the second type of storage may hold a file block.

It should be noted that the embodiment of the present invention may group the disk blocks in the second type memory. Specifically, in the embodiment of the present invention, at least one disk block in each second-type storage may be divided into one disk block group, where the disk block groups of the disk blocks in the same second-type storage disk may be the same or different. The positions of the disk blocks included in the same disk block group in the second type storage disk are fixed, and for the convenience of understanding, the following description is made: if a certain disk block group comprises a disk block Q, a disk block W, a disk block E and a disk block R, if the positions of the disk block Q and the disk block W in a second type storage disk are respectively a first position and a third position, the positions of the disk block E and the disk block R in another second type storage disk are also a first position and a third position (namely, the disk block E is in the first position and the disk block R is in the third position, or the disk block E is in the third position and the disk block R is in the first position); if the locations of the disk blocks Q, W, and E in one second-type storage disk are first, third, and fourth, respectively, the location of the disk block R in another second-type storage disk may be any one of the first, third, and fourth locations. In an alternative embodiment of the present invention, a group of file blocks may be stored in a group of disk blocks of the second type of storage disk.

The second type of memory may be a cache memory, and the cache memory may be a solid state disk, which is not limited herein. The second type memory may be communicatively connected to a cache server, and the cache server may be communicatively connected to the index server. In the subsequent use process, the embodiment of the present invention may determine, in the cache server, the data amount of the file block group in the second type memory according to the sequence number of the file block group.

The embodiment of the invention can compare the data amount of a certain file block group in the second type memory with the preset data amount after determining the data amount of the file block group in the second type memory. The preset data size may be a predetermined threshold value for determining that the data size of the file block group in the second type memory is different from the data size of the file block group in the first type memory, and since the increase of the data of the first file in the embodiment of the present invention may be performed on a file block whose data size actually corresponding to a file block in the plurality of file blocks divided by the first file does not reach the maximum data size that can be corresponding to the file block at first, the preset data size may be the sum of the maximum data sizes that can be corresponding to all file blocks in the file block group. When the data amount of a certain file block group in the second type memory is equal to the sum of the maximum data amounts which can be corresponded to all file blocks in the file block group, the data added by the first file is not in the file block group.

When the data volume of the file block group in the second type memory is smaller than the predetermined data volume, it is indicated that the data volume of a certain file block in the second type memory is different from the data volume of the file block in the first type memory in all the file blocks included in the file block group. In this case, in the embodiment of the present invention, for any file block in the file block group, through the cache server: and determining the data volume of the file block in the second type memory according to the sequence number of the file block, and judging whether the data volume of the file block in the second type memory is the same as the data volume of the file block in the first type memory. Optionally, in the embodiment of the present invention, the data of the first file may be added from small to large according to the sequence numbers of the file blocks, and the file blocks, of the plurality of file blocks divided by the first file, whose actual data volumes do not reach the maximum data volume that the file blocks can correspond to are added, so that if the data volume of a certain file block group in the second type memory is different from the data volume of the file block group in the first type memory, it may be determined first whether the data volume of the file block with the largest sequence number in the second type memory among the file blocks included in the file block group is the same as the data volume of the file block in the first type memory, and if the data volume of the file block is different, the file block stored in the second type memory is updated to the file block stored in the first type memory.

In an alternative embodiment of the present invention, the process of updating the file block stored in the second type memory to the file block stored in the first type memory may be: and copying the data corresponding to the file block stored in the first type memory to the file block stored in the second type memory.

Specifically, in the embodiment of the present invention, the data corresponding to the file block stored in the first type memory may be used to overwrite the file block stored in the second type memory, or the data that is added to the data corresponding to the file block stored in the first type memory relative to the data of the file block stored in the second type memory may be copied to the file block stored in the second type memory.

According to the cache data updating method provided by the embodiment of the invention, a first data request can be obtained, wherein the first data request carries a file identifier, a starting position and an offset of a first file; determining the sequence number of at least one file block which is stored in a first type memory and is matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request; determining the data amount of each file block in the first type memory; determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block; for any file block group with determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory. The technical means that the data quantity of the file blocks with the same sequence numbers of the file blocks and the same sequence numbers of the file block groups stored in the first type memory and the second type memory are kept consistent is adopted, so that the technical problem that the cache memory cannot be updated is solved, and the technical effect of keeping the data in the cache memory consistent with the data in the non-cache memory is achieved.

Corresponding to the foregoing method embodiment, an embodiment of the present invention further provides a cache data updating apparatus, where the structure of the apparatus is shown in fig. 6, and the cache data updating apparatus includes: a data request obtaining unit 100, a file block number determination unit 200, a file block data amount determination unit 300, a file block group number determination unit 400, and a file block update unit 500,

the data request obtaining unit 100 is configured to obtain a first data request, where the first data request carries a file identifier, a start position, and an offset of a first file;

specifically, the identifier of the first file may be a feature identifier for distinguishing the first file from other files. For example, the identity of the first file may be a characteristic identity a, by means of which the storage location of the first file may be determined in a memory holding a plurality of files. The data requested by the first data request may be determined by the starting location and offset of the first file.

The file block sequence number determining unit 200 is configured to determine, according to a file identifier, a starting position, and an offset of a first file in the first data request, a sequence number of at least one file block that is stored in a first type memory and matches with data requested by the first data request;

specifically, in the embodiment of the present invention, the first file may be divided into a plurality of file blocks, and the maximum data size that each file block may correspond to is the same, for example, the data size of the first file is 50MB, and the maximum data size that each file block may correspond to is 4MB, then the first file may be divided into 13 file blocks, in the 13 file blocks, the data size that 12 file blocks correspond to is 4MB, and the data size that 1 file block corresponds to is 2 MB. The maximum data amount that each file block can correspond to can be set according to actual requirements, and the invention is not further limited herein. The embodiment of the invention can allocate the sequence numbers to the file blocks according to the preset first sequence.

The first type of storage may include a plurality of first type storage disks, each of the first type storage disks may have one or more disk blocks, and storage spaces of the disk blocks in the first type storage disks may be the same, where the storage spaces of the disk blocks in the first type storage disks are the same as a maximum data amount to which the file blocks may correspond. Each disk block in the first type of storage may hold a file block. The first type of memory may be a non-cache memory, and the non-cache memory may be a mechanical hard disk, a hybrid hard disk, or the like, and the embodiment of the present invention is not limited herein. The first type memory may be communicatively coupled to an index server, and embodiments of the present invention may record in the index server a sequence number of at least one file block that matches data requested by the first data request. In the subsequent use process, the embodiment of the invention can determine the sequence number of at least one file block matched with the data requested by the first data request in the index server.

The file block data amount determining unit 300 is configured to determine the data amount of each file block in the first type memory;

specifically, the data amount of each file block corresponding to the first file may be recorded in an index server communicatively connected to the first type storage. Although the maximum amount of data that each file block may correspond to is the same, the amount of data that each file block actually corresponds to may be different. By recording the data volume actually corresponding to the file blocks in the first-type storage in the index server, the data volume of each file block in the first-type storage can be directly determined in the index server.

The file block group sequence number determining unit 400 is configured to determine, according to the sequence number of the file block, a sequence number of a file block group in which the data requested by the first data request is located;

the embodiment of the invention can group a plurality of file blocks obtained by dividing the first file. Specifically, in the embodiment of the present invention, the plurality of file blocks may be divided into one or more file block groups according to a preset number, and a relationship between the file blocks included in the file block group and the file block group is recorded. Meanwhile, the embodiment of the invention can allocate the sequence numbers to the file block groups according to a preset second sequence.

Meanwhile, the embodiment of the invention can record the sequence numbers of all the file block groups in the index server. In the subsequent use process, the embodiment of the present invention may determine, in the index server, the sequence number of the file block group in which the data requested by the first data request is located according to the sequence number of the file block.

Optionally, the file block group number determining unit 400 may be specifically configured to, for any file block in the at least one file block: and determining the sequence number of the file block group to which the file block belongs according to the sequence number of the file block, and determining the sequence number of the file block group as the sequence number of the file block group to which part or all of the data requested by the first data request belongs.

Optionally, the number of file blocks included in each file block group is the same, and the file block group number determining unit 400 is specifically configured to, for each file block: and dividing the sequence number of the file block minus the initial sequence number of the file block by the number of the file blocks contained in the file block group to obtain a quotient, rounding the quotient downwards, and determining the rounded result as the sequence number of the file block group to which the file block belongs.

The file block updating unit 500 is configured to, for any file block group with a determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory.

Specifically, by file identification, the embodiment of the present invention may determine the storage location of the first file in the second type memory, and then determine the data size of the file block group in the second type memory according to the file block group with the determined sequence number. It should be noted that the data amount of the file block group with the same sequence number in the first type memory and the data amount in the second type memory may be the same or different.

The second type of storage may include a plurality of second type storage disks, each of the second type storage disks may have one or more disk blocks, and storage space of each disk block in the second type storage disks may be the same, where the storage space of the disk block in the second type storage disk is the same as the maximum data amount to which the file block may correspond. Each disk block in the second type of storage may hold a file block.

It should be noted that the embodiment of the present invention may group the disk blocks in the second type memory. Specifically, in the embodiment of the present invention, at least one disk block in each second-type storage may be divided into one disk block group, where the disk block groups of the disk blocks in the same second-type storage disk may be the same or different. The position of the disk blocks contained in the same disk block group in the second type storage disk is fixed.

The second type of memory may be a cache memory, and the cache memory may be a solid state disk, which is not limited herein. The second type memory may be communicatively connected to a cache server, and the cache server may be communicatively connected to the index server. In the subsequent use process, the embodiment of the present invention may determine, in the cache server, the data amount of the file block group in the second type memory according to the sequence number of the file block group.

The embodiment of the invention can compare the data amount of a certain file block group in the second type memory with the preset data amount after determining the data amount of the file block group in the second type memory. The preset data size may be a predetermined threshold value for determining that the data size of the file block group in the second type memory is different from the data size of the file block group in the first type memory, and since the increase of the data of the first file in the embodiment of the present invention may be performed on a file block whose data size actually corresponding to a file block in the plurality of file blocks divided by the first file does not reach the maximum data size that can be corresponding to the file block at first, the preset data size may be the sum of the maximum data sizes that can be corresponding to all file blocks in the file block group. When the data amount of a certain file block group in the second type memory is equal to the sum of the maximum data amounts which can be corresponded to all file blocks in the file block group, the data added by the first file is not in the file block group.

When the data volume of the file block group in the second type memory is smaller than the predetermined data volume, it is indicated that the data volume of a certain file block in the second type memory is different from the data volume of the file block in the first type memory in all the file blocks included in the file block group. In this case, in the embodiment of the present invention, for any file block in the file block group, through the cache server: and determining the data volume of the file block in the second type memory according to the sequence number of the file block, and judging whether the data volume of the file block in the second type memory is the same as the data volume of the file block in the first type memory. Optionally, in the embodiment of the present invention, the data of the first file may be added from small to large according to the sequence numbers of the file blocks, and the file blocks, of the plurality of file blocks divided by the first file, whose actual data volumes do not reach the maximum data volume that the file blocks can correspond to are added, so that if the data volume of a certain file block group in the second type memory is different from the data volume of the file block group in the first type memory, it may be determined first whether the data volume of the file block with the largest sequence number in the second type memory among the file blocks included in the file block group is the same as the data volume of the file block in the first type memory, and if the data volume of the file block is different, the file block stored in the second type memory is updated to the file block stored in the first type memory.

Optionally, the file block updating unit 500 is specifically configured to copy the data corresponding to the file block stored in the first type memory to the file block stored in the second type memory.

Specifically, in the embodiment of the present invention, the data corresponding to the file block stored in the first type memory may be used to overwrite the file block stored in the second type memory, or the data that is added to the data corresponding to the file block stored in the first type memory relative to the data of the file block stored in the second type memory may be copied to the file block stored in the second type memory.

The cache data updating device provided by the embodiment of the invention can obtain a first data request, wherein the first data request carries a file identifier, a starting position and an offset of a first file; determining the sequence number of at least one file block which is stored in a first type memory and is matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request; determining the data amount of each file block in the first type memory; determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block; for any file block group with determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory. The technical means that the data quantity of the file blocks with the same sequence numbers of the file blocks and the same sequence numbers of the file block groups stored in the first type memory and the second type memory are kept consistent is adopted, so that the technical problem that the cache memory cannot be updated is solved, and the technical effect of keeping the data in the cache memory consistent with the data in the non-cache memory is achieved.

The storage medium provided by the embodiment of the present invention is characterized in that a computer-executable instruction is stored in the storage medium, and when the computer-executable instruction is loaded and executed by a processor, the cache data updating method according to any one of the above-mentioned embodiments is implemented.

The computer device provided by the embodiment of the invention is characterized by comprising a processor, a memory and a program which is stored on the memory and can run on the processor, wherein the processor at least realizes the following steps when executing the program:

acquiring a first data request, wherein the first data request carries a file identifier, a starting position and an offset of a first file;

determining the sequence number of at least one file block which is stored in a first type memory and is matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request;

determining the data amount of each file block in the first type memory;

determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block;

for any file block group with determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory.

The cache data updating device comprises a processor and a memory, wherein the data request obtaining unit 100, the file block sequence number determining unit 200, the file block data amount determining unit 300, the file block group sequence number determining unit 400, the file block updating unit 500 and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.

The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can set one or more than one kernel, and the data volume of the file blocks with the same sequence numbers of the file blocks and the same sequence numbers of the file block groups in the first type memory and the second type memory is kept consistent by adjusting kernel parameters.

The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.

An embodiment of the present invention provides a storage medium, on which a program is stored, and when the program is executed by a processor, the cache data updating method is implemented.

The embodiment of the invention provides a processor, which is used for running a program, wherein the cache data updating method is executed when the program runs.

The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein the processor executes the program and realizes the following steps:

acquiring a first data request, wherein the first data request carries a file identifier, a starting position and an offset of a first file;

determining the sequence number of at least one file block which is stored in a first type memory and is matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request;

determining the data amount of each file block in the first type memory;

determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block;

for any file block group with determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory.

Optionally, the determining, according to the sequence number of the file block, the sequence number of the file block group in which the data requested by the first data request is located includes:

for any file block of the at least one file block: and determining the sequence number of the file block group to which the file block belongs according to the sequence number of the file block, and determining the sequence number of the file block group as the sequence number of the file block group to which part or all of the data requested by the first data request belongs.

Optionally, the updating the file block stored in the second type memory to the file block stored in the first type memory includes:

and copying the data corresponding to the file block stored in the first type memory to the file block stored in the second type memory.

Optionally, the preset data size is a sum of maximum data sizes that all file blocks in the file block group can correspond to.

Optionally, the determining, according to the sequence number of the file block, the sequence number of the file block group in which the data requested by the first data request is located includes:

for each of the file blocks: and dividing the sequence number of the file block minus the initial sequence number of the file block by the number of the file blocks contained in the file block group to obtain a quotient, rounding the quotient downwards, and determining the rounded result as the sequence number of the file block group to which the file block belongs.

The device herein may be a server, a PC, a PAD, a mobile phone, etc.

The present application further provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device:

acquiring a first data request, wherein the first data request carries a file identifier, a starting position and an offset of a first file;

determining the sequence number of at least one file block which is stored in a first type memory and is matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request;

determining the data amount of each file block in the first type memory;

determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block;

for any file block group with determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory.

Optionally, the determining, according to the sequence number of the file block, the sequence number of the file block group in which the data requested by the first data request is located includes:

for any file block of the at least one file block: and determining the sequence number of the file block group to which the file block belongs according to the sequence number of the file block, and determining the sequence number of the file block group as the sequence number of the file block group to which part or all of the data requested by the first data request belongs.

Optionally, the updating the file block stored in the second type memory to the file block stored in the first type memory includes:

and copying the data corresponding to the file block stored in the first type memory to the file block stored in the second type memory.

Optionally, the preset data size is a sum of maximum data sizes that all file blocks in the file block group can correspond to.

Optionally, the determining, according to the sequence number of the file block, the sequence number of the file block group in which the data requested by the first data request is located includes:

for each of the file blocks: and dividing the sequence number of the file block minus the initial sequence number of the file block by the number of the file blocks contained in the file block group to obtain a quotient, rounding the quotient downwards, and determining the rounded result as the sequence number of the file block group to which the file block belongs.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (10)

1. A cache data updating method is characterized by comprising the following steps:

acquiring a first data request, wherein the first data request carries a file identifier, a starting position and an offset of a first file;

determining the sequence number of at least one file block which is stored in a first type memory and is matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request;

determining the data amount of each file block in the first type memory;

determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block;

for any file block group with determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory.

2. The method of claim 1, wherein determining the sequence number of the file block group in which the data requested by the first data request is located according to the sequence number of the file block comprises:

for any file block of the at least one file block: and determining the sequence number of the file block group to which the file block belongs according to the sequence number of the file block, and determining the sequence number of the file block group as the sequence number of the file block group to which part or all of the data requested by the first data request belongs.

3. The method of claim 1, wherein updating the file block stored in the second type of memory to the file block stored in the first type of memory comprises:

and copying the data corresponding to the file block stored in the first type memory to the file block stored in the second type memory.

4. The method of claim 1, wherein the predetermined amount of data is a sum of maximum data amounts that all file blocks in the set of file blocks can correspond to.

5. The method of claim 1, wherein the number of file blocks included in each file block group is the same, and the determining the sequence number of the file block group in which the data requested by the first data request is located according to the sequence number of the file block comprises:

for each of the file blocks: and dividing the sequence number of the file block minus the initial sequence number of the file block by the number of the file blocks contained in the file block group to obtain a quotient, rounding the quotient downwards, and determining the rounded result as the sequence number of the file block group to which the file block belongs.

6. A cache data update apparatus, comprising: a data request obtaining unit, a file block number determining unit, a file block data amount determining unit, a file block group number determining unit, and a file block updating unit,

the data request obtaining unit is used for obtaining a first data request, wherein the first data request carries a file identifier, a starting position and an offset of a first file;

the file block sequence number determining unit is used for determining the sequence number of at least one file block which is stored in a first type memory and matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request;

the file block data amount determining unit is used for determining the data amount of each file block in the first type memory;

the file block group sequence number determining unit is used for determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block;

the file block updating unit is configured to, for any file block group with a determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory.

7. The apparatus according to claim 6, wherein the file block group number determining unit is specifically configured to, for any file block in the at least one file block: and determining the sequence number of the file block group to which the file block belongs according to the sequence number of the file block, and determining the sequence number of the file block group as the sequence number of the file block group to which part or all of the data requested by the first data request belongs.

8. The apparatus according to claim 6, wherein the file block updating unit is specifically configured to copy data corresponding to the file block stored in the first type of memory to the file block stored in the second type of memory.

9. A storage medium having stored thereon computer-executable instructions which, when loaded and executed by a processor, carry out a cache data update method according to any one of claims 1 to 5.

10. A computer device comprising a processor, a memory, and a program stored on the memory and executable on the processor, the processor when executing the program performing at least the following steps:

acquiring a first data request, wherein the first data request carries a file identifier, a starting position and an offset of a first file;

determining the sequence number of at least one file block which is stored in a first type memory and is matched with the data requested by the first data request according to the file identifier, the initial position and the offset of the first file in the first data request;

determining the data amount of each file block in the first type memory;

determining the sequence number of the file block group where the data requested by the first data request is located according to the sequence number of the file block;

for any file block group with determined sequence number: determining the data volume of the file block group in a second type memory according to the file identification, determining the data volume of each file block in the file block group in the second type memory when the data volume of the file block group in the second type memory is smaller than a preset data volume, and for any file block in the file block group: and determining whether the data volume of the file block in the first type memory is the same as the data volume of the file block in the second type memory according to the sequence number of the file block, and if so, updating the file block stored in the second type memory into the file block stored in the first type memory.

Images