CN111159438A - Data storage and retrieval method, electronic device and storage medium - Google Patents

Data storage and retrieval method, electronic device and storage medium Download PDF

Info

Publication number
CN111159438A
CN111159438A CN201811323660.8A CN201811323660A CN111159438A CN 111159438 A CN111159438 A CN 111159438A CN 201811323660 A CN201811323660 A CN 201811323660A CN 111159438 A CN111159438 A CN 111159438A
Authority
CN
China
Prior art keywords
data
current
storage area
index
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811323660.8A
Other languages
Chinese (zh)
Other versions
CN111159438B (en
Inventor
朱明�
阮淼锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN201811323660.8A priority Critical patent/CN111159438B/en
Publication of CN111159438A publication Critical patent/CN111159438A/en
Application granted granted Critical
Publication of CN111159438B publication Critical patent/CN111159438B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the invention provides a data storage and retrieval method, electronic equipment and a storage medium, wherein the data storage method comprises the following steps: the method comprises the steps of obtaining the data volume and description information of current data to be stored, selecting a current data storage area, distributing a current storage block for the current data to be stored, obtaining the position information of the current storage block, storing the position information of the current storage block into a third-level index area, obtaining the position information of the current data storage area from a second-level index area, generating a first-level index according to the position information of the current storage block and the position information of the current data storage area, correspondingly storing the first-level index and the description information of the current data to be stored into the first-level index area, and storing the current data to be stored into the current storage block. The data can be stored in the same storage mode aiming at different types of data, the workload of developers is reduced, the time cost for searching in the storage medium adopting the storage mode is reduced, and the searching efficiency is improved.

Description

Data storage and retrieval method, electronic device and storage medium
Technical Field
The present invention relates to the field of data storage technologies, and in particular, to a data storage and retrieval method, an electronic device, and a storage medium.
Background
With the development of information technology, a large amount of application data generated by more and more different applications needs to be stored in a storage medium. At present, the method for storing data in a storage medium is generally to pre-establish a multi-level index according to data content, for example, establish a primary index according to an event code of a picture, a channel number of the picture and a secondary index file address, establish a secondary index according to the earliest time, the latest time and the address of the tertiary index of the picture, and establish a tertiary index according to the length of the picture, the shooting time of the picture and the picture data packet address. The picture is then stored in a picture data packet.
When retrieving a picture, a second-level index file address matched with the event code of the picture and the channel number of the picture is retrieved in a first-level index according to the event code of the picture and the channel number of the picture, a third-level index file address matched with the earliest time and the latest time of the picture is retrieved in a second-level index corresponding to the second-level index file address, the address of a picture data packet is determined in a third-level index corresponding to the third-level index file address, and finally the picture is read from the address of the picture data packet.
The above storage and retrieval method is only used for picture data, and if different types of data, such as text data, are to be stored in the storage medium, developers need to write different index interfaces according to the above method, which increases the workload of the developers.
In addition, when the method stores the picture, three levels of indexes need to be established for the picture, and when the picture is retrieved, the three levels of indexes need to be sequentially retrieved to determine the position of the picture. The time overhead of retrieval is increased, and the retrieval efficiency is reduced.
Disclosure of Invention
The embodiment of the invention aims to provide a data storage and retrieval method, an electronic device and a storage medium, which are used for storing data by adopting the same storage mode aiming at different types of data, so that the workload of developers is reduced, the time overhead of retrieval in the storage medium is reduced, and the retrieval efficiency is improved. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a data storage method, which is used for a storage medium for storing data, where the storage medium for storing data includes: the system comprises a first-level index area, a second-level index area, a plurality of third-level index areas and a plurality of data storage areas which are in one-to-one correspondence with the third-level index areas;
the first-level index area is used for storing description information of stored data and a corresponding first-level index, and the first-level index is used for identifying storage position information of a storage block where the stored data is located in the data storage area and storage position information of the data storage area in a storage medium;
the secondary index area is used for storing the storage position information of each preset data storage area in the storage medium; the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area;
the method comprises the following steps:
acquiring the data volume of the current data to be stored and the description information of the current data to be stored;
selecting a current data storage area for storing current data to be stored from a plurality of data storage areas, and distributing a current storage block for the current data to be stored in the current data storage area;
obtaining the position information of the current storage block in the current data storage area;
storing the position information of the current storage block in the current data storage area into a third-level index area corresponding to the current data storage area;
acquiring the position information of the current data storage area in the storage medium from the secondary index area;
generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium according to the position information of the current storage block in the current data storage area and the position information of the current data storage area in the storage medium;
correspondingly storing the generated primary index and the description information of the current data to be stored into a primary index area;
and storing the data to be stored to the current storage block.
In a second aspect, a further embodiment of the present invention provides a data retrieval method, which is used for a storage medium storing data, where the storage medium storing data includes: the system comprises a first-level index area, a second-level index area, a plurality of third-level index areas and a plurality of data storage areas which are in one-to-one correspondence with the third-level index areas;
the first-level index area is used for storing description information of stored data and a corresponding first-level index, and the first-level index is used for identifying the storage position of a storage block where the stored data is located in the data storage area and the storage position of the data storage area in a storage medium;
the secondary index area is used for storing the storage position information of each preset data storage area in the storage medium; the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area; the retrieval method comprises the following steps:
obtaining a current retrieval condition;
inquiring current description information of the target data which accords with the current retrieval condition in a primary index area;
aiming at each target data, obtaining a primary index corresponding to the current description information of the target data;
according to the first-level index, the position of a data storage area where the target data is located in the storage medium is obtained from the second-level index area, and the position of a storage block for storing the target data in the data storage area is obtained from the third-level index area;
and reading the target data according to the position of the data storage area where the target data is located in the storage medium and the position of the storage block storing the target data in the data storage area.
In a third aspect, an embodiment of the present invention further provides a data storage method, which is applied to a storage medium for storing data, where the storage medium for storing data includes: the system comprises a first-level index area, a second-level index area, a plurality of third-level index areas and a plurality of data storage areas which are in one-to-one correspondence with the third-level index areas;
the first-level index area is used for storing description information of stored data and a corresponding first-level index, and the first-level index is used for identifying the storage position of a storage block where the stored data is located in the data storage area and the storage position of the data storage area in a storage medium;
the secondary index area is used for storing the storage position information of each preset data storage area in the storage medium; the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area;
the method comprises the following steps:
acquiring current data to be stored;
selecting a current data storage area for storing current data to be stored from a plurality of data storage areas, and distributing a current storage block for the current data to be stored in the current data storage area;
and storing the data to be stored currently into the current storage block.
In a fourth aspect, a further embodiment of the present invention provides an electronic device, including a processor, a program memory, and a data memory;
a program memory for storing a computer program;
a data storage, comprising: the system comprises a first-level index area, a second-level index area, a plurality of third-level index areas and a plurality of data storage areas which are in one-to-one correspondence with the third-level index areas;
the first-level index area is used for storing description information of stored data and a corresponding first-level index, and the first-level index is used for identifying the storage position of a storage block where the stored data is located in the data storage area and the storage position of the data storage area in a storage medium;
the secondary index area is used for storing the storage position information of each preset data storage area in the storage medium; the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area;
and a processor for realizing the steps of the above data storage method or the above data retrieval method when executing the program stored in the program memory.
In a fifth aspect, the embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned data storage method or the steps of the above-mentioned data retrieval method are implemented.
According to the data storage and retrieval method, the electronic device and the storage medium provided by the embodiment of the invention, after the data volume of the current data to be stored and the description information of the current data to be stored are obtained, the current data storage area for storing the current data to be stored is selected from a plurality of data storage areas, the current storage block is allocated to the current data to be stored in the current data storage area, and the current data to be stored can be stored in the current storage block; storing the position information of the current storage block in the current data storage area into a three-level index area corresponding to the current data storage area by obtaining the position information of the current storage block in the current data storage area; acquiring the position information of a current data storage area in a storage medium from a secondary index area, generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium according to the position information of the current storage block in the current data storage area and the position information of the current data storage area in the storage medium, and correspondingly storing the generated primary index and the description information of the current data to be stored in the primary index area; therefore, during subsequent retrieval, the primary index corresponding to the data to be retrieved can be retrieved by retrieving the description information corresponding to the data to be retrieved, and then the position of the storage block storing the data to be retrieved in the data storage area can be determined, wherein the position of the data storage area in the storage medium is further determined. That is to say, the description information is obtained from the data to be stored currently in the storage process, so that the type of the data to be stored currently does not need to be known in advance, that is, business personnel do not need to write different index interfaces for different types of data in advance, and thus the workload of developers can be reduced. Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of a storage medium partition according to an embodiment of the present invention;
fig. 2 is a flowchart of a first implementation of a data storage method according to an embodiment of the present invention;
FIG. 3 is a flowchart of a second implementation of a data storage method according to an embodiment of the present invention;
FIG. 4 is a flowchart of a third implementation of a data storage method according to an embodiment of the present invention;
FIG. 5 is a flowchart of a fourth implementation of a data storage method according to an embodiment of the present invention;
fig. 6 is a flowchart of a fifth implementation manner of a data storage method according to an embodiment of the present invention;
FIG. 7 is a flowchart of a first implementation of a data retrieval method according to an embodiment of the present invention;
FIG. 8 is a flowchart of a second implementation of a data retrieval method according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to solve the problems in the prior art, embodiments of the present invention provide a data storage and retrieval method, an electronic device, and a storage medium, so as to implement that different types of data are stored in the same storage manner, reduce the time overhead of retrieval in the storage medium adopting the storage manner, and improve the retrieval efficiency.
In the following, a method for storing data according to an embodiment of the present invention is first described, where a storage medium for storing data in the method includes: the system comprises a first-level index area, a second-level index area, a plurality of third-level index areas and a plurality of data storage areas which are in one-to-one correspondence with the third-level index areas;
the first-level index area is used for storing description information of stored data and a corresponding first-level index, and the first-level index is used for identifying storage position information of a storage block where the stored data is located in the data storage area and storage position information of the data storage area in a storage medium;
in some examples, the primary index may be an index including storage location information of a storage block in which stored data is located in the data storage area and storage location information of the data storage area in the storage medium, that is, storage location information of a storage block in which stored data is located in the data storage area and storage location information of the data storage area in the storage medium may be directly obtained through the primary index;
in still other examples, the first-level index may be an index that includes a second-level index and a third-level index, which may include: the secondary index may include storage location information of the data storage area in the storage medium. That is to say, the second-level index and the third-level index corresponding to the stored data can be obtained through the first-level index, and further, the storage location information of the storage block where the stored data is located in the data storage area and the storage location information of the data storage area in the storage medium can be obtained through the second-level index and the third-level index corresponding to the stored data.
The secondary index area is used for storing the storage position information of each preset data storage area in the storage medium; the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area.
In order to more clearly illustrate the data storage method according to the embodiment of the present invention, the index area and the data storage area in the storage medium are described below with reference to fig. 1. As shown in fig. 1, which is a schematic diagram of a division of a storage medium in an embodiment of the present invention, in fig. 1, the storage medium may be divided into: a primary index area 110, a secondary index area 120, a plurality of tertiary index areas, and a plurality of data storage areas 130 in one-to-one correspondence with the plurality of tertiary index areas. Wherein, the third-level index area 1 corresponds to the data storage area 1, and the third-level index area n corresponds to the data storage area n. The secondary index area 120 is used for storing the storage location information of the preset data storage areas 1 to n in the storage medium.
Specifically, as shown in fig. 2, which is a flowchart of a first implementation manner of a data storage method according to an embodiment of the present invention, in fig. 2, the method may include:
s210, obtaining the data volume of the current data to be stored and the description information of the current data to be stored.
In some examples, when there is data to be stored in an opposite device connected to an electronic device to which the data storage method according to the embodiment of the present invention is applied, the data size of the data to be stored and description information of the current data to be stored may be first sent to the electronic device to which the data storage method according to the embodiment of the present invention is applied.
In still other examples, the peer device may further send data to be stored to the electronic device, and the electronic device may extract the description information from the data to be stored. After the electronic device acquires the data to be stored and the description information of the data to be stored, the data to be stored can be used as the current data to be stored, and the electronic device can also acquire the data volume of the data to be stored after acquiring the data to be stored. This is all possible.
S220, based on the data amount of the current data to be stored, selecting a current data storage area for storing the current data to be stored from the plurality of data storage areas, and allocating a current storage block for the current data to be stored in the current data storage area.
After the electronic device obtains the current data to be stored, in order to store the current data to be stored, a current data storage area for storing the current data to be stored may be selected from a plurality of data storage areas.
In some examples, the electronic device may randomly select a current data storage area for storing data to be currently stored from the plurality of data storage areas. Or selecting a current data storage area for storing the current data to be stored from the plurality of data storage areas according to a preset selection rule. This is all possible.
After the electronic device selects the current data storage area, a current storage block may be allocated to the current data to be stored in the current data storage area.
Specifically, the electronic device may assign an address in the current data storage area to the current data to be stored as a current storage block of the current data to be stored.
In some examples, the current data storage area may be one or more, and correspondingly, the current storage block may be one or more. This is all possible.
In some examples, the maximum capacity of the plurality of data storage areas may be the same or different.
S230, obtaining the position information of the current storage block in the current data storage area.
After the electronic device obtains the current storage block of the current data to be stored, because the electronic device allocates a segment of address in the current data storage area to the current data to be stored as the current storage block of the current data to be stored, the electronic device can obtain the location information of the current storage block in the current data storage area.
In some examples, the location information may be represented by a first address of the segment address and a length of the segment address, a last address of the segment address and a length of the segment address, or a first address and a last address of the segment address.
In some examples, when there are a plurality of current memory blocks, the location information of the current memory block in the current data storage area may also be a plurality of the current memory blocks.
S240, storing the position information of the current storage block in the current data storage area into a third-level index area corresponding to the current data storage area.
Specifically, the electronic device may store the location information of the current storage block in the current data storage area into the third-level index area corresponding to the current data storage area. So that the position of the data to be retrieved in the current data storage area can be obtained from the three-level index area when retrieving.
In other embodiments, after obtaining the location information of the current storage block in the current data storage area, in order to improve the retrieval efficiency during the retrieval, an index may be established for the location information.
And S250, acquiring the position information of the current data storage area in the storage medium from the secondary index area.
Specifically, since the secondary index area stores preset storage location information of each data storage area in the storage medium, the electronic device may obtain location information of the current data storage area in the storage medium from the secondary index area.
In some examples, when the current data storage area is multiple, the obtained location information of the current storage area in the storage medium may also be multiple.
In some examples, to further improve the retrieval efficiency, an index may be established in the secondary index area in advance for the location information of each data storage area on the storage medium.
S260, according to the position information of the current storage block in the current data storage area and the position information of the current data storage area in the storage medium, a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium is generated.
Further, in order to enable the electronic device to quickly determine the location of the data storage area storing the data to be retrieved in the storage medium during retrieval, and further determine the location of the data to be retrieved in the data storage area, the electronic device may generate a primary index for identifying the location of the current storage block in the current data storage area and the location of the current data storage area in the storage medium according to the location information of the current storage block in the current data storage area and the location information of the current data storage area in the storage medium.
For example, the electronic device may use the location information of the current storage block in the current data storage area as an index of the current storage block in the current data storage area, and store the location information of the current storage block in the third-level index area, and may use the location information of the current data storage area in the storage medium as an index of the current data storage area in the storage medium. And then, generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium according to the index of the current storage block in the current data storage area and the index of the current data storage area in the storage medium.
It should be understood that the process of establishing the primary index by using the data storage method according to the embodiment of the present invention is only an exemplary one, and is not intended to limit the process of establishing the primary index by using the data storage method according to the embodiment of the present invention. The process of generating the primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium based on the position information of the current storage block in the current data storage area and the position information of the current data storage area in the storage medium is within the protection scope of the embodiment of the present invention.
By generating the primary index, the electronic equipment can determine the position of a storage block for storing the data to be retrieved in the data storage area and the position of the data storage area in the storage medium through the primary index when retrieving, so that the retrieval efficiency can be improved.
And S270, correspondingly storing the generated primary index and the description information of the current data to be stored into the primary index area.
Specifically, after the electronic device generates the primary index, in order to enable the primary index corresponding to the data to be retrieved in the subsequent retrieval, the generated primary index and the description information of the data to be currently stored may be stored in the primary index area correspondingly. Therefore, during subsequent retrieval, the first-level index corresponding to the data to be retrieved can be retrieved by retrieving the description information corresponding to the data to be retrieved, and the position of the storage block storing the data to be retrieved in the data storage area, which is the position of the data storage area in the storage medium, can be determined.
In some examples, the description information may include information such as a data type, a data amount, and a keyword of the data to be currently stored.
Compared with the prior art, the data storage method provided by the embodiment of the invention can be used for retrieving the addresses of the second-level index files without at least one retrieval condition or adopting at least one other retrieval condition during subsequent retrieval, retrieving the addresses of the third-level index files from the second-level index files, and determining the positions of the retrieved image data packets without the third-level index files. But directly determines the position of the storage block storing the data to be retrieved in the data storage area in the storage medium through the primary index. The retrieval efficiency can be obviously improved.
And S280, storing the current data to be stored into the current storage block.
According to the data storage method provided by the embodiment of the invention, after the data volume of the current data to be stored and the description information of the current data to be stored are obtained, the current data storage area for storing the current data to be stored is selected from the multiple data storage areas, the current storage block is allocated to the current data to be stored in the current data storage area, and the current data to be stored can be stored in the current storage block; storing the position information of the current storage block in the current data storage area into a three-level index area corresponding to the current data storage area by obtaining the position information of the current storage block in the current data storage area; acquiring the position information of a current data storage area in a storage medium from a secondary index area, generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium according to the position information of the current storage block in the current data storage area and the position information of the current data storage area in the storage medium, and correspondingly storing the generated primary index and the description information of the current data to be stored in the primary index area; therefore, during subsequent retrieval, the primary index corresponding to the data to be retrieved can be retrieved by retrieving the description information corresponding to the data to be retrieved, and then the position of the storage block storing the data to be retrieved in the data storage area can be determined, wherein the position of the data storage area in the storage medium is further determined. That is to say, the description information is obtained from the data to be stored currently in the storage process, so that the type of the data to be stored currently does not need to be known in advance, that is, business personnel do not need to write different index interfaces for different types of data in advance, and thus the workload of developers can be reduced.
In some examples, the secondary index area is further to store an index identifying a location of each data storage area on the storage medium; the index of the position of each data storage area in the storage medium is obtained by performing data processing on the storage position information of each data storage area in the storage medium according to a preset first data processing algorithm.
In some examples, to increase retrieval speed at the time of retrieval, the secondary index area may store an index identifying a location of each data storage area on the storage medium in a tree structure.
Specifically, the pre-processing the storage location information of each data storage area in the storage medium according to a preset first data processing algorithm to obtain the index of each data storage area may include:
and calculating the storage position information of each data storage area in the storage medium by adopting a Hash algorithm to obtain the index of each data storage area.
By adopting the hash algorithm, the storage position information with longer bits can be converted into the hash value with fixed length, and the hash value is used as the index of the data storage area.
In some examples, the storage location information of each data storage area in the storage medium may also be converted in an array manner to obtain a data value, and the array value is used as an index of the data storage area.
On the basis of the data storage method shown in fig. 2, an embodiment of the present invention further provides a possible implementation manner, and as shown in fig. 3, is a flowchart of a second implementation manner of the data storage method according to the embodiment of the present invention, and includes the following steps:
S210-S230 may be identical to steps S210-S230 of FIG. 2, and will not be repeated here.
And S290, according to a preset first data processing algorithm, performing data processing on the position information of the current storage block in the current data storage area to obtain an index of the current storage block in the current data storage area.
Specifically, in order to reduce the occupation of the storage capacity of the storage medium by the three-level index area, the electronic device may process the location information of the current data storage area to obtain an index with a smaller number of bits than the location information of the current data storage area.
In some examples, the location information of the current storage block in the current data storage area is subjected to data processing according to a preset first data processing algorithm, and the index of the current storage block in the current data storage area may be obtained by:
and calculating the position information of the current storage block in the current data storage area by adopting a Hash algorithm to obtain the index of the current storage block in the current data storage area.
S241, correspondingly storing the obtained index of the current storage block in the current data storage area and the position information of the current storage block in the current data storage area into a third-level index area corresponding to the current data storage area.
Specifically, after obtaining the index of the current storage block in the current data storage area, in order to enable the electronic device to retrieve the address of the current storage block in the third-level index area through the obtained index of the current storage block in the current data storage area during subsequent retrieval, the obtained index of the current storage block in the current data storage area and the position information of the current storage block in the current data storage area may be correspondingly stored in the third-level index area corresponding to the current data storage area.
In some examples, when the capacity of the current storage block is represented by the head address information and the tail address information of the current storage block in the current data storage area, the number of the position information of the current storage block in the current data storage area is two, and indexes may be respectively established for the head address information and the tail address information. That is, when the location information of the current storage block in the current data storage area is plural, the obtained index of the current storage block in the current data storage area may be plural.
The plurality of indexes may be stored in the three-level index areas of the corresponding current data storage area, respectively.
And S251, acquiring the index of the current data storage area and the position information of the current data storage area in the storage medium from the secondary index area.
And S261, performing data processing on the index of the current storage block in the current data storage area and the index of the current data storage area according to a preset second data processing algorithm, and generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium.
In some examples, the data processing is performed on the index of the current storage block in the current data storage area and the index of the current data storage area according to a preset second data processing algorithm, and a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium may be generated by:
merging the index of the current storage block in the current data storage area and the index of the current data storage area to obtain merged index data, and taking the merged index data as a primary index, wherein the primary index is used for identifying the position of the current storage block in the current data storage area and the storage position of the current data storage area in a storage medium.
S270 to S280 may be identical to steps S270 to S280 in FIG. 2, and are not repeated here.
In this embodiment, the pre-established index of the position of each data storage area in the storage medium is further stored in the secondary index area, and after the position information of the current storage block in the current data storage area is obtained, the index of the current storage block in the current data storage area is established and stored in the tertiary index area; then, a simple data processing algorithm is adopted, and the index of the current storage block in the current data storage area and the index of the current data storage area are used for calculating a primary index.
Therefore, during subsequent retrieval, the position of the storage block storing the data to be retrieved in the data storage area, which is the position of the data storage area in the storage medium, can be determined through the primary index corresponding to the data to be retrieved. Therefore, the time overhead in subsequent retrieval can be reduced, and the retrieval efficiency is improved.
On the basis of the data storage method shown in fig. 2, an embodiment of the present invention further provides a possible implementation manner, and as shown in fig. 4, is a flowchart of a third implementation manner of a data storage method according to an embodiment of the present invention, including:
s210, obtaining the data volume of the current data to be stored and the description information of the current data to be stored.
S221, selecting a data storage area with the residual storage capacity larger than the data quantity of the current data to be stored from the plurality of data storage areas as a current data storage area.
S222, according to the data volume of the current data to be stored, allocating a storage block which is larger than or equal to the data volume of the current data to be stored in the current data storage area as a current storage block.
The embodiment of the invention also provides a possible implementation mode for distributing the current storage block for the current data to be stored.
Specifically, the electronic device may obtain the data size of the current data to be stored when allocating the current storage block to the current data to be stored. And then selecting a data storage area with the residual storage capacity larger than the data quantity of the data to be stored currently as a current data storage area from the plurality of data storage areas.
By selecting the data storage area with the residual storage capacity larger than the data volume of the current data to be stored as the current data storage area, the current data to be stored can be stored by adopting one current storage block.
After the electronic device obtains the current data storage area, a storage block with a data size larger than or equal to the data size of the current data to be stored can be allocated in the current data storage area as the current storage block.
In some examples, the capacity of the current storage block may be a multiple of the data amount of the data to be currently stored, which may be a number greater than or equal to 1.
S230-S280 may be identical to steps S230-S280 of FIG. 2, and will not be repeated here.
In this embodiment, when the data storage area is selected, a data storage area with a remaining storage amount larger than the data amount of the current data to be stored may be selected from the plurality of data storage areas as a current data storage area; therefore, when the current storage block is allocated, only one current storage block is allocated to the current data to be stored in the current data storage area, and the current data to be stored can be stored. The data to be stored currently can be stored as a whole.
In addition, in this example, when allocating the storage block, the current storage block is allocated according to the data size of the current data to be stored, and compared with the prior art in which the size of the storage block is fixed in advance, the situation of occurrence of storage fragments can be effectively reduced, and the utilization rate of the storage medium is further improved.
On the basis of the data storage method shown in fig. 2, an embodiment of the present invention further provides a possible implementation manner, and as shown in fig. 5, is a flowchart of a fourth implementation manner of a data storage method according to an embodiment of the present invention, and includes:
s210, obtaining the data volume of the current data to be stored and the description information of the current data to be stored.
S223, selecting a data storage area which is not identified as used from the plurality of data storage areas, and selecting a data storage area with the residual storage capacity larger than the data quantity of the current data to be stored as a current data storage area;
s224, according to the data volume of the current data to be stored, allocating a storage block which is larger than or equal to the data volume of the current data to be stored in the current data storage area as a current storage block.
Specifically, when a data storage area added with a used identifier exists in the plurality of data storage areas, when the electronic device allocates a current storage block for current data to be stored, a data storage area which is not identified as used may be selected from the plurality of data storage areas, and a data storage area whose remaining storage capacity is greater than the data amount of the current data to be stored is selected as the current data storage area.
Then, according to the data volume of the current data to be stored, allocating a storage block which is larger than or equal to the data volume of the current data to be stored in the current data storage area as a current storage block.
Through the step, the electronic equipment can select only the data storage area without the used identifier, and the time for selecting the current data storage area is reduced. Therefore, the storage efficiency of storing the current data to be stored can be improved.
S240 to S280 may be identical to steps S240 to S280 of FIG. 2, and will not be repeated here.
In fig. 5, after step S280 is executed to store the data to be currently stored in the current storage block, the method for storing data according to the embodiment of the present invention may further include:
s300, judging whether the data amount stored in the current data storage area reaches a preset threshold value.
If so, step S310 is performed. If not, the process ends.
S310, the current data storage area is marked as used.
Specifically, in order to enable the electronic device to quickly find the current data storage area capable of storing the current data to be stored. After the electronic device stores the current data to be stored, whether the data amount stored in the current data storage area reaches a preset threshold value or not can be judged. If yes, the current data storage area cannot store other data to be stored in the subsequent steps. Therefore, a used identification may be added to the current data storage area to identify that the current data storage area cannot store other data to be stored.
In some examples, the preset threshold may be an empirically preset threshold that may be set to 95% of the total capacity of the current data storage area. The preset threshold of the current data storage area of 20G can also be set to 19.5G for the current data storage area. This is all possible.
In this embodiment, by adding the used identifier to the current data storage area where the stored data amount reaches the preset threshold, the electronic device can exclude the data storage area to which the used identifier is added when storing other data to be stored in the subsequent steps, so that the electronic device can quickly find the data storage area where other data to be stored can be stored, and the speed of storing the data to be stored in the storage medium is improved.
On the basis of the data storage method shown in fig. 2, an embodiment of the present invention further provides a possible implementation manner, and as shown in fig. 6, is a flowchart of a fifth implementation manner of a data storage method according to an embodiment of the present invention, and includes:
s227, selecting a plurality of data storage areas as a plurality of current data storage areas;
and S228, respectively allocating current storage blocks for storing part of the current data to be stored in the plurality of current data storage areas based on the data volume of the current data to be stored.
In some examples, each of the data storage areas has an available capacity, but the available capacity is smaller than the data amount of the data to be currently stored.
Specifically, the electronic device may select a plurality of current data storage areas among the plurality of data storage areas, so as to allocate a current storage block in each current data storage area for a part of data of the data to be currently stored.
After the plurality of current data storage areas are selected, the electronic device may allocate current storage blocks for storing part of the current data to be stored in the plurality of current data storage areas, respectively. That is, each current storage block stores part of the data of the current data to be stored, so as to store the current data to be stored by using a plurality of current storage blocks in different current data storage areas.
S230-S280 may be identical to steps S230-S280 of FIG. 2, and will not be repeated here.
In this embodiment, by allocating current storage blocks for storing part of the current data to be stored in the plurality of current data storage areas, respectively, it is possible to effectively utilize the storage space of the plurality of current data storage areas when each of the plurality of current data storage areas cannot store the current data to be stored. Thereby, the utilization rate of the storage space of the storage medium of the embodiment of the invention can be improved.
Fig. 3 to 6 each describe a different processing procedure in detail. In practical applications, the method is not limited to the embodiments. Those skilled in the art can arrange and combine the steps in different figures according to actual needs to form embodiments, and these embodiments are also within the scope of the embodiments of the present invention.
Corresponding to the above data storage method, an embodiment of the present invention further provides a data retrieval method, where the data retrieval method is used for a storage medium for storing data, and the storage medium includes: the system comprises a first-level index area, a second-level index area, a plurality of third-level index areas and a plurality of data storage areas which are in one-to-one correspondence with the third-level index areas;
the first-level index area is used for storing description information of the stored data and a corresponding first-level index, and the first-level index is used for identifying the storage position of a storage block where the stored data is located in the data storage area and the storage position of the data storage area in the storage medium.
The secondary index area is used for storing the storage position information of each preset data storage area in the storage medium; the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area.
In some examples, the structure of the storage medium may be a partitioned structure of the storage medium as shown in fig. 1.
Specifically, as shown in fig. 7, which is a flowchart of a first implementation manner of a data retrieval method according to an embodiment of the present invention, in fig. 7, the method may include:
s710, obtaining the current searching condition.
Specifically, when a service person queries stored data in the storage medium, a retrieval condition may be input to an electronic device to which the data retrieval method according to the embodiment of the present invention is applied.
After receiving the retrieval condition input by the service personnel, the electronic equipment can take the retrieval condition as the current retrieval condition so as to retrieve the data to be retrieved through the retrieval condition in the subsequent steps.
S720, inquiring the current description information of the target data which accords with the current retrieval condition in the primary index area.
After obtaining the current retrieval condition, the electronic device may query the current description information of the target data that meets the current retrieval condition in the primary index area in the storage medium that adopts the data storage method shown in fig. 2.
For example, assume that the current search condition is: the data type is wma, the data volume is 50M, and the keywords are characteristic data of running red light. The electronic device may query, according to the current retrieval condition, current description information of target data that meets the retrieval condition, "data type is wma, data size is 50M, and keyword is red light running feature data" in the primary index area.
S730, aiming at each target data, a primary index corresponding to the current description information of the target data is obtained.
In some examples, by performing the search according to the search condition, the current description information of one target data may be searched, and the current description information of a plurality of target data may also be searched.
In this regard, the electronic device described above may obtain, for each target data, a primary index corresponding to the current description information of the target data.
S740, according to the first-level index, obtaining the position of the data storage area where the target data is located in the storage medium from the second-level index area, and obtaining the position of the storage block storing the target data in the data storage area from the third-level index area.
Specifically, in the primary index area, the secondary index and the tertiary index can be directly obtained through the primary index. Further, the position of the data storage area in which the target data is located on the storage medium can be obtained in the secondary index area by the secondary index. Through the three-level index, the position of the storage block storing the target data in the data storage area is obtained in the three-level index area.
Compared with the prior art, the data retrieval method in the embodiment of the present invention can determine the secondary index of the data storage area where the target data is located and the tertiary index of the storage block storing the target data directly through the primary index, and then can obtain the location of the data storage area where the target data is located in the storage medium and the location of the storage block storing the target data in the data storage area through the secondary index and the tertiary index. And searching is carried out through a plurality of searching conditions without an index area at each level. Therefore, the time overhead of the retrieval process can be saved, and the retrieval efficiency is improved.
S750, reading the target data according to the position of the data storage area where the target data is located in the storage medium and the position of the storage block storing the target data in the data storage area.
Specifically, after determining the location of the data storage area where the target data is located in the storage medium and the location of the storage block storing the target data in the data storage area, the electronic device may read the target data according to the location of the data storage area where the target data is located in the storage medium and the location of the storage block storing the target data in the data storage area.
The data retrieval method provided by the embodiment of the invention is used for storing a storage medium of data by adopting the data storage method of the embodiment of the invention, when the storage medium is used for storing data, after the data volume of the current data to be stored and the description information of the current data to be stored are obtained, the current data storage area for storing the current data to be stored is selected from a plurality of data storage areas, and a current storage block is allocated to the current data to be stored in the current data storage area, so that the current data to be stored can be stored in the current storage block, and the position information of the current storage block in the current data storage area is stored in a three-level index area corresponding to the current data storage area by obtaining the position information of the current storage block in the current data storage area; acquiring the position information of the current data storage area in the storage medium from the secondary index area, and generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium according to the position information of the current storage block in the current data storage area and the position information of the current data storage area in the storage medium; and correspondingly storing the generated primary index and the description information of the current data to be stored into the primary index area. Therefore, during retrieval, the first-level index corresponding to the data to be retrieved can be retrieved by retrieving the description information corresponding to the data to be retrieved, and the position of the storage block storing the data to be retrieved in the data storage area in the storage medium can be determined. The process of searching by adopting a plurality of searching conditions in each level of index area in the prior art is not carried out. Therefore, the time overhead of the retrieval process can be saved, and the retrieval efficiency is improved.
In some examples, the secondary index area is further to store an index identifying a location of each data storage area on the storage medium; the index of the position of each data storage area in the storage medium is obtained by performing data processing on the storage position information of each data storage area in the storage medium according to a preset first data processing algorithm.
In some examples, the index of the location of each data storage area on the storage medium may be obtained by data processing the storage location information of each data storage area on the storage medium by:
and calculating the storage position information of each data storage area in the storage medium by adopting a Hash algorithm to obtain the index of the position of each data storage area in the storage medium.
In some examples, the electronic device may further calculate storage location information of each data storage area in the storage medium by using an array method, and obtain an index of a location of each data storage area in the storage medium.
Each tertiary index area is also used for storing the index of the position of each storage block in the data storage area corresponding to the tertiary index area; the index of the position of each storage block in the data storage area corresponding to the tertiary index area is as follows: and in the data storage process, according to a preset first data processing algorithm, performing data processing on the position information of the storage block of the currently stored data in the current data storage area to obtain the position information.
In some examples, the electronic device may perform data processing on the location information of the current storage block in the current data storage area to obtain an index of the location of each storage block in the data storage area corresponding to the tertiary index area by:
and calculating the position information of the current storage block in the current data storage area by adopting a Hash algorithm to obtain the index of the position of each storage block in the data storage area corresponding to the third-level index area.
In some examples, the electronic device may further calculate storage location information of each data storage area in the storage medium by using an array method, and obtain an index of a location of each data storage area in the storage medium.
The primary index is: and according to a preset second data processing algorithm, carrying out data processing on the index of the storage block in the data storage area and the index of the data storage area in the storage medium to obtain the index.
In some examples, the electronic device may perform data processing on the index of the storage block in the data storage area and the index of the data storage area in the storage medium to obtain a primary index by:
and merging the index of the storage block in the data storage area and the index of the data storage area in the storage medium to obtain merged index data, and taking the merged index data as a primary index, wherein the primary index is used for identifying the position of the storage block in the data storage area and the storage position of the data storage area in the storage medium.
In an aspect of the embodiment of the present invention, when the service person inputs the search condition to the electronic device, the service person may further input an operation instruction to the electronic device. The operation instruction may be a read instruction, a modify instruction, or a delete instruction.
When the operation command is a read command, the electronic device may read the target data from the storage medium after retrieving the target data corresponding to the retrieval condition.
When the operation instruction is a modification instruction, after the electronic device retrieves the target data corresponding to the retrieval condition, the electronic device may store the new data to be stored, which is input by the service staff, in the storage block in which the target data is stored, so as to modify the target data.
On the basis of the data retrieval method shown in fig. 7, an embodiment of the present invention further provides a possible implementation manner, and as shown in fig. 8, is a flowchart of a second implementation manner of the data retrieval method according to the embodiment of the present invention, and includes:
s741, adopting a second data recovery algorithm corresponding to the second data processing algorithm to recover the data of the first-level index, and obtaining the index of the storage block where the target data is located in the data storage area and the index of the data storage area in the storage medium.
In a possible implementation manner of the embodiment of the present invention, after obtaining the primary index, in order to obtain an index of the storage block where the target data is located in the data storage area and an index of the data storage area in the storage medium, the electronic device may perform data recovery on the primary index by using a second data recovery algorithm corresponding to the second data processing algorithm. Thereby obtaining the index of the storage block where the target data is located in the data storage area and the index of the data storage area in the storage medium.
In some examples, when the electronic device generates the primary index, in addition to the hash algorithm, another preset algorithm may be used, and the other preset algorithm may perform data recovery on the generated primary index.
S742, using a first data recovery algorithm corresponding to the first data processing algorithm, respectively performing data recovery on the index of the storage block where the target data is located in the data storage area and the index of the data storage area in the storage medium, and obtaining the location information of the storage block where the target data is located in the data storage area and the location information of the data storage area in the storage medium.
Accordingly, in order to obtain the location information of the storage block where the target data is located in the data storage area and the location information of the data storage area in the storage medium, the electronic device may perform data recovery on the index of the storage block where the target data is located in the data storage area and the index of the data storage area in the storage medium by using a first data recovery algorithm corresponding to the first data processing algorithm, so as to obtain the location information of the storage block where the target data is located in the data storage area and the location information of the data storage area in the storage medium.
In some examples, when generating the index of the storage block in which the target data is located in the data storage area and the index of the data storage area in the storage medium, the electronic device may further use another preset algorithm in addition to the hash algorithm, where the other preset algorithm may perform data recovery on the generated index of the storage block in which the target data is located in the data storage area and the index of the data storage area in the storage medium.
In this embodiment, by performing data recovery on the first-level index by using the second data recovery algorithm, the index of the storage block in which the target data is located in the data storage area and the index of the data storage area in the storage medium can be obtained. And then, respectively performing data recovery on the index of the storage block where the target data is located in the data storage area and the index of the data storage area in the storage medium by adopting a first data recovery algorithm, so as to obtain the position information of the storage block where the target data is located in the data storage area and the position information of the data storage area in the storage medium. Because the second data processing algorithm and the first data processing algorithm are simple algorithms, the position information of the storage block where the target data is located in the data storage area and the position information of the data storage area in the storage medium can be quickly obtained by adopting the second data recovery algorithm and the first data recovery algorithm. The time overhead in the retrieval process is reduced, and the retrieval efficiency is improved.
An embodiment of the present invention further provides a data storage method, which is applied to a storage medium for storing data shown in fig. 1, where the storage medium includes: a primary index area 110, a secondary index area 120, a plurality of tertiary index areas, and a plurality of data storage areas 130 corresponding to the plurality of tertiary index areas one-to-one;
the primary index area 110 is used for storing description information of stored data and a corresponding primary index, and the primary index is used for identifying a storage location of a storage block where the stored data is located in the data storage area and a storage location of the data storage area in a storage medium;
the secondary index area 120 is used for storing the storage location information of each preset data storage area in the storage medium;
the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area.
The method can comprise the following steps:
step A, obtaining current data to be stored;
b, selecting a current data storage area for storing the current data to be stored from the plurality of data storage areas, and distributing a current storage block for the current data to be stored in the current data storage area;
and C, storing the data to be stored into the current storage block.
In some examples, the storage medium may have a read-write circuit disposed therein, the read-write circuit may be configured to perform a write operation on the current data to be stored, and the read-write circuit may further read data from the storage medium according to a read instruction of the processor.
After the read-write circuit obtains the current data to be stored, the current data to be stored can be stored in the current storage block according to the distributed current storage block.
Because the storage medium for storing the data adopts the first-level index area 110, the second-level index area 120 and the plurality of third-level index areas, and the storage medium further comprises the plurality of data storage areas which are in one-to-one correspondence with the plurality of third-level index areas, the data to be stored can be stored in the storage blocks in the data storage areas when the data to be stored is stored, and further, the first-level index which identifies the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium can be conveniently established. Furthermore, during retrieval, the location of the storage block storing the data to be retrieved in the data storage area, which is the location of the data storage area in the storage medium, can be conveniently and quickly found through the primary index. Thereby improving the efficiency of retrieval.
Specifically, the secondary index area 120 is further used for storing an index of the location of each data storage area on the storage medium; the index of the position of each data storage area in the storage medium is obtained by performing data processing on the storage position information of each data storage area in the storage medium according to a preset first data processing algorithm;
the third index area is further used for storing the index of each storage block in the data storage area; the index of the location of each memory block in its data store is: and in the data storage process, according to a preset first data processing algorithm, performing data processing on the position information of the storage block of the currently stored data in the current data storage area to obtain the position information.
In some examples, the storage medium may store: storing the interface program, deleting the interface program and reading the interface program.
The storage interface is used for storing the data to be stored in the storage medium by adopting the data storage method.
In some examples, the storage interface is further configured to modify the target data in the storage block after retrieving the storage block storing the target data by using the above data retrieving method.
And the deleting interface is used for deleting the target data in the storage block after the storage block storing the target data is retrieved by adopting the data retrieval method so as to release the storage space of the storage medium.
And the reading interface is used for reading the target data in the storage block after the storage block storing the target data is searched by adopting the data searching method.
The storage medium is used for data storage and data retrieval, and the description information and the primary index are stored in a primary index area in the data storage process; during retrieval, the description information of the target data is obtained in the first-level index area according to the target data, the first-level index can be obtained according to the description information of the target data, and the first-level index can identify the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium, so that the position of the storage block for storing the target data in the data storage area and the position of the data storage area in the storage medium can be directly obtained according to the first-level index, the storage position of the target data is obtained, and the target data is read according to the position of the storage block in the data storage area and the position of the data storage area in the storage medium.
An embodiment of the present invention further provides an electronic device, as shown in fig. 9, the electronic device includes a processor 910, a program memory 920 and a data memory 930;
a program memory 920 for storing computer programs;
a data storage 930 comprising: the system comprises a first-level index area, a second-level index area, a plurality of third-level index areas and a plurality of data storage areas which are in one-to-one correspondence with the third-level index areas;
the first-level index area is used for storing description information of stored data and a corresponding first-level index, and the first-level index is used for identifying the storage position of a storage block where the stored data is located in the data storage area and the storage position of the data storage area in a storage medium;
the secondary index area is used for storing the storage position information of each preset data storage area in the storage medium; the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area;
the processor 910 is configured to implement the following steps when executing the program stored in the program storage 920:
acquiring the data volume of the current data to be stored and the description information of the current data to be stored;
selecting a current data storage area for storing current data to be stored from a plurality of data storage areas, and distributing a current storage block for the current data to be stored in the current data storage area;
obtaining the position information of the current storage block in the current data storage area;
storing the position information of the current storage block in the current data storage area into a third-level index area corresponding to the current data storage area;
acquiring the position information of the current data storage area in the storage medium from the secondary index area;
generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium according to the position information of the current storage block in the current data storage area and the position information of the current data storage area in the storage medium;
correspondingly storing the generated primary index and the description information of the current data to be stored into a primary index area;
and storing the data to be stored to the current storage block.
Alternatively, the first and second electrodes may be,
the following searching steps are carried out:
obtaining a current retrieval condition;
inquiring current description information of the target data which accords with the current retrieval condition in a primary index area;
aiming at each target data, obtaining a primary index corresponding to the current description information of the target data;
according to the first-level index, the position of a data storage area where the target data is located in the storage medium is obtained from the second-level index area, and the position of a storage block for storing the target data in the data storage area is obtained from the third-level index area;
and reading the target data according to the position of the data storage area where the target data is located in the storage medium and the position of the storage block storing the target data in the data storage area.
After the data volume of the current data to be stored and the description information of the current data to be stored are obtained, a current data storage area for storing the current data to be stored is selected from a plurality of data storage areas, a current storage block is allocated to the current data to be stored in the current data storage area, and the current data to be stored can be stored in the current storage block; storing the position information of the current storage block in the current data storage area into a three-level index area corresponding to the current data storage area by obtaining the position information of the current storage block in the current data storage area; acquiring the position information of a current data storage area in a storage medium from a secondary index area, generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium according to the position information of the current storage block in the current data storage area and the position information of the current data storage area in the storage medium, and correspondingly storing the generated primary index and the description information of the current data to be stored in the primary index area; therefore, during subsequent retrieval, the primary index corresponding to the data to be retrieved can be retrieved by retrieving the description information corresponding to the data to be retrieved, and then the position of the storage block storing the data to be retrieved in the data storage area, which is the position of the data storage area in the storage medium, can be determined. Therefore, business personnel are not required to write different index interfaces for different types of data, and the workload of developers can be reduced.
The processor 910, the program memory 920 and the data memory 930 in the electronic device may be connected by a communication bus, which may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
Program Memory 920 and data Memory 930 may include Random Access Memory (RAM) and may also include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and when executed by a processor, the computer program implements the following storage steps:
acquiring the data volume of the current data to be stored and the description information of the current data to be stored;
selecting a current data storage area for storing current data to be stored from a plurality of data storage areas, and distributing a current storage block for the current data to be stored in the current data storage area;
obtaining the position information of the current storage block in the current data storage area;
storing the position information of the current storage block in the current data storage area into a third-level index area corresponding to the current data storage area;
acquiring the position information of the current data storage area in the storage medium from the secondary index area;
generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium according to the position information of the current storage block in the current data storage area and the position information of the current data storage area in the storage medium;
correspondingly storing the generated primary index and the description information of the current data to be stored into a primary index area;
and storing the data to be stored to the current storage block.
Or, the following searching steps:
obtaining a current retrieval condition;
inquiring current description information of the target data which accords with the current retrieval condition in a primary index area;
aiming at each target data, obtaining a primary index corresponding to the current description information of the target data;
according to the first-level index, the position of a data storage area where the target data is located in the storage medium is obtained from the second-level index area, and the position of a storage block for storing the target data in the data storage area is obtained from the third-level index area;
and reading the target data according to the position of the data storage area where the target data is located in the storage medium and the position of the storage block storing the target data in the data storage area.
According to the computer-readable storage medium provided by the embodiment of the invention, after the data volume of the current data to be stored and the description information of the current data to be stored are obtained, the current data storage area for storing the current data to be stored is selected from a plurality of data storage areas, a current storage block is allocated to the current data to be stored in the current data storage area, and the current data to be stored can be stored in the current storage block; storing the position information of the current storage block in the current data storage area into a three-level index area corresponding to the current data storage area by obtaining the position information of the current storage block in the current data storage area; acquiring the position information of a current data storage area in a storage medium from a secondary index area, generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium according to the position information of the current storage block in the current data storage area and the position information of the current data storage area in the storage medium, and correspondingly storing the generated primary index and the description information of the current data to be stored in the primary index area; therefore, during subsequent retrieval, the primary index corresponding to the data to be retrieved can be retrieved by retrieving the description information corresponding to the data to be retrieved, and then the position of the storage block storing the data to be retrieved in the data storage area can be determined, wherein the position of the data storage area in the storage medium is determined. Therefore, business personnel are not required to write different index interfaces for different types of data, and the workload of developers can be reduced.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims (13)

1. A method for storing data, wherein a storage medium storing the data comprises: the system comprises a first-level index area, a second-level index area, a plurality of third-level index areas and a plurality of data storage areas which are in one-to-one correspondence with the third-level index areas;
the first-level index area is used for storing description information of stored data and a corresponding first-level index, and the first-level index is used for identifying storage position information of a storage block where the stored data is located in the data storage area and storage position information of the data storage area in the storage medium;
the secondary index area is used for storing the storage position information of each preset data storage area in the storage medium; the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area;
the method comprises the following steps:
acquiring the data volume of the current data to be stored and the description information of the current data to be stored;
selecting a current data storage area for storing the current data to be stored from the plurality of data storage areas based on the data volume of the current data to be stored, and allocating a current storage block for the current data to be stored in the current data storage area;
obtaining the position information of the current storage block in the current data storage area;
storing the position information of the current storage block in the current data storage area into a third-level index area corresponding to the current data storage area;
acquiring the position information of the current data storage area in the storage medium from the secondary index area;
generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium according to the position information of the current storage block in the current data storage area and the position information of the current data storage area in the storage medium;
correspondingly storing the generated primary index and the description information of the current data to be stored into a primary index area;
and storing the current data to be stored into the current storage block.
2. The method of claim 1, wherein the secondary index area is further configured to store an index identifying a location of each data storage area on the storage medium; the index of the position of each data storage area in the storage medium is obtained by performing data processing on the storage position information of each data storage area in the storage medium according to a preset first data processing algorithm;
before the step of storing the location information of the current storage block in the current data storage area into the third-level index area corresponding to the current data storage area, the method further includes:
according to the preset first data processing algorithm, performing data processing on the position information of the current storage block in the current data storage area to obtain an index of the current storage block in the current data storage area;
the step of storing the position information of the current storage block in the current data storage area into the third-level index area corresponding to the current data storage area includes:
correspondingly storing the obtained index of the current storage block in the current data storage area and the position information of the current storage block in the current data storage area into a third-level index area corresponding to the current data storage area;
the step of obtaining the location information of the current data storage area on the storage medium from the secondary index area comprises:
acquiring the index of the current data storage area and the position information of the current data storage area in the storage medium from the secondary index area;
the step of generating a primary index for identifying the location of the current storage block in the current data storage area and the location of the current data storage area in the storage medium according to the location information of the current storage block in the current data storage area and the location information of the current data storage area in the storage medium includes:
and according to the preset second data processing algorithm, performing data processing on the index of the current storage block in the current data storage area and the index of the current data storage area, and generating a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium.
3. The method of claim 2, wherein performing data processing on the storage location information of each data storage area in the storage medium in advance according to a preset first data processing algorithm to obtain an index of each data storage area comprises:
calculating the storage position information of each data storage area in the storage medium by adopting a Hash algorithm to obtain an index of each data storage area;
the step of performing data processing on the position information of the current storage block in the current data storage area according to the preset first data processing algorithm to obtain the index of the current storage block in the current data storage area includes:
calculating the position information of the current storage block in the current data storage area by adopting a Hash algorithm to obtain the index of the current storage block in the current data storage area;
the step of performing data processing on the index of the current storage block in the current data storage area and the index of the current data storage area according to the preset second data processing algorithm to generate a primary index for identifying the position of the current storage block in the current data storage area and the position of the current data storage area in the storage medium includes:
merging the index of the current storage block in the current data storage area and the index of the current data storage area to obtain merged index data, and using the merged index data as the primary index, wherein the primary index is used for identifying the position of the current storage block in the current data storage area and the storage position of the current data storage area in the storage medium.
4. The method according to claim 1, wherein the step of selecting a current data storage area for storing the current data to be stored from the plurality of data storage areas based on the data amount of the current data to be stored, and allocating a current storage block for the current data to be stored in the current data storage area comprises:
selecting a data storage area with the residual storage capacity larger than the data quantity of the current data to be stored from the plurality of data storage areas as a current data storage area;
and according to the data volume of the current data to be stored, allocating a storage block which is larger than or equal to the data volume of the current data to be stored in the current data storage area as a current storage block.
5. The method according to claim 1, further comprising, after the step of storing the current data to be stored into the current storage block:
judging whether the stored data amount of the current data storage area reaches a preset threshold value or not;
if so, identifying the current data store as used;
the step of selecting a current data storage area for storing the current data to be stored from the plurality of data storage areas based on the data volume of the current data to be stored, and allocating a current storage block for the current data to be stored in the current data storage area includes:
selecting a data storage area which is not identified as a used data storage area from the plurality of data storage areas, and selecting a data storage area with the residual storage capacity larger than the data volume of the current data to be stored as a current data storage area;
and according to the data volume of the current data to be stored, allocating a storage block which is larger than or equal to the data volume of the current data to be stored in the current data storage area as a current storage block.
6. The method according to claim 1, wherein the step of selecting a current data storage area for storing the current data to be stored from the plurality of data storage areas based on the data amount of the current data to be stored, and allocating a current storage block for the current data to be stored in the current data storage area comprises:
selecting a plurality of data storage areas as a plurality of current data storage areas;
and respectively allocating current storage blocks for storing part of the current data to be stored in the plurality of current data storage areas based on the data volume of the current data to be stored.
7. A method for retrieving data, wherein a storage medium storing said data comprises: the system comprises a first-level index area, a second-level index area, a plurality of third-level index areas and a plurality of data storage areas which are in one-to-one correspondence with the third-level index areas;
the first-level index area is used for storing description information of stored data and a corresponding first-level index, and the first-level index is used for identifying storage position information of a storage block where the stored data is located in the data storage area and storage position information of the data storage area in the storage medium;
the secondary index area is used for storing the storage position information of each preset data storage area in the storage medium; the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area; the retrieval method comprises the following steps:
obtaining a current retrieval condition;
inquiring current description information of the target data which accords with the current retrieval condition in the primary index area;
for each target data, obtaining a primary index corresponding to the current description information of the target data;
according to the primary index, obtaining the position information of a data storage area where target data are located in the storage medium from the secondary index area, and obtaining the position information of a storage block for storing the target data in the data storage area from the tertiary index area;
and reading the target data according to the position information of the data storage area where the target data is located in the storage medium and the position information of the storage block storing the target data in the data storage area.
8. The method of claim 7, wherein the secondary index area is further configured to store an index identifying a location of each data storage area on the storage medium; the index of the position of each data storage area in the storage medium is obtained by performing data processing on the storage position information of each data storage area in the storage medium according to a preset first data processing algorithm;
each third-level index area is also used for storing the index of the position of each storage block in the data storage area corresponding to the third-level index area; the index of each storage block at the position of the data storage area corresponding to the third-level index area is as follows: in the data storage process, according to the preset first data processing algorithm, performing data processing on the position information of the storage block of the current storage data in the current data storage area to obtain the position information;
the first-level index is: according to a preset second data processing algorithm, carrying out data processing on the index of the storage block in the data storage area and the index of the data storage area in the storage medium to obtain the index;
the step of obtaining the position of the data storage area where the target data is located in the storage medium from the secondary index area and obtaining the position of the storage block where the target data is stored in the data storage area from the tertiary index area according to the primary index includes:
performing data recovery on the primary index by adopting a second data recovery algorithm corresponding to the second data processing algorithm to obtain an index of a storage block where the target data is located in the data storage area and an index of the data storage area in the storage medium;
and respectively performing data recovery on the index of the storage block where the target data is located in the data storage area and the index of the data storage area in the storage medium by adopting a first data recovery algorithm corresponding to the first data processing algorithm, and acquiring the position information of the storage block where the target data is located in the data storage area and the position information of the data storage area in the storage medium.
9. The method according to claim 8, wherein performing data processing on the storage location information of each data storage area in the storage medium in advance according to a preset first data processing algorithm to obtain an index of the location of each data storage area in the storage medium comprises:
calculating the storage position information of each data storage area in the storage medium by adopting a Hash algorithm to obtain the index of the position of each data storage area in the storage medium;
according to the preset first data processing algorithm, data processing is carried out on the position information of the current storage block in the current data storage area, and an index of the position of each storage block in the data storage area corresponding to the third-level index area is obtained, wherein the step comprises the following steps:
calculating the position information of the current storage block in the current data storage area by adopting a Hash algorithm to obtain the index of each storage block in the position of the data storage area corresponding to the third-level index area;
according to the preset second data processing algorithm, data processing is carried out on the index of the storage block in the data storage area and the index of the data storage area in the storage medium to obtain the primary index, and the step comprises the following steps:
merging the index of the storage block in the data storage area and the index of the data storage area in the storage medium to obtain merged index data, and using the merged index data as the primary index, wherein the primary index is used for identifying the position of the storage block in the data storage area and the position of the data storage area in the storage medium.
10. A method for storing data, the method being applied to a storage medium for storing the data, the storage medium for storing the data comprising: the system comprises a first-level index area, a second-level index area, a plurality of third-level index areas and a plurality of data storage areas which are in one-to-one correspondence with the third-level index areas;
the first-level index area is used for storing description information of stored data and a corresponding first-level index, and the first-level index is used for identifying the storage position of a storage block where the stored data is located in the data storage area and the storage position of the data storage area in the storage medium;
the secondary index area is used for storing the storage position information of each preset data storage area in the storage medium; the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area;
the method comprises the following steps:
acquiring current data to be stored;
selecting a current data storage area for storing the current data to be stored from the plurality of data storage areas, and allocating a current storage block for the current data to be stored in the current data storage area;
and storing the data to be stored currently into the current storage block.
11. The method of claim 10, wherein the secondary index area is further configured to store an index of a location of each data storage area on the storage medium; the index of the position of each data storage area in the storage medium is obtained by performing data processing on the storage position information of each data storage area in the storage medium according to a preset first data processing algorithm;
the third index area is further used for storing the index of each storage block in the data storage area; the index of the location of each storage block in its data storage area is: and in the data storage process, according to the preset first data processing algorithm, performing data processing on the position information of the storage block of the currently stored data in the current data storage area to obtain the position information.
12. An electronic device comprising a processor, a program memory and a data memory;
the program memory is used for storing computer programs;
the data storage, comprising: the system comprises a first-level index area, a second-level index area, a plurality of third-level index areas and a plurality of data storage areas which are in one-to-one correspondence with the third-level index areas;
the first-level index area is used for storing description information of stored data and a corresponding first-level index, and the first-level index is used for identifying the storage position of a storage block where the stored data is located in the data storage area and the storage position of the data storage area in the storage medium;
the secondary index area is used for storing the storage position information of each preset data storage area in the storage medium; the third-level index area is used for storing the position information of each storage block of the stored data in the data storage area;
a processor for implementing the method steps of any one of claims 1 to 11 when executing a program stored in the program memory.
13. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method steps of any one of claims 1 to 11.
CN201811323660.8A 2018-11-08 2018-11-08 Data storage and retrieval method, electronic device and storage medium Active CN111159438B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811323660.8A CN111159438B (en) 2018-11-08 2018-11-08 Data storage and retrieval method, electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811323660.8A CN111159438B (en) 2018-11-08 2018-11-08 Data storage and retrieval method, electronic device and storage medium

Publications (2)

Publication Number Publication Date
CN111159438A true CN111159438A (en) 2020-05-15
CN111159438B CN111159438B (en) 2022-06-03

Family

ID=70555458

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811323660.8A Active CN111159438B (en) 2018-11-08 2018-11-08 Data storage and retrieval method, electronic device and storage medium

Country Status (1)

Country Link
CN (1) CN111159438B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112100184A (en) * 2020-10-26 2020-12-18 上海擎感智能科技有限公司 Vehicle total mileage data storage method and computer storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001022270A2 (en) * 1999-09-20 2001-03-29 Ut-Battelle, Llc Method for indexing and retrieving manufacturing-specific digital imagery based on image content
WO2013162954A1 (en) * 2012-04-27 2013-10-31 Netapp, Inc. Efficient data object storage and retrieval
CN103412962A (en) * 2013-09-04 2013-11-27 国家测绘地理信息局卫星测绘应用中心 Storage method and reading method for mass tile data
CN104008111A (en) * 2013-02-27 2014-08-27 深圳市腾讯计算机系统有限公司 Data storage management method and device
CN104363403A (en) * 2014-11-14 2015-02-18 浙江宇视科技有限公司 Method and device for storing video data in video monitoring system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001022270A2 (en) * 1999-09-20 2001-03-29 Ut-Battelle, Llc Method for indexing and retrieving manufacturing-specific digital imagery based on image content
WO2013162954A1 (en) * 2012-04-27 2013-10-31 Netapp, Inc. Efficient data object storage and retrieval
CN104008111A (en) * 2013-02-27 2014-08-27 深圳市腾讯计算机系统有限公司 Data storage management method and device
CN103412962A (en) * 2013-09-04 2013-11-27 国家测绘地理信息局卫星测绘应用中心 Storage method and reading method for mass tile data
CN104363403A (en) * 2014-11-14 2015-02-18 浙江宇视科技有限公司 Method and device for storing video data in video monitoring system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
SHOSHANI等: ""Multidimensional Indexing and Query Coordination for Tertiary Storage Management"", 《 PROCEEDINGS. ELEVENTH INTERNATIONAL CONFERENCE ON SCIENTIFIC AND STATISTICAL DATABASE MANAGEMENT》 *
张新宇等: ""基于三级索引机制的并行数据仓库的存储结构"", 《计算机科学》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112100184A (en) * 2020-10-26 2020-12-18 上海擎感智能科技有限公司 Vehicle total mileage data storage method and computer storage medium

Also Published As

Publication number Publication date
CN111159438B (en) 2022-06-03

Similar Documents

Publication Publication Date Title
US7610468B2 (en) Modified buddy system memory allocation
US11347787B2 (en) Image retrieval method and apparatus, system, server, and storage medium
CN106407207B (en) Real-time newly-added data updating method and device
CN107015985B (en) Data storage and acquisition method and device
CN109669622B (en) File management method, file management device, electronic equipment and storage medium
CN107957848B (en) Deduplication processing method and storage device
CN111061752B (en) Data processing method and device and electronic equipment
US20160335177A1 (en) Cache Management Method and Apparatus
CN107430551B (en) Data caching method, storage control device and storage equipment
CN108614837B (en) File storage and retrieval method and device
CN110858162B (en) Memory management method and device and server
CN110740164B (en) Server determination method, regulation and control method, device, equipment and storage medium
CN107122130B (en) Data deduplication method and device
CN109240607B (en) File reading method and device
CN105243027A (en) Method for storing data in storage device and memory controller
CN111782707A (en) Data query method and system
CN109766318A (en) File reading and device
CN109669621B (en) File management method, file management system, electronic device and storage medium
CN112148690A (en) File caching method, file access request processing method and device
CN111159438B (en) Data storage and retrieval method, electronic device and storage medium
US20180196610A1 (en) Database Memory Monitoring and Defragmentation of Database Indexes
CN106502786A (en) A kind of interrupt distribution method and device
CN113157600A (en) Space allocation method of shingled hard disk, file storage system and server
CN114077690A (en) Vector data processing method, device, equipment and storage medium
CN114528231A (en) Data dynamic storage method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant