CN114661249B - Data storage method and device, computer equipment and storage medium - Google Patents

Data storage method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN114661249B
CN114661249B CN202210578793.XA CN202210578793A CN114661249B CN 114661249 B CN114661249 B CN 114661249B CN 202210578793 A CN202210578793 A CN 202210578793A CN 114661249 B CN114661249 B CN 114661249B
Authority
CN
China
Prior art keywords
storage
data
determining
stored
size
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210578793.XA
Other languages
Chinese (zh)
Other versions
CN114661249A (en
Inventor
张民遐
孙志伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Gaodeng Computer Technology Co ltd
Original Assignee
Shenzhen Gaodeng Computer Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Gaodeng Computer Technology Co ltd filed Critical Shenzhen Gaodeng Computer Technology Co ltd
Priority to CN202210578793.XA priority Critical patent/CN114661249B/en
Publication of CN114661249A publication Critical patent/CN114661249A/en
Application granted granted Critical
Publication of CN114661249B publication Critical patent/CN114661249B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0662—Virtualisation aspects
    • G06F3/0665—Virtualisation aspects at area level, e.g. provisioning of virtual or logical volumes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0674—Disk device
    • G06F3/0676—Magnetic disk device

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to a data storage method, a data storage device, computer equipment and a storage medium. The method comprises the following steps: when the data to be stored is obtained, determining data information of the data to be stored; the data information comprises storage parameters and the data size of data to be stored; determining a target storage strategy corresponding to the data to be stored according to the storage parameters; determining a candidate storage area corresponding to the target storage strategy, and determining an area structure type corresponding to the candidate storage area; and determining a target storage area in the candidate storage areas according to the data size and the area structure type, and storing the data to be stored into the target storage area. By adopting the method, the flexibility of data storage can be improved.

Description

Data storage method and device, computer equipment and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data storage method and apparatus, a computer device, and a storage medium.
Background
With the development of computer technology, technologies for storing data of different types have appeared, and data in a storage space can be read according to different requirements of a user after the data is stored in the corresponding storage space.
At present, the existing data storage mode generally needs to determine the use frequency of different types of data, then store the same type of data into a specified storage space according to the size of the use frequency, and when the storage space reaches a storage threshold, then manually migrate the data exceeding the storage threshold into another storage space. However, since data can only be stored in a designated storage space, the storage manner is single and data isolation between different types of data is caused, and therefore, how to improve flexibility of data storage is a problem to be solved by the present disclosure.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a data storage method, an apparatus, a computer device, a computer readable storage medium, and a computer program product, which can improve flexibility of data storage.
In a first aspect, the present application provides a data storage method. The method comprises the following steps:
when data to be stored is obtained, determining data information of the data to be stored; the data information comprises storage parameters and the data size of the data to be stored;
determining a target storage strategy corresponding to the data to be stored according to the storage parameters;
determining a candidate storage area corresponding to the target storage strategy, and determining an area structure type corresponding to the candidate storage area;
and determining a target storage area in the candidate storage areas according to the data size and the area structure type, and storing the data to be stored into the target storage area.
In one embodiment, the determining, according to the storage parameter, a target storage policy corresponding to the data to be stored includes: when the storage parameter is a first parameter, determining that the target storage strategy is a first storage strategy; the first storage policy is a policy for storing data in a storage space managed by a single logical volume; the determining the candidate storage area corresponding to the target storage policy comprises: and taking each storage space managed by the single logical volume as a candidate storage area corresponding to the first storage strategy.
In one embodiment, the determining, according to the storage parameter, a target storage policy corresponding to the data to be stored includes: when the storage parameter is a second parameter, determining that the target storage strategy is a second storage strategy; the second storage policy is a policy for storing data in a storage space managed by a multi-logical volume; the determining the candidate storage area corresponding to the target storage policy comprises: determining a current logical volume in the plurality of logical volumes, and acquiring a first storage threshold of the current logical volume; the first storage threshold is determined according to the movement factor and the size of the total storage space managed by the current logical volume; determining the size of the historical space occupation managed by the current logical volume; overlapping the historical space occupation size and the data size to obtain the actual space occupation size; and when the actual space occupation size is smaller than or equal to the first storage threshold, taking each storage space managed by the current logical volume as a candidate storage area corresponding to the second storage policy.
In one embodiment, the method further includes: and when the actual space occupation size is larger than the first storage threshold, taking the next logical volume as a new current logical volume, and returning to the step of obtaining the first storage threshold of the current logical volume to continue executing until a candidate storage area is determined.
In one embodiment, the determining, according to the storage parameter, a target storage policy corresponding to the data to be stored includes: when the storage parameter is a third parameter, determining that the target storage strategy is a third storage strategy; the third storage strategy is a strategy for storing data into a preset storage space; the determining the candidate storage area corresponding to the target storage policy comprises: and taking a preset storage space as a candidate storage area.
In one embodiment, the region structure type comprises a disk structure type and a partition structure type; determining a target storage area in the candidate storage areas according to the data size and the area structure type includes: when the area structure type is a disk structure type, determining a target storage area according to the disk space size of the candidate storage area and the data size; and when the region structure type is a partition structure type, determining a target storage region according to the partition space size of the candidate storage region and the data size.
In one embodiment, when the area structure type is a disk structure type, determining a target storage area according to the disk space size of the candidate storage area and the data size includes: determining a current candidate storage area in the plurality of candidate storage areas, and determining a second storage threshold and a historical disk occupation size of the current candidate storage area; overlapping the historical disk occupation size and the data size to obtain the actual disk occupation size; and when the actual disk occupation size is smaller than or equal to the second storage threshold, taking the current candidate storage area as a target storage area.
In one embodiment, the method further includes: and when the actual disk occupation size is larger than the second storage threshold of the current candidate storage area, taking the next candidate storage area as a new current candidate storage area, and returning to the step of determining the second storage threshold of the current candidate storage area and the historical disk occupation size to continue executing until a target storage area is determined.
In one embodiment, the method further includes: and storing the data to be stored into a target storage area, and updating the historical disk occupation size of the target storage area according to the data size of the data to be stored to obtain the updated historical disk occupation size.
In one embodiment, when the region structure type is a partition structure type, determining a target storage region according to the size of the partition space of the candidate storage region and the size of the data includes: determining a current candidate storage area in the plurality of candidate storage areas, and determining a third storage threshold and a historical partition occupation size of the current candidate storage area; overlapping the historical partition occupation size and the data size to obtain an actual partition occupation size; and when the actual partition occupation size is smaller than or equal to the third storage threshold, taking the current candidate storage area as a target storage area.
In one embodiment, the method further includes: and when the actual partition occupation size is larger than the third storage threshold of the current candidate storage area, taking the next candidate storage area as a new current candidate storage area, and returning to the step of determining the third storage threshold of the current candidate storage area and the historical partition occupation size to continue executing until a target storage area is determined.
In a second aspect, the present application further provides a data storage device. The device comprises:
the data acquisition module is used for determining data information of the data to be stored when the data to be stored is acquired; the data information comprises storage parameters and the data size of the data to be stored;
the strategy determining module is used for determining a target storage strategy corresponding to the data to be stored according to the storage parameters;
the region determining module is used for determining a candidate storage region corresponding to the target storage strategy and determining a region structure type corresponding to the candidate storage region; and determining a target storage area in the candidate storage areas according to the data size and the area structure type, and storing the data to be stored into the target storage area.
In a third aspect, the application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the following steps when executing the computer program:
when data to be stored is obtained, determining data information of the data to be stored; the data information comprises storage parameters and the data size of the data to be stored;
determining a target storage strategy corresponding to the data to be stored according to the storage parameters;
determining a candidate storage area corresponding to the target storage strategy, and determining an area structure type corresponding to the candidate storage area;
and determining a target storage area in the candidate storage areas according to the data size and the area structure type, and storing the data to be stored into the target storage area.
In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of:
when data to be stored is obtained, determining data information of the data to be stored; the data information comprises storage parameters and the data size of the data to be stored;
determining a target storage strategy corresponding to the data to be stored according to the storage parameters;
determining a candidate storage area corresponding to the target storage strategy, and determining an area structure type corresponding to the candidate storage area;
and determining a target storage area in the candidate storage areas according to the data size and the area structure type, and storing the data to be stored into the target storage area.
In a fifth aspect, the present application further provides a computer program product. The computer program product comprising a computer program which when executed by a processor performs the steps of:
when data to be stored are obtained, determining data information of the data to be stored; the data information comprises storage parameters and the data size of the data to be stored;
determining a target storage strategy corresponding to the data to be stored according to the storage parameters;
determining a candidate storage area corresponding to the target storage strategy, and determining an area structure type corresponding to the candidate storage area;
and determining a target storage area in the candidate storage areas according to the data size and the area structure type, and storing the data to be stored into the target storage area.
According to the data storage method, the data storage device, the computer equipment, the storage medium and the computer program product, the data to be stored are obtained through acquisition, the data information of the data to be stored is determined, the obtained data information comprises the storage parameters and the data size of the data to be stored, and further the target storage strategy corresponding to the data to be stored can be determined according to the storage parameters; through the target storage strategy, the candidate storage area corresponding to the target storage strategy can be determined, so that the area structure type corresponding to the candidate storage area is determined, and thus the target storage area in the candidate storage area can be determined according to the data size and the area structure type. According to the method and the device, the target storage strategy is determined in advance, and then the target storage area is determined based on the target storage strategy, so that compared with the traditional mode that the data can only be stored in the designated storage space, different target storage strategies can be determined according to the storage parameters, the problem of insufficient flexibility of data storage is solved, and the subsequent reading efficiency of the stored data is improved. Meanwhile, the data can be stored only after the target storage area is determined, so that the utilization efficiency of the storage space is improved.
Drawings
FIG. 1 is a diagram of an exemplary data storage system;
FIG. 2 is a schematic flow chart diagram illustrating a data storage method according to one embodiment;
FIG. 3 is a flowchart illustrating a method for determining candidate storage regions according to one embodiment;
FIG. 4 is a schematic diagram of a data storage method in one embodiment;
FIG. 5 is a block diagram of a data storage device in one embodiment;
FIG. 6 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The data storage method provided by the embodiment of the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104, or may be located on the cloud or other network server. The terminal 102 is configured to obtain data to be stored, and send the data to be stored to the server 104. When the server 104 obtains the data to be stored, the data information of the data to be stored is determined, and the target storage strategy corresponding to the data to be stored is determined according to the data information. The server 104 is further configured to determine a target storage area corresponding to the target storage policy, and store the data to be stored in the target storage area. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices and portable wearable devices, and the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart car-mounted devices, and the like. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, and the like. The server 104 may be implemented as a stand-alone server or a server cluster comprised of multiple servers.
In one embodiment, as shown in fig. 2, a data storage method is provided, which is described by taking the method as an example applied to a computer device, which may be a terminal or a server in fig. 1, and includes the following steps:
step 202, when data to be stored is obtained, determining data information of the data to be stored; the data information includes storage parameters and a data size of data to be stored.
The data to be stored may be configuration data that needs to be stored in advance when the user configures the application, may also be script data stored when the system is tested, may also be document data stored by the user, and the like, and the application is not particularly limited herein. Data to be stored is generally stored in a storage space, and a computer device can read the data in the storage space according to different requirements of a user, for example, script data can be read from the storage space to perform system testing.
Specifically, when the computer device acquires the data to be stored, the data to be stored can be subjected to data analysis, so that data information of the data to be stored is analyzed, that is, the data size of the data to be stored and the storage parameters of the data to be stored are obtained, wherein the unit of the data size can be kilobyte KB, megabyte MB, and the like; the storage parameter is an additional description information of the data to be stored, and is used for determining a storage strategy of the data to be stored.
In one embodiment, the computer device converts the data information of the data to be stored into a temporary variable, i.e. a bytecode, and can treat the temporary variable as an identifier. The computer equipment can realize the storage of the data to be stored by acquiring the temporary variable.
And step 204, determining a target storage strategy corresponding to the data to be stored according to the storage parameters.
Specifically, the computer device determines a target storage policy corresponding to the data to be stored according to a storage parameter of the data to be stored, wherein the target storage policy includes a first storage policy, a second storage policy and a third storage policy. For example, when the storage parameter of the data to be stored is a single volume value, determining that a target storage policy corresponding to the data to be stored is a first storage policy; when the storage parameters of the data to be stored are multi-volume values, determining a target storage strategy corresponding to the data to be stored as a second storage strategy; and when the storage parameter of the data to be stored is a NULL value or a NULL value, determining that the target storage strategy corresponding to the data to be stored is a third storage strategy.
In one embodiment, the computer device displays a storage strategy selection interface and responds to the selection operation of a user on the storage strategy selection interface, and a target storage strategy selected by the selection operation is determined.
In one embodiment, the computer device recommends a target storage strategy corresponding to the data to be stored for the user according to the data size of the data to be stored, and displays the recommended target storage strategy in a storage strategy selection interface.
Step 206, determining a candidate storage area corresponding to the target storage policy, and determining an area structure type corresponding to the candidate storage area.
Specifically, the computer device obtains a correspondence between the target storage policy and the storage space, where the correspondence includes a first correspondence and a second correspondence. When the target storage strategy is a first storage strategy, the computer equipment determines a first storage space corresponding to the first storage strategy according to the first corresponding relation, and takes the first storage space as a candidate storage area; when the target storage strategy is a second storage strategy, the computer equipment determines a second storage space corresponding to the second storage strategy according to the second corresponding relation, and takes the second storage space as a candidate storage area; and when the target storage strategy is the third storage strategy, the computer equipment directly acquires the preset storage space and takes the preset storage space as a candidate storage area.
Further, a structure processing application for analyzing the area structure type of the storage space is run in the computer device, that is, the structure processing application can identify the area structure type of the candidate storage area, and it is determined that the area structure type may include a disk structure type and a partition structure type. The storage space is created in advance, so that the storage spaces with different structure types can be formed, and the storage space can be a disk. For example, if the computer C disk is used as the primary partition and the primary partition occupies all the capacity of the storage space, the storage space is regarded as the disk structure type; if the storage space is divided into a plurality of independent areas, for example, the main partition is a C-disk partition, and the plurality of logical partitions are a D-disk partition and an E-disk partition, different partitions can be used to install an operating system, install an application program, and install a data file, and the storage space at this time is regarded as a partition structure type. It is easy to understand that when data is stored between different disks or different partitions, the data is usually in an isolated state, and the more disks are used or the more partitions are divided, the difficulty in managing the space can be improved.
In one embodiment, when multiple logical partitions are needed for a particular operating system to complete configuration, the computer device prioritizes the partition structure type of storage space as a candidate storage area for configuration data.
And step 208, determining a target storage area in the candidate storage area according to the data size and the area structure type, and storing the data to be stored into the target storage area.
Specifically, when the area structure type of the candidate storage area is the disk structure type, the computer device takes the candidate storage area as the target storage area according to the disk space size and the data size of the candidate storage area, that is, when the disk space corresponding to the candidate storage area can also store the data to be stored. When the area structure type of the candidate storage area is the partition structure type, the computer device takes the candidate storage area as a target storage area according to the partition space size and the data size of the candidate storage area, that is, when the partition space corresponding to the candidate storage area can also store data to be stored.
In one embodiment, when a configuration file of an operating system is stored, the configuration file needs to be subjected to data splitting in advance to obtain at least one piece of sub-configuration data corresponding to the operating system, and position information of each piece of sub-configuration data in the configuration file is determined, wherein each piece of sub-configuration data carries a data identifier which can uniquely identify the configuration data. And for each sub-configuration data in the plurality of sub-configuration data, the computer equipment performs data conversion on the sub-configuration data according to the data type of the current sub-configuration data to obtain a data character string to be stored corresponding to the current sub-configuration data. And the computer equipment respectively determines the target storage areas corresponding to the sub-configuration data according to the data size and the position information of the sub-configuration data, and respectively stores the character strings of the data to be stored corresponding to the sub-configuration data into the corresponding target storage areas. For example, when the data type of the child configuration data includes configuration item data and annotation data, if the location information of the configuration item data is a parent directory location in the configuration file, regarding the candidate storage area 1 as a target storage area 1 corresponding to the configuration item data, and storing a to-be-stored data character string corresponding to the configuration item data into the target storage area 1; and if the position information of the annotation data is the subdirectory position in the configuration file, regarding the candidate storage area 2 as a target storage area 2 corresponding to the annotation data, and storing the character string of the data to be stored corresponding to the annotation data into the target storage area 2.
In one embodiment, when the configuration file needs to be read to run a specific operating system, the computer device reads the sub-configuration data corresponding to the data identifier from the at least one target storage area, and splices the configuration files according to the position information of each sub-configuration data in the configuration files to obtain spliced configuration files.
According to the data storage method, the data to be stored are obtained, the data information of the data to be stored is determined, the obtained data information comprises the storage parameters and the data size of the data to be stored, and then the target storage strategy corresponding to the data to be stored can be determined according to the storage parameters; through the target storage strategy, the candidate storage area corresponding to the target storage strategy can be determined, so that the area structure type corresponding to the candidate storage area is determined, and thus the target storage area in the candidate storage area can be determined according to the data size and the area structure type. According to the method and the device, the target storage strategy is determined in advance, and then the target storage area is determined based on the target storage strategy, so that compared with the traditional mode that the data can only be stored in the designated storage space, different target storage strategies can be determined according to the storage parameters, the problem of insufficient flexibility of data storage is solved, and the subsequent reading efficiency of the stored data is improved. Meanwhile, the data can be stored only after the target storage area is determined, so that the utilization efficiency of the storage space is improved.
In one embodiment, determining a target storage policy corresponding to data to be stored according to a storage parameter includes: when the storage parameter is a first parameter, determining that the target storage strategy is a first storage strategy; the first storage policy is a policy for storing data in a storage space managed by a single logical volume; determining a candidate storage region corresponding to the target storage policy, comprising: each storage space managed by the single logical volume is used as a candidate storage area corresponding to the first storage policy.
The storage spaces with different physical structures can be constructed into a virtual disk form, namely, a single logical volume, and the capacity of the plurality of storage spaces can be dynamically adjusted and managed through the single logical volume. For example, when there is one 8GB disk and one 12GB disk, the entire storage space may be divided into two 10GB logical disks by a single logical volume, or the like; if the 8GB disk and the 12GB disk are of partition structure types, namely both comprise a C disk partition, a D disk partition and an E disk partition, the whole storage space can be divided into 5 4GB logical partitions through a single logical volume.
Specifically, when the storage parameter of the data to be stored is a parameter associated with the single logical volume, the computer device determines that the target storage policy corresponding to the data to be stored is the first storage policy, where the parameter associated with the single logical volume may be a "single volume value" or a user-defined parameter value. And the computer equipment takes each storage space managed by the single logical volume as a candidate storage area corresponding to the first storage strategy. For example, 10GB logical disk 1 and 10GB logical disk 2 managed by a single logic are taken as candidate storage areas.
In one embodiment, the computer device determines, in real time, respective corresponding historical space occupation sizes of storage spaces managed by the single logical volume, determines a difference between the historical space occupation sizes of any two storage spaces, and if the difference is greater than a preset threshold, takes the storage space with the smaller historical space occupation size as a candidate target storage area; and if the difference is smaller than or equal to a preset threshold value, taking the two storage spaces as candidate target storage areas. For example, when the historical space occupation size in the logical disk 1 is 7GB, the historical space occupation size in the logical disk 2 is 2GB, and the difference between the two is greater than the preset threshold 4GB, the logical disk 2 is taken as the candidate target storage area. By the method, the storage data in the storage spaces managed by the single logical volume are balanced, and the probability that one storage space reaches the storage threshold value quickly, but the rest storage spaces are still large in storage capacity is reduced.
In one embodiment, when the computer device executes the target data reading task for the first time, the number of data read from each storage space managed by the single logical volume within a preset time period is determined, and when the number of data read from any storage space exceeds a preset number threshold, it may be considered that too many data required to be read by the target data reading task are stored in the storage space. For example, the target data reading task may be a task that needs to read data within a time threshold that is less than or equal to a preset time threshold.
Further, in order to increase the time consumed for executing the target data reading task again, the single logical volume determines the remaining storage capacity in each storage space, and the storage space is subdivided according to the remaining storage capacity, so that the dynamic adjustment of the storage space capacity is realized. For example, a single logical volume divides a new storage space according to the remaining storage capacity in each storage space, and the computer device determines that data that cannot be read from any storage space within a preset time period is first read out when a target data reading task is executed, and stores the unread data in the new storage space. Illustratively, referring to the above example, when the data read from the logical disk 1 within the preset time period exceeds the preset number threshold, the single logical volume may dynamically divide the logical disk 1 in the storage space into a 7.5GB logical disk, dynamically divide the logical disk 2 in the storable space into a 6.5GB logical disk, and divide a new 6GB logical disk 3 according to the remaining storage capacity 3GB of the logical disk 1 and the remaining storage capacity 8GB of the logical disk 2, and migrate the data that cannot be read from the logical disk 1 within the preset time period into the 6GB logical disk 3 for data storage.
In this embodiment, although the time consumed for executing the target data reading task for the first time may exceed the preset time, the time consumed for subsequently executing the target data reading task may be greatly reduced by dynamically adjusting the storage space, so that the timeliness of the target data reading task is improved.
In the embodiment, the corresponding first storage strategy is determined according to the storage parameters of the data to be stored, so that the data to be stored determines the candidate storage area for data storage based on each storage space managed by the single logical volume, a possible storage strategy is provided for the data to be stored with different storage parameters, and the flexibility of data storage is improved. Meanwhile, the capacity of the candidate storage area managed by the single logical volume can be dynamically adjusted, so that the utilization efficiency of the storage space and the subsequent parallelism of reading the data are improved.
In one embodiment, the storage space managed by the single logical volume needs to be configured in advance, that is, a first correspondence between the first storage policy and the storage space is established. The computer device obtains the logical volume construction requirement information provided by the user, for example, the logical volume construction requirement information may include a logical volume name, logical volume rank information, a logical volume size, a storage space constituting the logical volume, a storage space combination mode, and the like. The computer device constructs the demand information according to the logical volume, and obtains a first corresponding relation between the first storage strategy and the storage space. Similarly, the computer device constructs the requirement information according to the respective corresponding logical volumes of different logical volumes, and obtains a second corresponding relationship between the second storage policy and the storage space.
In the embodiment, by establishing the corresponding relation between the target storage strategy and the storage space, the user can set the attributes of different storage strategies in a user-defined manner, and the expandability of the scheme is enhanced.
In one embodiment, determining a target storage policy corresponding to data to be stored according to a storage parameter includes: when the storage parameter is a second parameter, determining that the target storage strategy is a second storage strategy; the second storage policy is a policy for storing data in a storage space managed by the multi-logical volume; candidate storage regions corresponding to the target storage policy are determined.
Specifically, when the storage parameter of the data to be stored is a parameter associated with the multiple logical volumes, the computer device determines that the target storage policy corresponding to the data to be stored is the second storage policy, where the parameter associated with the multiple logical volumes may be a "multiple volume value" or a user-defined parameter value. The computer device takes each storage space managed by the multi-logical volume as a candidate storage area corresponding to the second storage policy. For example, when there is one 8GB disk, one 12GB disk, and one 15GB disk, the 8GB disk and the 12GB disk may be managed by volume 1 in the multi-logical volume, and the entire storage space is divided into two 10GB logical disks and the like; the 12GB disk and the 15GB disk can be managed by the volume 2 in the multi-logical volume, and the entire storage space is divided into a 13GB logical disk and a 14GB logical disk, and the like, and therefore, two 10GB logical disks, one 13GB logical disk, and one 14GB logical disk in the storage space managed by the multi-logical volume can be used as candidate storage areas.
Further, as shown in fig. 3, determining the candidate storage area corresponding to the target storage policy further includes the following steps:
step 302, determining a current logical volume in the plurality of logical volumes, and acquiring a first storage threshold of the current logical volume; the first storage threshold is determined according to the movement factor and the size of the total storage space managed by the current logical volume.
Specifically, the computer device may determine a current logical volume of the plurality of logical volumes according to the logical volume level information preset by the user, for example, the volume 1 with a high priority may be used as the current logical volume, or any one of the plurality of logical volumes may be randomly selected as the current logical volume. The migration factor is used to characterize a maximum storage condition of data among different logical volumes, for example, when the size of a total storage space managed by the volume 1 is 20GB and the migration factor is 0.2, a first storage threshold of the volume 1 is 16GB, that is, when the size of an occupied storage space in the volume 1 exceeds the first storage threshold after the data a to be stored is stored in the volume 1, the data a to be stored is transferred to a candidate storage area managed by the volume 2 with a low priority for storage.
Step 304, determine the size of the historical space occupation managed by the current logical volume.
And step 306, overlapping the historical space occupation size and the data size to obtain the actual space occupation size.
After the partial data is stored, if the computer device determines that the historical space occupied size managed by the volume 1 is 15GB, the data size of the data to be stored is analyzed to be 0.5GB, and therefore the actual space occupied size corresponding to the volume 1 is 15.5 GB.
And step 308, when the occupied size of the actual space is smaller than or equal to the first storage threshold, taking each storage space managed by the current logical volume as a candidate storage area corresponding to the second storage policy.
Specifically, the computer device determines a difference between the actual space occupation size and a first storage threshold, and when the actual space occupation size is smaller than or equal to the first storage threshold, takes each storage space managed by the current logical volume as a candidate storage area corresponding to the second storage policy. As shown in fig. 4, fig. 4 is a schematic diagram illustrating a data storage method in an embodiment, and when the computer device determines that the target storage policy is the second storage policy, both 10GB logical disks in the storage space managed by the volume 1 are used as candidate storage areas.
In one embodiment, the method further includes: and when the actual space occupation size is larger than the first storage threshold, taking the next logical volume as a new current logical volume, and returning to the step of obtaining the first storage threshold of the current logical volume to continue executing until a candidate storage area is determined.
Specifically, when the actual space occupation size is larger than the first storage threshold, the computer device determines, according to the logical volume level information, the volume 2 with the priority lower than that of the volume 1, and uses the volume 2 as a new current logical volume until one 13GB logical disk and one 14GB logical disk in the storage space managed by the volume 2 are both used as candidate storage areas.
In this embodiment, the corresponding second storage policy is determined according to the storage parameter of the data to be stored, so that the data to be stored can determine the candidate storage areas corresponding to different logical volumes based on multiple logical volumes, thereby providing a possible storage policy for the data to be stored with different storage parameters, and improving the flexibility of data storage. Meanwhile, the polling movement of the data to be stored among different logical disks is realized based on the grade information and the data size among different logical volumes, and the problems of mutual data isolation and low data reading efficiency caused by storing the data in a disk or a partition based on a physical structure in a traditional mode are solved.
In one embodiment, when the computer device acquires a plurality of data to be stored, a plurality of logical volumes may be triggered at the same time, and a candidate storage area corresponding to each logical volume may be determined.
In one embodiment, when the computer device respectively acquires data to be stored from a plurality of terminals, a terminal identifier of the terminal is determined, and a logical volume corresponding to the terminal identifier is triggered.
In one embodiment, the computer device triggers a plurality of logical volumes simultaneously, and if a candidate storage area corresponding to each of the logical volumes includes a special area, a special identifier is marked on data to be stored in the special area, where the special area represents an area commonly managed by at least two logical volumes, that is, a storage space commonly managed. When the computer equipment needs to store the data to be stored in the special area, namely the same data exists in the data to be stored sent by different terminals, the computer equipment generates a corresponding mutual exclusion lock request according to the logic volume corresponding to the terminal identifier and the special identifier of the data to be stored, and sends the mutual exclusion lock request to a mutual exclusion lock pool in the computer equipment. If the mutual exclusion lock corresponding to the mutual exclusion lock request exists in the mutual exclusion lock pool, the mutual exclusion lock is distributed to the data to be stored corresponding to the special identifier, and the data to be stored is managed through the logic volume corresponding to the mutual exclusion lock request; and if the mutual exclusion lock corresponding to the mutual exclusion lock request does not exist in the mutual exclusion lock pool, the data to be stored is indicated to be managed by other logic volumes. And if the data to be stored are the same, the required mutual exclusion locks are the same.
For example, when the computer device obtains a document 1 and a document 2 at the same time, if the data of the document 1 is stored through the storage space managed by the volume 1 and the data of the document 2 is stored through the storage space managed by the volume 2, for both the document 1 and the document 2, the data a to be stored exists, if a mutex lock request 1 is generated according to the special identifier of the logical volume 1 and the data a to be stored, and the mutex lock a corresponding to the mutex lock request 1 is found in the mutex lock pool, it indicates that the data a to be stored is stored and managed by the logical volume 1; if the mutual exclusion lock A is not found in the mutual exclusion lock pool, the mutual exclusion lock A is assigned to the mutual exclusion lock request 2 correspondingly generated by the document 2, and the data A to be stored is indicated to be stored and managed through the logic volume 2.
In this embodiment, for an area commonly managed by multiple logical volumes, after the same data to be stored is obtained from the mutex lock, only one thread may perform a storage operation on the data, thereby avoiding repeated storage of the data, improving utilization efficiency of a storage space, and ensuring flexibility of data storage of a shared storage space.
In one embodiment, when the area structure type is a disk structure type, determining the target storage area according to the disk space size and the data size of the candidate storage area includes: determining a current candidate storage area in the plurality of candidate storage areas, and determining a second storage threshold value and a historical disk occupation size of the current candidate storage area; overlapping the historical disk occupation size and the data size to obtain the actual disk occupation size; and when the actual disk occupation size is smaller than or equal to the second storage threshold, taking the current candidate storage area as the target storage area.
The second storage threshold value can be obtained by setting by a user in advance, and can also be understood as a disk reserved space of a logical disk. For example, when the area structure type is the disk structure type, the second storage threshold of the logical disk 1 managed by the volume 1 is 7GB, and the second storage threshold of the logical disk 2 is 9 GB.
Specifically, the method for determining the current candidate storage area in the plurality of candidate storage areas by the computer device may be determined in a random manner, or may be determined according to the size of the remaining disk space in the candidate storage area, which is not limited herein. After partial data are stored, the computer equipment determines the historical disk occupation size and the second storage threshold value of the current candidate storage area, and superposes the historical disk occupation size and the data size of the data to be processed to obtain the actual disk occupation size of the current candidate storage area. And the computer equipment determines the difference between the actual disk occupation size and the second storage threshold, and takes the current candidate storage area as a target storage area of the data to be stored when the actual disk occupation size is smaller than or equal to the second storage threshold.
For example, when the current candidate storage area is the logical disk 1 managed by the volume 1, if it is determined that the historical disk occupancy size of the logical disk 1 is 6GB and the data size is 0.5GB, the actual disk occupancy size corresponding to the logical disk 1 is 6.5 GB. At this time, the actual disk occupancy size of 6.5GB is smaller than the second storage threshold of 7GB, and therefore the logical disk 1 is taken as the target storage area. Referring to fig. 4, that is, the computer device uses the path 1 corresponding to the logical disk 1 as a storage path of the data to be stored.
In one embodiment, the method further includes: and when the actual disk occupation size is larger than the second storage threshold of the current candidate storage area, taking the next candidate storage area as a new current candidate storage area, and returning to the step of determining the second storage threshold of the current candidate storage area and the historical disk occupation size to continue executing until the target storage area is determined.
When the actual disk occupation size is larger than the second storage threshold of the current candidate storage area, the computer device may use the logical disk 2 managed by the volume 1 as a new current candidate storage area until the target storage area is determined. Referring to fig. 4, that is, the computer device uses the path 2 corresponding to the logical disk 2 as a storage path of the data to be stored.
In one embodiment, the method further includes: and storing the data to be stored into the target storage area, and updating the historical disk occupation size of the target storage area according to the data size of the data to be stored to obtain the updated historical disk occupation size.
In this embodiment, when the candidate storage area is of a disk structure type, the candidate storage areas managed under the current logical volume are screened, so that the target storage area of the data to be stored can be accurately and efficiently determined.
In one embodiment, when the region structure type is a partition structure type, determining the target storage region according to the partition space size and the data size of the candidate storage region includes: determining a current candidate storage area in the plurality of candidate storage areas, and determining a third storage threshold and a historical partition occupation size of the current candidate storage area; overlapping the historical partition occupation size and the data size to obtain the actual partition occupation size; and when the actual partition occupation size is smaller than or equal to the third storage threshold, taking the current candidate storage area as the target storage area.
The third storage threshold may be set by a user in advance, and may also be understood as a partition size of the logical partition. For example, when the area structure type is the partition structure type, the third storage threshold of 5 4GB logical partitions managed by volume 1 is 3 GB. It is to be understood that, when the partition sizes of the logical partitions are the same, the corresponding third storage thresholds are not necessarily the same, and when the partition sizes of the logical partitions are different, the corresponding third storage thresholds are not necessarily different. When the area structure type is the partition structure type, a specific process of determining the target storage area according to the partition space size and the data size of the candidate storage area may refer to the specific process of determining the target storage area according to the disk space size and the data size of the candidate storage area when the area structure type is the disk structure type, which is not described herein again in the embodiments of the present application.
In one embodiment, the method further includes: and when the actual partition occupation size is larger than the third storage threshold of the current candidate storage area, taking the next candidate storage area as a new current candidate storage area, and returning to the step of determining the third storage threshold of the current candidate storage area and the historical partition occupation size to continue executing until the target storage area is determined.
In this embodiment, when the candidate storage area is of a partition structure type, the candidate storage areas managed under the current logical volume are screened, so that the target storage area of the data to be stored can be determined in a targeted and efficient manner.
In one embodiment, determining a target storage policy corresponding to data to be stored according to a storage parameter includes: when the storage parameter is a third parameter, determining that the target storage strategy is a third storage strategy; the third storage strategy is a strategy for storing data into a preset storage space; determining a candidate storage region corresponding to the target storage policy, comprising: and taking a preset storage space as a candidate storage area.
Wherein the third parameter can be used for a preset parameter for distinguishing other strategies, such as a NULL value or a NULL value. The preset storage space may be a specific path preset by the user, for example, a path corresponding to the a file directory in the C disk. When the computer device determines that the storage parameter is the third parameter, it may determine that the target storage policy is the third storage policy, that is, the preset storage space is used as the candidate storage area, and the data to be stored is stored in the preset storage space.
In one embodiment, if the computer device does not resolve the storage parameters of the data to be stored, the third storage policy may be considered as the preferred target storage policy.
In this embodiment, the data to be stored is directly stored in the preset storage space by using the third storage policy, so that the configuration cost of the storage device is reduced and the efficiency of reading the data in the storage space is improved in a scenario where the requirement on the data reading amount is not high.
In one embodiment, another data storage method is provided, in which when obtaining data to be stored, a computer device determines data information of the data to be stored, where the data information includes a storage parameter and a data size of the data to be stored. When the computer device determines that the storage parameter is the first parameter, the target storage policy is determined to be the first storage policy, and each storage space managed by the single logical volume is used as a candidate storage area corresponding to the first storage policy. When the computer equipment determines that the storage parameter is the second parameter, determining that the target storage strategy is the second storage strategy, determining a current logical volume in the plurality of logical volumes, and acquiring a first storage threshold of the current logical volume; the computer equipment determines the historical space occupation size managed by the current logical volume, and superposes the historical space occupation size and the data size to obtain the actual space occupation size, and when the actual space occupation size is smaller than or equal to a first storage threshold value, the computer equipment takes each storage space managed by the current logical volume as a candidate storage area corresponding to a second storage strategy; and when the actual space occupation size is larger than the first storage threshold value, the computer equipment takes the next logical volume as a new current logical volume, and returns to the step of obtaining the first storage threshold value of the current logical volume to continue executing until a candidate storage area is determined. And when the computer equipment determines that the storage parameter is the third parameter, taking the preset storage space as a candidate storage area. The computer equipment determines the area structure type corresponding to the candidate storage area, and when the area structure type is the disk structure type, the computer equipment determines the current candidate storage area in the candidate storage areas, and determines a second storage threshold value and the historical disk occupation size of the current candidate storage area; the computer equipment superposes the historical disk occupation size and the data size to obtain the actual disk occupation size, and when the actual disk occupation size is smaller than or equal to a second storage threshold value, the current candidate storage area is used as a target storage area; and when the actual disk occupation size is larger than the second storage threshold of the current candidate storage area, the computer equipment takes the next candidate storage area as a new current candidate storage area, and returns to the step of determining the second storage threshold of the current candidate storage area and the historical disk occupation size to continue to execute until the target storage area is determined.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be rotated or alternated with other steps or at least a part of the steps or stages in other steps.
Based on the same inventive concept, the embodiment of the present application further provides a data storage device for implementing the above-mentioned data storage method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so specific limitations in one or more embodiments of the data storage device provided below can refer to the limitations on the data storage method in the foregoing, and details are not described herein again.
In one embodiment, as shown in FIG. 5, there is provided a data storage device 500 comprising: a data acquisition module 502, a policy determination module 504, and a region determination module 506, wherein:
a data obtaining module 502, configured to determine data information of data to be stored when the data to be stored is obtained; the data information includes storage parameters and a data size of data to be stored.
And a policy determining module 504, configured to determine, according to the storage parameter, a target storage policy corresponding to the data to be stored.
A region determining module 506, configured to determine a candidate storage region corresponding to the target storage policy, and determine a region structure type corresponding to the candidate storage region; and determining a target storage area in the candidate storage areas according to the data size and the area structure type, and storing the data to be stored into the target storage area.
In one embodiment, the policy determination module 504 includes a first policy module 5041 configured to determine the target storage policy as a first storage policy when the storage parameter is the first parameter; the first storage policy is a policy for storing data in a storage space managed by a single logical volume; determining a candidate storage region corresponding to the target storage policy, comprising: each storage space managed by the single logical volume is used as a candidate storage area corresponding to the first storage policy.
In one embodiment, the policy determination module 504 includes a second policy module 5042, configured to determine the target storage policy as a second storage policy when the storage parameter is the second parameter; the second storage policy is a policy for storing data in a storage space managed by the multi-logical volume; determining a candidate storage region corresponding to the target storage policy, including: determining a current logical volume in the plurality of logical volumes, and acquiring a first storage threshold value of the current logical volume; the first storage threshold value is determined according to the movement factor and the size of the total storage space managed by the current logical volume; determining the size of the historical space occupied by the current logical volume; overlapping the historical space occupation size and the data size to obtain the actual space occupation size; and when the actual space occupation size is smaller than or equal to the first storage threshold, taking each storage space managed by the current logical volume as a candidate storage area corresponding to the second storage strategy.
In one embodiment, the second policy module 5042 is further configured to, when the actual space usage size is greater than the first storage threshold, take the next logical volume as a new current logical volume, and return to the step of obtaining the first storage threshold of the current logical volume to continue executing until the candidate storage area is determined.
In one embodiment, the policy determination module 504 includes a third policy module 5043 configured to determine the target storage policy as a third storage policy when the storage parameter is the third parameter; the third storage strategy is a strategy for storing data into a preset storage space; determining a candidate storage region corresponding to the target storage policy, comprising: and taking a preset storage space as a candidate storage area.
In one embodiment, the zone structure type comprises a disk structure type and a partition structure type; the region determining module 506 includes a structure determining module 5061, configured to determine, when the region structure type is a disk structure type, a target storage region according to a disk space size and a data size of the candidate storage region; and when the area structure type is the partition structure type, determining the target storage area according to the partition space size and the data size of the candidate storage area.
In one embodiment, the region determination module 506 includes a disk space module 5062 configured to determine a current candidate storage region of the plurality of candidate storage regions, and determine a second storage threshold and a historical disk footprint size for the current candidate storage region; overlapping the historical disk occupation size and the data size to obtain the actual disk occupation size; and when the actual disk occupation size is smaller than or equal to the second storage threshold, taking the current candidate storage area as the target storage area.
In one embodiment, the disk space module 5062 is further configured to, when the actual disk occupancy size is greater than the second storage threshold of the current candidate storage area, use the next candidate storage area as a new current candidate storage area, and return to the step of determining the second storage threshold of the current candidate storage area and the historical disk occupancy size to continue to be performed until the target storage area is determined.
In one embodiment, the area determining module 506 further includes a space updating module 5063, configured to store the data to be stored in the target storage area, and update the historical disk occupation size of the target storage area according to the data size of the data to be stored, so as to obtain an updated historical disk occupation size.
In one embodiment, the region determination module 506 further includes a partition space module 5064 configured to determine a current candidate storage region of the plurality of candidate storage regions, and determine a third storage threshold and a historical partition occupancy size for the current candidate storage region; overlapping the historical partition occupation size and the data size to obtain the actual partition occupation size; and when the actual partition occupation size is smaller than or equal to the third storage threshold, taking the current candidate storage area as the target storage area.
In one embodiment, the partition space module 5064 is further configured to, when the actual partition occupancy size is greater than the third storage threshold of the current candidate storage area, use the next candidate storage area as a new current candidate storage area, and return to the step of determining the third storage threshold of the current candidate storage area and the historical partition occupancy size, and continue to be performed until the target storage area is determined.
The various modules in the data storage device described above may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 6. The computer device includes a processor, a memory, an Input/Output interface (I/O for short), and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data to be stored. The input/output interface of the computer device is used for exchanging information between the processor and an external device. The communication interface of the computer device is used for connecting and communicating with an external terminal through a network. The computer program is executed by a processor to implement a data storage method.
Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.
In an embodiment, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps in the above-mentioned method embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), Magnetic Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The databases referred to in various embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims (10)

1. A method of data storage, the method comprising:
when data to be stored is obtained, determining data information of the data to be stored; the data information comprises storage parameters and the data size of the data to be stored;
determining a target storage strategy corresponding to the data to be stored according to the storage parameters;
determining a candidate storage area corresponding to the target storage strategy, and determining an area structure type corresponding to the candidate storage area;
determining a target storage area in the candidate storage areas according to the data size and the area structure type, and storing the data to be stored into the target storage area;
wherein, the determining a target storage policy corresponding to the data to be stored according to the storage parameter includes:
when the storage parameter is a first parameter, determining that the target storage strategy is a first storage strategy; the first storage policy is a policy for storing data in a storage space managed by a single logical volume; the single logical volume can dynamically adjust the capacity of the storage space;
the determining the candidate storage area corresponding to the target storage policy comprises:
and taking each storage space managed by the single logical volume as a candidate storage area corresponding to the first storage strategy.
2. The method of claim 1, wherein the storage space is a disk.
3. The method according to claim 1, wherein the determining a target storage policy corresponding to the data to be stored according to the storage parameter comprises:
when the storage parameter is a second parameter, determining that the target storage strategy is a second storage strategy; the second storage policy is a policy for storing data in a storage space managed by a multi-logical volume;
the determining the candidate storage area corresponding to the target storage policy comprises:
determining a current logical volume in a plurality of logical volumes, and acquiring a first storage threshold of the current logical volume; the first storage threshold is determined according to the movement factor and the size of the total storage space managed by the current logical volume;
determining the size of the historical space occupation managed by the current logical volume;
overlapping the historical space occupation size and the data size to obtain the actual space occupation size;
and when the actual space occupation size is smaller than or equal to the first storage threshold, taking each storage space managed by the current logical volume as a candidate storage area corresponding to the second storage policy.
4. The method of claim 3, further comprising:
and when the actual space occupation size is larger than the first storage threshold, taking the next logical volume as a new current logical volume, and returning to the step of obtaining the first storage threshold of the current logical volume to continue executing until a candidate storage area is determined.
5. The method according to claim 1, wherein the determining a target storage policy corresponding to the data to be stored according to the storage parameter comprises:
when the storage parameter is a third parameter, determining that the target storage strategy is a third storage strategy; the third storage strategy is a strategy for storing data into a preset storage space;
the determining the candidate storage area corresponding to the target storage policy comprises:
and taking a preset storage space as a candidate storage area.
6. The method of claim 1, wherein the zone structure types include a disk structure type and a partition structure type; determining a target storage area in the candidate storage areas according to the data size and the area structure type includes:
when the area structure type is a disk structure type, determining a target storage area according to the disk space size of the candidate storage area and the data size;
and when the region structure type is a partition structure type, determining a target storage region according to the partition space size of the candidate storage region and the data size.
7. The method according to claim 6, wherein when the area structure type is a disk structure type, determining a target storage area according to the disk space size of the candidate storage area and the data size includes:
determining a current candidate storage area in the plurality of candidate storage areas, and determining a second storage threshold value and a historical disk occupation size of the current candidate storage area;
overlapping the historical disk occupation size and the data size to obtain the actual disk occupation size;
and when the actual disk occupation size is smaller than or equal to the second storage threshold, taking the current candidate storage area as a target storage area.
8. A data storage device, characterized in that the device comprises:
the data acquisition module is used for determining data information of the data to be stored when the data to be stored is acquired; the data information comprises storage parameters and the data size of the data to be stored;
the strategy determining module is used for determining a target storage strategy corresponding to the data to be stored according to the storage parameters; wherein, the determining a target storage policy corresponding to the data to be stored according to the storage parameter includes: when the storage parameter is a first parameter, determining that the target storage strategy is a first storage strategy; the first storage policy is a policy for storing data in a storage space managed by a single logical volume; the single logical volume can dynamically adjust the capacity of the storage space; the region determining module is used for determining a candidate storage region corresponding to the target storage strategy and determining a region structure type corresponding to the candidate storage region; the determining the candidate storage area corresponding to the target storage policy comprises: taking each storage space managed by the single logical volume as a candidate storage area corresponding to the first storage policy; and determining a target storage area in the candidate storage areas according to the data size and the area structure type, and storing the data to be stored into the target storage area.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
CN202210578793.XA 2022-05-26 2022-05-26 Data storage method and device, computer equipment and storage medium Active CN114661249B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210578793.XA CN114661249B (en) 2022-05-26 2022-05-26 Data storage method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210578793.XA CN114661249B (en) 2022-05-26 2022-05-26 Data storage method and device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114661249A CN114661249A (en) 2022-06-24
CN114661249B true CN114661249B (en) 2022-09-27

Family

ID=82038466

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210578793.XA Active CN114661249B (en) 2022-05-26 2022-05-26 Data storage method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114661249B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106775488A (en) * 2016-12-27 2017-05-31 维沃移动通信有限公司 The storage method and mobile terminal of a kind of file
CN109408509A (en) * 2018-10-15 2019-03-01 北京金山云网络技术有限公司 Date storage method, device and computer equipment
CN110309328A (en) * 2018-03-14 2019-10-08 深圳云天励飞技术有限公司 Date storage method, device, electronic equipment and storage medium
CN112181303A (en) * 2020-09-29 2021-01-05 广东艾科技术股份有限公司 Data storage method and device, computer equipment and storage medium
CN112306411A (en) * 2020-10-29 2021-02-02 西安万像电子科技有限公司 Data storage method and device, nonvolatile storage medium and processor
CN112684987A (en) * 2021-01-07 2021-04-20 南方电网数字电网研究院有限公司 Data classification storage method and device based on double-core intelligent electric meter
CN112783445A (en) * 2020-11-17 2021-05-11 北京旷视科技有限公司 Data storage method, device, system, electronic equipment and readable storage medium
CN114528231A (en) * 2020-10-30 2022-05-24 北京金山云网络技术有限公司 Data dynamic storage method and device, electronic equipment and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103530236A (en) * 2013-10-21 2014-01-22 华为技术有限公司 Method and device for obtaining hybrid hard disk
CN111597272B (en) * 2020-05-26 2023-10-27 广州迈安信息科技有限公司 Method and device for storing engineering cost data, data server and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106775488A (en) * 2016-12-27 2017-05-31 维沃移动通信有限公司 The storage method and mobile terminal of a kind of file
CN110309328A (en) * 2018-03-14 2019-10-08 深圳云天励飞技术有限公司 Date storage method, device, electronic equipment and storage medium
CN109408509A (en) * 2018-10-15 2019-03-01 北京金山云网络技术有限公司 Date storage method, device and computer equipment
CN112181303A (en) * 2020-09-29 2021-01-05 广东艾科技术股份有限公司 Data storage method and device, computer equipment and storage medium
CN112306411A (en) * 2020-10-29 2021-02-02 西安万像电子科技有限公司 Data storage method and device, nonvolatile storage medium and processor
CN114528231A (en) * 2020-10-30 2022-05-24 北京金山云网络技术有限公司 Data dynamic storage method and device, electronic equipment and storage medium
CN112783445A (en) * 2020-11-17 2021-05-11 北京旷视科技有限公司 Data storage method, device, system, electronic equipment and readable storage medium
CN112684987A (en) * 2021-01-07 2021-04-20 南方电网数字电网研究院有限公司 Data classification storage method and device based on double-core intelligent electric meter

Also Published As

Publication number Publication date
CN114661249A (en) 2022-06-24

Similar Documents

Publication Publication Date Title
US10372723B2 (en) Efficient query processing using histograms in a columnar database
US11082206B2 (en) Layout-independent cryptographic stamp of a distributed dataset
US9367574B2 (en) Efficient query processing in columnar databases using bloom filters
US10515228B2 (en) Commit and rollback of data streams provided by partially trusted entities
EP3264254A1 (en) System and method for a simulation of a block storage system on an object storage system
CN114661249B (en) Data storage method and device, computer equipment and storage medium
CN113849482A (en) Data migration method and device and electronic equipment
US20190377501A1 (en) Data shaping to reduce memory wear in a multi-tenant database
US11170000B2 (en) Parallel map and reduce on hash chains
CN112148461A (en) Application scheduling method and device
CN112860694B (en) Service data processing method, device and equipment
CN116860564B (en) Cloud server data management method and data management device thereof
CN114036171B (en) Application data management method, device, computer equipment and storage medium
US11243939B1 (en) Extended query families
CN117555487A (en) Data splitting method, device, computer equipment and storage medium
CN117455501A (en) Request processing method, apparatus, computer device and storage medium
CN114281238A (en) Data storage method and device
CN115373582A (en) Bloom filter configuration method, device and equipment
CN118331961A (en) Method and device for storing target account in separate storage, computer equipment and readable storage medium
CN114968357A (en) Method and device for generating operation command for multiple storage device versions
US20180210670A1 (en) Storage management of metadata

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant