CN112230854A - Method, system, device and medium for managing storage space - Google Patents

Method, system, device and medium for managing storage space Download PDF

Info

Publication number
CN112230854A
CN112230854A CN202011114107.0A CN202011114107A CN112230854A CN 112230854 A CN112230854 A CN 112230854A CN 202011114107 A CN202011114107 A CN 202011114107A CN 112230854 A CN112230854 A CN 112230854A
Authority
CN
China
Prior art keywords
storage space
data
health
processing
data processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202011114107.0A
Other languages
Chinese (zh)
Inventor
任广磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Inspur Intelligent Technology Co Ltd
Original Assignee
Suzhou Inspur Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Inspur Intelligent Technology Co Ltd filed Critical Suzhou Inspur Intelligent Technology Co Ltd
Priority to CN202011114107.0A priority Critical patent/CN112230854A/en
Publication of CN112230854A publication Critical patent/CN112230854A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614Improving the reliability of storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/0223User address space allocation, e.g. contiguous or non contiguous base addressing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a management method of a storage space, which comprises the following steps: acquiring the percentage of the current residual space of each directory in the storage space; calculating a health value by respectively using the weight value corresponding to each catalog and the percentage of the current remaining space; adding the health value of each directory to obtain the health of the storage space; and acquiring and processing the data of the storage space according to a plurality of data processing strategies corresponding to the health degree of the storage space. The invention also discloses a system, a computer device and a readable storage medium. According to the scheme provided by the invention, the weight is set for each directory, so that the data under the directory with low weight can be processed preferentially, and the system is not easy to damage.

Description

Method, system, device and medium for managing storage space
Technical Field
The present invention relates to the field of data processing, and in particular, to a method, a system, a device, and a storage medium for managing a storage space.
Background
In the information age, a large amount of data is generated and needs to be stored. When the amount of data is large, the storage space is always consumed, so that a problem to be solved is to store a large amount of data and ensure that the system does not fail due to the depletion of the storage. At present, data dump or compression is performed, but the space provided by some service scenarios is fixed, historical data can be deleted, and no additional storage is available, and because the amounts of data generated in different periods are different, manual detection is required periodically to prevent system failure caused by space exhaustion.
Disclosure of Invention
In view of the above, in order to overcome at least one aspect of the above problems, an embodiment of the present invention provides a method for managing a storage space, including:
acquiring the percentage of the current residual space of each directory in the storage space;
calculating a health value by respectively using the weight value corresponding to each catalog and the percentage of the current remaining space;
adding the health value of each directory to obtain the health of the storage space;
and acquiring and processing the data of the storage space according to a plurality of data processing strategies corresponding to the health degree of the storage space.
In some embodiments, obtaining and processing the data of the storage space according to a number of data processing policies corresponding to the health of the storage space further comprises:
and in response to the fact that the health degree of the storage space is smaller than a first preset value, sequentially processing the data under the catalogue with the weight value smaller than a threshold value according to the priorities of the data processing strategies.
In some embodiments, acquiring and sequentially processing data in a directory having a weight value smaller than a threshold according to the priorities of the plurality of data processing policies further includes:
screening the data of the catalog with the weight value smaller than the threshold value according to a preset rule;
and processing the data obtained by screening.
In some embodiments, obtaining and processing the data of the storage space according to a number of data processing policies corresponding to the health of the storage space further comprises:
and in response to the fact that the health degree of the storage space is smaller than a first preset value, processing the data under the catalogue with the weight value smaller than the threshold value by using the plurality of data processing strategies.
In some embodiments, the processing the data under the directory with the weight value less than the threshold value by using the plurality of data processing strategies simultaneously further comprises:
screening the data under the catalog with the weight value smaller than the threshold value according to a preset rule;
and processing the data obtained by screening.
In some embodiments, further comprising:
and responding to the fact that the health degree of the storage space is larger than a first preset value, and not performing data processing.
In some embodiments, further comprising:
and receiving a data processing strategy set by a user by using a preset interface.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a management system of a storage space, including:
an obtaining module configured to obtain a percentage of a current remaining space of each directory in a storage space;
a calculating module configured to calculate a health value using the weight value corresponding to each of the directories and the percentage of the current remaining space, respectively;
an adding module configured to add the health value of each of the directories to obtain the health of the storage space;
the data processing module is configured to acquire and process the data of the storage space according to a plurality of data processing strategies corresponding to the health degree of the storage space.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer apparatus, including:
at least one processor; and
a memory storing a computer program operable on the processor, wherein the processor executes the program to perform any of the steps of the method for managing storage space as described above.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which when executed by a processor performs the steps of any of the above-described methods for managing storage space.
The invention has one of the following beneficial technical effects: according to the scheme provided by the invention, the weight is set for each directory, so that the data under the directory with low weight can be processed preferentially, and the system is not easy to damage.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a method for managing a storage space according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a system for managing storage space according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a computer device provided in an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
According to an aspect of the present invention, an embodiment of the present invention provides a method for managing a storage space, as shown in fig. 1, which may include the steps of:
s1, acquiring the percentage of the current remaining space of each directory in the storage space;
s2, calculating health values by respectively using the weight value corresponding to each catalog and the percentage of the current remaining space;
s3, adding the health value of each directory to obtain the health degree of the storage space;
and S4, acquiring and processing the data of the storage space according to a plurality of data processing strategies corresponding to the health degree of the storage space.
According to the scheme provided by the invention, the weight is set for each directory, so that the data under the directory with low weight can be processed preferentially, and the system is not easy to damage. For example, the system includes 2 directories, 1C directory, and 1D directory. When the C directory is about to be consumed and the D directory space remains a lot, if a weight value is not set for each directory, when space recovery is performed, it is possible to recover the more important C directory data, but the D directory is mainly used by the user. After the weight is set for the directory, the space recovery is not performed because the space of the D directory is more remained and the health degree is higher.
In some embodiments, obtaining and processing the data of the storage space according to a number of data processing policies corresponding to the health of the storage space further comprises:
and in response to the fact that the health degree of the storage space is smaller than a first preset value, sequentially processing the data under the catalogue with the weight value smaller than a threshold value according to the priorities of the data processing strategies.
Specifically, the data in the directory with the weight value smaller than the threshold may be processed according to the priority of the policy. For example, the data processing policies may be a data compression policy and a data deletion policy, and the data compression policy has a higher priority than the data deletion policy. In this way, the data compression policy can be preferentially executed, if the health degree of the storage space is greater than the first preset value, the subsequent policy is not executed, and if the health degree of the storage space is still less than the first preset value after the policy with the high priority is executed, the subsequent policy is executed until the requirement of the health degree is met.
In some embodiments, acquiring and sequentially processing data in a directory having a weight value smaller than a threshold according to the priorities of the plurality of data processing policies further includes:
screening the data of the catalog with the weight value smaller than the threshold value according to a preset rule;
and processing the data obtained by screening.
Specifically, when data processing is performed, in order to prevent damage to data in use, the data may be screened, for example, to determine whether a file is in use, and if not, may be processed if not (configurable) for a specified time. In some embodiments, data compression may be based on Huffman tree coding. When the data deletion policy is executed, a path may be designated for deletion or a file type may be designated for deletion.
In some embodiments, obtaining and processing the data of the storage space according to a number of data processing policies corresponding to the health of the storage space further comprises:
and in response to the fact that the health degree of the storage space is smaller than a first preset value, processing the data under the catalogue with the weight value smaller than the threshold value by using the plurality of data processing strategies.
Specifically, data under a directory with a weight value smaller than a threshold value can be processed in parallel by multiple strategies. For example, the plurality of data processing policies may be a data compression policy and a data deletion policy, and the data compression policy and the data deletion policy may be executed simultaneously.
In some embodiments, the processing the data under the directory with the weight value less than the threshold value by using the plurality of data processing strategies simultaneously further comprises:
screening the data under the catalog with the weight value smaller than the threshold value according to a preset rule;
and processing the data obtained by screening.
In some embodiments, the health degrees are different, and the corresponding data processing strategies are different, for example, when the health degree is smaller than a first preset value and larger than a second preset value, compression processing may be performed on the history file of the low-weight directory; when the health degree is smaller than the second preset value and larger than the third preset value, the deletion processing and the like of the non-high-weight directory can be selected.
In some embodiments, further comprising:
and responding to the fact that the health degree of the storage space is larger than a first preset value, and not performing data processing.
Specifically, the weights may be divided according to the importance of different directory structures, for example, the weight value of the system-level directory is 30, the weight value of the application-level directory is 20, and the weight value of the directory such as the application temporary file log is 10. And calculating the system space health degree according to the weights of different catalogues and the current space remaining percentage, and when the health degree is higher than a preset value, indicating that the system is very healthy and data processing can be omitted.
In some embodiments, further comprising:
and receiving a data processing strategy set by a user by using a preset interface.
Specifically, the user may select to edit a default policy or a new policy, and after the configuration of the data processing policy is completed, may select a storage module currently supported by the system to store the policy (for example, a magnetic disk, a removable hard disk, a flash memory, or other media).
In some embodiments, when a user manages data of a storage space, whether threshold setting is needed or not can be confirmed according to own requirements, if the user does not select, the user defaults to use a space health degree detection mechanism to manage the data of the storage space, namely, a target storage directory structure is analyzed, and weight division is performed according to importance degrees of different directory structures; calculating the health degree based on the percentage and the weight of the residual space of each directory; and selecting data processing strategies according to the health degree value. If the user sets the threshold, space management can be performed in conjunction with the threshold set by the customer.
According to the scheme provided by the invention, the weight is set for each directory, so that the data under the directory with low weight can be processed preferentially, and the system is not easy to damage.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a system 400 for managing storage space, as shown in fig. 2, including:
an obtaining module 401, where the obtaining module 401 is configured to obtain a percentage of a current remaining space of each directory in the storage space;
a calculating module 402, wherein the calculating module 402 is configured to calculate a health value by using the weight value corresponding to each directory and the percentage of the current remaining space;
an adding module 403, wherein the adding module 403 is configured to add the health value of each directory to obtain the health of the storage space;
a data processing module 404, wherein the data processing module 404 is configured to obtain and process data of the storage space according to a plurality of data processing strategies corresponding to the health degree of the storage space.
In some embodiments, the data processing module 404 is further configured to:
and in response to the fact that the health degree of the storage space is smaller than a first preset value, sequentially processing the data under the catalogue with the weight value smaller than a threshold value according to the priorities of the data processing strategies.
In some embodiments, the data processing module 404 is further configured to:
screening the data of the catalog with the weight value smaller than the threshold value according to a preset rule;
and processing the data obtained by screening.
In some embodiments, the data processing module 404 is further configured to:
and in response to the fact that the health degree of the storage space is smaller than a first preset value, processing the data under the catalogue with the weight value smaller than the threshold value by using the plurality of data processing strategies.
In some embodiments, the data processing module 404 is further configured to:
screening the data under the catalog with the weight value smaller than the threshold value according to a preset rule;
and processing the data obtained by screening.
In some embodiments, the data processing module 404 is further configured to:
and responding to the fact that the health degree of the storage space is larger than a first preset value, and not performing data processing.
In some embodiments, the system further comprises an interface module configured to:
and receiving a data processing strategy set by a user by using a preset interface.
According to the scheme provided by the invention, the weight is set for each directory, so that the data under the directory with low weight can be processed preferentially, and the system is not easy to damage.
Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 3, an embodiment of the present invention further provides a computer apparatus 501, comprising:
at least one processor 520; and
the memory 510, the memory 510 stores a computer program 511 that can be run on the processor, and the processor 520 executes the program to perform the steps of any of the above methods for managing memory space.
Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 4, an embodiment of the present invention further provides a computer-readable storage medium 601, where the computer-readable storage medium 601 stores computer program instructions 610, and the computer program instructions 610, when executed by a processor, perform the steps of any of the above methods for managing storage space.
Finally, it should be noted that, as will be understood by those skilled in the art, all or part of the processes of the methods of the above embodiments may be implemented by a computer program to instruct related hardware to implement the methods.
Further, it should be appreciated that the computer-readable storage media (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps of implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.

Claims (10)

1. A method for managing storage space, comprising the steps of:
acquiring the percentage of the current residual space of each directory in the storage space;
calculating a health value by respectively using the weight value corresponding to each catalog and the percentage of the current remaining space;
adding the health value of each directory to obtain the health of the storage space;
and acquiring and processing the data of the storage space according to a plurality of data processing strategies corresponding to the health degree of the storage space.
2. The method of claim 1, wherein obtaining and processing data of the storage space according to a number of data processing policies corresponding to the health of the storage space further comprises:
and in response to the fact that the health degree of the storage space is smaller than a first preset value, sequentially processing the data under the catalogue with the weight value smaller than a threshold value according to the priorities of the data processing strategies.
3. The method of claim 2, wherein obtaining and sequentially processing data under a directory having a weight value less than a threshold according to priorities of the plurality of data processing policies further comprises:
screening the data of the catalog with the weight value smaller than the threshold value according to a preset rule;
and processing the data obtained by screening.
4. The method of claim 1, wherein obtaining and processing data of the storage space according to a number of data processing policies corresponding to the health of the storage space further comprises:
and in response to the fact that the health degree of the storage space is smaller than a first preset value, processing the data under the catalogue with the weight value smaller than the threshold value by using the plurality of data processing strategies.
5. The method of claim 4, wherein the processing data under the category having a weight value less than a threshold using the plurality of data processing policies at the same time further comprises:
screening the data under the catalog with the weight value smaller than the threshold value according to a preset rule;
and processing the data obtained by screening.
6. The method of claim 2 or 4, further comprising:
and responding to the fact that the health degree of the storage space is larger than a first preset value, and not performing data processing.
7. The method of claim 1, further comprising:
and receiving a data processing strategy set by a user by using a preset interface.
8. A system for managing storage space, comprising:
an obtaining module configured to obtain a percentage of a current remaining space of each directory in a storage space;
a calculating module configured to calculate a health value using the weight value corresponding to each of the directories and the percentage of the current remaining space, respectively;
an adding module configured to add the health value of each of the directories to obtain the health of the storage space;
the data processing module is configured to acquire and process the data of the storage space according to a plurality of data processing strategies corresponding to the health degree of the storage space.
9. A computer device, comprising:
at least one processor; and
memory storing a computer program operable on the processor, wherein the processor executes the program to perform the steps of the method according to any of claims 1-7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, is adapted to carry out the steps of the method according to any one of claims 1 to 7.
CN202011114107.0A 2020-10-18 2020-10-18 Method, system, device and medium for managing storage space Withdrawn CN112230854A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011114107.0A CN112230854A (en) 2020-10-18 2020-10-18 Method, system, device and medium for managing storage space

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011114107.0A CN112230854A (en) 2020-10-18 2020-10-18 Method, system, device and medium for managing storage space

Publications (1)

Publication Number Publication Date
CN112230854A true CN112230854A (en) 2021-01-15

Family

ID=74117891

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011114107.0A Withdrawn CN112230854A (en) 2020-10-18 2020-10-18 Method, system, device and medium for managing storage space

Country Status (1)

Country Link
CN (1) CN112230854A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023138264A1 (en) * 2022-01-21 2023-07-27 苏州浪潮智能科技有限公司 Ssd data management method and related component

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023138264A1 (en) * 2022-01-21 2023-07-27 苏州浪潮智能科技有限公司 Ssd data management method and related component

Similar Documents

Publication Publication Date Title
CN104025057B (en) Collaborative storage management
CN109981702B (en) File storage method and system
WO2018006721A1 (en) Method and apparatus for storing log file
US8554736B2 (en) Backup method
CN113625945A (en) Distributed storage slow disk processing method, system, terminal and storage medium
US9996599B2 (en) Using access count of the remote site to optimize file transfer order for asynchronous replication
WO2009122528A1 (en) Integrated configuration management device, disparate configuration management device, and backup data management system
CN112230854A (en) Method, system, device and medium for managing storage space
CN112130761A (en) Capacity expansion method and system for storage system and related device
CN104199687B (en) A kind of method and apparatus of the processing data in dummy machine system
CN111126965B (en) Auditing rule optimization method, auditing rule optimization device, computer equipment and storage medium
CN108363727A (en) A kind of date storage method and device based on ZFS file system
US20230109530A1 (en) Synchronous object placement for information lifecycle management
CN109684279B (en) Data processing method and system
CN115756333A (en) Abnormal node processing method and device, metadata server and storage medium
CN107154960B (en) Method and apparatus for determining service availability information for distributed storage systems
CN116820323A (en) Data storage method, device, electronic equipment and computer readable storage medium
CN109144503B (en) Method, device and equipment for storing data through Redox and readable storage medium
CN113918638A (en) Data processing link determining method, system, device and storage medium
CN112596959A (en) Distributed storage cluster data backup method and device
CN112269763A (en) File aggregation method and related device
CN116055464B (en) Download preservation path selection method, device and medium
US9912727B1 (en) Multiple concurrent in-flight replies in a distributed state system
CN109324887B (en) Auditing data parallel processing method and system
CN112596952A (en) Cluster data processing method, system, device and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication
WW01 Invention patent application withdrawn after publication

Application publication date: 20210115