CN111221468B - Storage block data deleting method and device, electronic equipment and cloud storage system - Google Patents
Storage block data deleting method and device, electronic equipment and cloud storage system Download PDFInfo
- Publication number
- CN111221468B CN111221468B CN201811419430.1A CN201811419430A CN111221468B CN 111221468 B CN111221468 B CN 111221468B CN 201811419430 A CN201811419430 A CN 201811419430A CN 111221468 B CN111221468 B CN 111221468B
- Authority
- CN
- China
- Prior art keywords
- storage
- time
- block
- target
- storage block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/062—Securing storage systems
- G06F3/0623—Securing storage systems in relation to content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the application provides a method and a device for deleting storage block data, electronic equipment and a cloud storage system, wherein the method comprises the following steps: acquiring a target moment and a last writing moment of each storage block; determining a target storage block with the last writing time of the block being smaller than the target time, and deleting data in the target storage block; acquiring the last writing time of the video segments in each storage block; judging the last writing time and the target time of each video segment; and deleting the video segment, wherein the last written time of the video segment is smaller than the target time. In the method for deleting the storage block data, when the storage space of the resource pool is cleaned, the video segment with the last writing time of the video segment being smaller than the target time is deleted, and the video segment is taken as the minimum detection unit, so that the release of the storage space in the resource pool can be increased, and the effective utilization rate of storage resources can be improved. Meanwhile, the time span of data in the same storage block can be reduced, and the time update of the resource pool is facilitated.
Description
Technical Field
The present application relates to the field of data storage technologies, and in particular, to a method and apparatus for deleting storage block data, an electronic device, and a cloud storage system.
Background
In the existing cloud storage system, storage resources are managed through a virtualized resource pool, and the whole architecture is shown in fig. 1. The cloud storage system comprises management nodes and storage nodes, wherein the storage nodes are physical media for storing data, block equipment on the storage nodes is divided into storage blocks according to fixed sizes, and the management nodes logically manage the storage blocks on each storage node through a virtual storage resource pool.
The resource pool is divided into a covered pool and an uncovered pool according to the user demand. The covered pool will be space reclaimed according to the strategy. Specifically, in the overlay pool, the data in the memory block before the designated target time when the memory block is last written is cleared by taking the memory block as a unit, so as to release the memory space.
However, by adopting the method, the release of the storage space in the resource pool is small, and the effective utilization rate of the storage resources is low.
Disclosure of Invention
The embodiment of the application aims to provide a method and a device for deleting storage block data, electronic equipment and a cloud storage system, so as to increase the effective utilization rate of storage resources. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present application provides a method for deleting storage block data, where the method includes:
acquiring a target moment and a last writing moment of each storage block;
determining a target storage block with the last writing time of the block smaller than the target time, and deleting data in the target storage block;
acquiring the last writing time of the video segments in each storage block;
judging the last writing time of each video segment and the target time;
and deleting the video segment of which the last writing time is smaller than the target time.
Optionally, the acquiring the target time and the last writing time of each storage block includes:
and when the preset cleaning time is reached, acquiring the target time and the last writing time of each storage block.
Optionally, the obtaining the last writing time of the video segment of the video segments in each storage block includes:
and acquiring the last writing time of the video segments of each video segment in other storage blocks except the target storage block.
Optionally, after the deleting the video segment and the last writing of the video segment with the time less than the target time, the method further includes:
detecting the storage space utilization rate of each storage block to be sorted, wherein the storage blocks to be sorted are storage blocks in which video segments are deleted in each storage block;
judging the storage space utilization rate of each storage block to be sorted and the size of a preset storage threshold value:
and performing defragmentation on the storage blocks to be defragmented, wherein the storage space utilization rate of the storage blocks is smaller than a preset storage threshold value.
In a second aspect, an embodiment of the present application provides a device for deleting storage block data, including:
the first time acquisition module is used for acquiring a target time and the last writing time of each storage block;
the first data deleting module is used for determining a target storage block with the last writing time of the block smaller than the target time and deleting data in the target storage block;
the second moment acquisition module is used for acquiring the last writing moment of the video segments in each storage block;
the time judging module is used for judging the last writing time of each video segment and the target time;
and the second data deleting module is used for deleting the video segments with the last writing time of the video segments being smaller than the target time.
Optionally, the first time acquisition module is specifically configured to:
and when the preset cleaning time is reached, acquiring the target time and the last writing time of each storage block.
Optionally, the second time obtaining module is specifically configured to:
and acquiring the last writing time of the video segments of each video segment in other storage blocks except the target storage block.
Optionally, the storage block data deleting device of the embodiment of the present application further includes:
the storage space utilization rate detection module is used for detecting the storage space utilization rate of each storage block to be tidied, wherein the storage blocks to be tidied are storage blocks in which video segments are deleted in each storage block;
the threshold judging module is used for judging the storage space utilization rate of each storage block to be sorted and the size of a preset storage threshold:
and the defragmentation module is used for defragmenting the storage blocks to be defragmented, wherein the storage space utilization rate of the storage blocks is smaller than a preset storage threshold value.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory;
the memory is used for storing a computer program;
the processor is configured to implement any one of the methods for deleting storage block data described in the first aspect when executing the program stored in the memory.
In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, in which a computer program is stored, where the computer program is executed by a processor to implement the method for deleting storage block data according to any one of the first aspects.
In a fifth aspect, an embodiment of the present application provides a cloud storage system, including: the system comprises a management node and a storage node, wherein the storage node comprises a storage block;
the storage block is used for storing data;
the management node is configured to implement, for the storage block, the storage block data deletion method according to any one of the first aspect.
The storage block data deleting method, the storage block data deleting device, the electronic equipment and the cloud storage system provided by the embodiment of the application acquire target time and block last writing time of each storage block; determining a target storage block with the last writing time of the block being smaller than the target time, and deleting data in the target storage block; acquiring the last writing time of the video segments in each storage block; judging the last writing time and the target time of each video segment; and deleting the video segment, wherein the last written time of the video segment is smaller than the target time. When the storage space of the resource pool is cleaned, deleting the video segment with the last writing time of the video segment being smaller than the target time, and taking the video segment as the minimum detection unit, the release of the storage space in the resource pool can be increased, and the effective utilization rate of storage resources can be improved. Meanwhile, the time span of data in the same storage block can be reduced, and the time update of the resource pool is facilitated. Of course, it is not necessary for any one product or method of practicing the application to achieve all of the advantages set forth above at the same time.
Drawings
In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of an application scenario of a method for deleting storage block data according to an embodiment of the present application;
FIG. 2 is a schematic diagram of writing time of a video segment in a memory block according to an embodiment of the present application;
FIG. 3 is a flowchart of a method for deleting data in a memory block according to an embodiment of the present application;
FIG. 4 is a flowchart illustrating another method for deleting data in a memory block according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a memory block data deleting apparatus according to an embodiment of the present application;
FIG. 6 is a schematic diagram of an electronic device according to an embodiment of the application;
fig. 7 is a schematic diagram of a cloud storage system according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
First, terms in the embodiments of the present application will be explained:
video cloud storage: in the security industry, video data is continuously stored according to a time sequence and is required to be circularly covered according to a certain period or capacity, and a certain difference exists in the video streaming storage characteristics in the traditional file storage mode. The video cloud storage data storage mode enables video data storage to be more consistent with the characteristics of stream data storage by simplifying the storage flow and optimizing the data block structure, so that the read-write efficiency and the later application expansion are optimized, and the security application can be better served.
CVM (Cloud Virtual Machine, cloud Server): and (5) managing the cloud storage.
CVS (Cloud Video Storage ): and (5) a storage node of cloud storage.
And (3) a resource pool: virtually divided logical storage resources.
Storage block: the minimum unit of space allocation management on the storage node.
And (3) a cavity block: there are two or more small segments of video data and the time between segments is very different.
In the existing cloud storage system, storage resources are managed through a virtualized resource pool, and the whole architecture is shown in fig. 1. The cloud storage system comprises management nodes and storage nodes, wherein the storage nodes are physical media for storing data, block equipment on the storage nodes is divided into storage blocks according to fixed sizes, and the management nodes logically manage the storage blocks on each storage node through a virtual storage resource pool. The resource pool records the number of all storage blocks used in the pool, the minimum time is the resource pool starting time, the maximum time is the resource pool ending time, and the coverage time related to the strategy.
The resource pool is divided into a covered pool and an uncovered pool according to the user demand. The data which is stored in the uncovered pool and is expected to be permanently stored by the user cannot be deleted, and the coverage time of the uncovered pool is always 0. The coverage pool is a pool which can recycle the space according to the strategy and the resources are reused, and the coverage time is the target time in the strategy executed for the last time. The coverage pool is divided into a periodic pool and a capacity pool according to a strategy. The periodic pool corresponds to a storage period, the time for storing the data stored in the storage block is checked regularly, and if the storage time meets the storage period, the cleaning of the storage block is triggered, and the data is deleted; and the capacity coverage pool corresponds to a total storage space size and a capacity threshold, if the used storage blocks reach the capacity threshold, determining a target time for cleaning data according to the data time stored in the resource pool, cleaning the data of the storage blocks with the data storage time before the target time, and deleting the data.
In the existing method for deleting data of a storage block, when the data is cleaned, the data is deleted by taking the data block as a unit. The following will specifically describe the case of fig. 2.
When the space is cleaned, the final cleaning instruction is converted into data before deleting a certain target time T. The current cleaning target time is T, and T4 is less than T5, the storage node searches the end time of the storage block, namely the last writing time of the storage block, and the storage block before T executes block deletion. The end time T2< T of memory block 1, the data in memory block 1 will be deleted, freeing up space. The end time of the storage block 2 and the end time of the storage block 3 are respectively T6 and T8 which are larger than T, and the storage blocks can not be queried and deleted during cleaning. The space of memory block 1 is freed up and the start time of the resource pool is updated to T3.
The flow is as follows: there are two small segments of video data in memory block 2: [ T3, T4] and [ T5, T6], are referred to as hole blocks when the time difference between the two segments is relatively large, and the video of the [ T3, T4] time segment in the block is also out of date and needs to be deleted. However, since the search blocks are all searched according to the storage blocks during the block deletion, the block deletion is only executed if the end time of the storage blocks is expired. Therefore, the expiration data is always stored in the storage node, the starting time of the resource pool can only be updated to be T3, and the starting time cannot be updated normally, so that the estimation of the coverage time of the capacity coverage is influenced.
The video segmentation scene in the storage block 2 is also common, for example, during the process of distributing one storage block to write data, some anomalies (network, power off, etc.) appear, after half of the data are written, the video is written on other storage nodes, after a period of time, the node is redistributed, the video is continuously written, and two blocks with larger time span can be obtained. In extreme cases, a user stopping writing a video or closing a certain storage node and continuing to use it long later can result in a very large time span of such blocks, which can exceed a month or two.
Because of the existence of the blocks, the starting time and even the coverage time of the resource pool cannot be updated, the coverage strategy of the management node for the resource pool can be greatly influenced, the calculation coverage time point in the coverage strategy is lost and is invalid, the space of the resource pool cannot be released, and the user cannot write data in space, so that the data is lost.
In view of this, an embodiment of the present application provides a method for deleting data of a storage block, referring to fig. 3, the method includes:
s301, acquiring target time and block last writing time of each storage block.
The method for deleting the storage block data in the embodiment of the application can be realized through a storage system, and the storage system is any system capable of realizing the method for deleting the storage block data in the embodiment of the application. For example:
the storage system may be a device comprising: a processor, a memory, a communication interface, and a bus; the processor, the memory and the communication interface are connected through a bus and complete communication; the memory stores executable program code; the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory, for executing the memory block data deletion method of the embodiment of the present application.
The storage system may also be an application program for executing the method for deleting storage block data according to the embodiment of the present application at runtime.
The storage system may also be a storage medium for storing executable code for performing the method for deleting storage block data according to the embodiment of the present application.
And when the preset coverage condition is met, the storage system acquires the target moment. The preset coverage condition may be set according to an actual situation, for example, when reaching the preset coverage period, it is determined that the preset coverage condition is satisfied; or when the occupancy rate of the storage space reaches a preset occupancy rate threshold value, judging that a preset coverage condition is met, and the like. The target time is a specified data retention point, namely, the video segment before the target time needs to be deleted, the target video segment after the target time needs to be retained, and the target time can be set according to actual requirements. For example, in the security field, the time required to save the video segment is generally one month, and when the cleaning time is set to be zero point every day, the target time may be zero point one month before the current cleaning time. The last writing time of the block of the storage block, that is, the end time of the storage block, taking fig. 2 as an example, the last writing time of the block of the storage block 1, the block of the storage block 2 and the block of the storage block 3 are sequentially: t2, T6 and T8.
Optionally, the acquiring the target time and the last writing time of each storage block includes: and when the preset cleaning time is reached, acquiring the target time and the last writing time of each storage block. For example, when the resource pool is a periodic pool, a storage period corresponding to the resource pool is acquired, and the storage space of each storage block in the resource pool is released at fixed time. And subtracting the storage period time length on the basis of the reference time by taking the current checking time as the reference time to obtain the target time.
Optionally, the resource pool is a capacity overlay Chi Zishi, and if the total capacity of each used storage block reaches the capacity threshold, the target time for cleaning the data is determined according to the data time stored in the resource pool.
S302, determining a target storage block with the last writing time of the block smaller than the target time, and deleting data in the target storage block.
The storage system judges the sizes of the last writing time and the target time of the blocks of each storage block, and takes the storage block with the last writing time of the block smaller than the target time as the target storage block. The storage system deletes the data in each target storage block. When many video segments exist in the resource pool, the video segments are directly used as basic units for processing, the performance of the system is excessively consumed, and the processing efficiency is affected, so that the data can be deleted by taking the storage block as the basic unit and then the video segments as the basic unit, thereby increasing the processing efficiency.
S303, obtaining the last writing time of the video segment of the video segments in each storage block.
A storage block may include a plurality of video segments, and the storage system may traverse each storage block to obtain a last writing time of the video segment of each video segment in each storage block. For example, the storage block 2 shown in fig. 2 includes two video segments, where the last writing time of the video segments of the two video segments is sequentially T4 and T6.
Optionally, the obtaining the last writing time of the video segment in each storage block includes:
and acquiring the last writing time of the video segments of each video segment in other storage blocks except the target storage block.
The data in the target memory block has been deleted and therefore does not include a video segment. Therefore, the target storage block is not detected, and the last writing time of the video segment can be obtained only in other storage blocks except the target storage block. In the embodiment of the application, the storage system detects the last writing time of the video segments of each video segment in other storage blocks except the target storage block, thereby saving the time for detecting the target storage block and further increasing the processing efficiency.
S304, judging the last writing time of each video segment and the target time.
The storage system judges the last writing time and the target time of the video segments of each video segment in each storage block.
S305, deleting the video segment with the last writing time of the video segment being smaller than the target time.
Aiming at the video segment with the last writing time of deleting the video segment not less than the target time, the storage system does not process the video segment; aiming at the video segment with the last writing time of deleting the video segment being smaller than the target time, the storage system deletes the video segment. Optionally, after deleting the video segment whose last write time is less than the target time, the method further includes: and updating the starting time of the resource pool where each storage block is located as the target time.
In the embodiment of the application, when the storage space of the resource pool is cleaned, the video segment with the last writing time of the video segment smaller than the target time is deleted, and the video segment is taken as the minimum judging unit, so that the release of the storage space in the resource pool can be increased, and the effective utilization rate of storage resources can be improved. The time span of data in the same storage block is reduced, and the time update of the resource pool is facilitated.
Optionally, referring to fig. 4, after the last writing time of the deleted video segment is less than the target time, the method further includes:
s306, detecting the storage space utilization rate of each storage block to be sorted, wherein the storage blocks to be sorted are storage blocks in which video segments are deleted from the storage blocks.
And taking the storage block with the video segments deleted as a storage block to be tidied. The storage system detects the storage space utilization rate of each storage block to be sorted.
S307, judging the storage space utilization rate of each storage block to be sorted and the size of a preset storage threshold.
The preset storage threshold is set according to actual requirements, for example, the preset storage threshold is set to 5%, 10% or 20%, etc.
S308, defragmentation is carried out on the storage blocks to be defragmented, wherein the storage space utilization rate of the storage blocks is smaller than a preset storage threshold value.
If the storage space utilization rate of the storage block to be sorted is smaller than a preset storage threshold value, a defragmentation task is generated, the background asynchronously carries out data migration on the storage block to be sorted, the storage space utilization rate of which is smaller than the preset storage threshold value, and the data migration is carried out on other storage blocks, and the data in the original storage block are deleted, so that space resources are released.
In the embodiment of the application, the storage blocks to be sorted, of which the storage space utilization rate is smaller than the preset storage threshold value, are subjected to defragmentation, so that the data management can be facilitated, more storage space can be released, and the utilization rate of storage resources can be improved.
Next, taking fig. 2 as an example, a method for deleting storage block data according to an embodiment of the present application is illustrated:
when executing the overlay strategy, the CVS firstly searches according to the storage blocks, queries out the blocks with the end time of the storage blocks before the overlay target time T to execute the deletion, wherein the end time T2 of the storage block 1 is less than T, and firstly deletes the data in the storage blocks meeting the conditions when deleting the blocks.
Then, the video segment search is executed, the video segment before the target time is covered is searched, the video segment deletion is executed, at this time, the searched video segments are not particularly many (the segment search deletion is executed directly, the searched segments can be very many, the performance can be affected), the video segments [ T3 and T4] in the storage block 2 are searched and deleted, the block start time of the storage block 2 is updated to be T5, the start time of the corresponding resource pool can be successfully updated to be T5, and the subsequent updating and operation maintenance of the resource pool can be normally performed.
If the segment is successfully deleted, checking the proportion of the effective video segment in the current storage block to the total capacity of the storage block, if the proportion is smaller than a preset storage threshold, generating a defragmentation task, and performing data migration on the defragmentation block by using a background asynchronously to migrate the defragmentation block to a new block, wherein the original defragmentation block performs block deletion to release space resources.
In the embodiment of the application, when the storage space of the resource pool is cleaned, the video segment with the last writing time of the video segment smaller than the target time is deleted, and the video segment is taken as a basic unit, so that the release of the storage space in the resource pool can be increased, and the effective utilization rate of storage resources can be improved. The time span of data in the same storage block is reduced, and the time update of the resource pool is facilitated.
An embodiment of the present application provides a device for deleting data of a storage block, referring to fig. 5, the device includes:
a first time acquisition module 501, configured to acquire a target time and a last writing time of each block of the storage blocks;
a first data deleting module 502, configured to determine a target storage block whose last write time of the block is less than the target time, and delete data in the target storage block;
a second time obtaining module 503, configured to obtain a last writing time of a video segment of the video segments in each storage block;
a time judging module 504, configured to judge the last writing time of each video segment and the target time;
the second data deleting module 505 is configured to delete the video segment whose last writing time is less than the target time.
Optionally, in the storage block data deleting apparatus according to the embodiment of the present application, the first time acquisition module is specifically configured to:
and when the preset cleaning time is reached, acquiring a target time and the last writing time of each storage block.
Optionally, the second time obtaining module 503 is specifically configured to:
and acquiring the last writing time of the video segments of each video segment in other storage blocks except the target storage block.
Optionally, the storage block data deleting device of the embodiment of the present application further includes:
the storage space utilization rate detection module is used for detecting the storage space utilization rate of each storage block to be tidied, wherein the storage blocks to be tidied are storage blocks for deleting video segments in the storage blocks;
the threshold judging module is used for judging the storage space utilization rate of each storage block to be sorted and the size of a preset storage threshold:
and the defragmentation module is used for defragmenting the storage blocks to be defragmented, wherein the storage space utilization rate of the storage blocks is smaller than a preset storage threshold value.
In the embodiment of the application, when the storage space of the resource pool is cleaned, the video segment with the last writing time of the video segment smaller than the target time is deleted, and the video segment is taken as a basic unit, so that the release of the storage space in the resource pool can be increased, and the effective utilization rate of storage resources can be improved. The time span of data in the same storage block is reduced, and the time update of the resource pool is facilitated.
An embodiment of the present application provides an electronic device, referring to fig. 6, including a processor 601 and a memory 602;
the memory 602 is used for storing a computer program;
the processor 601 is configured to execute the program stored in the memory 602, thereby implementing the following steps:
acquiring a target moment and a last writing moment of each storage block;
determining a target storage block with the last writing time of the block smaller than the target time, and deleting data in the target storage block;
acquiring the last writing time of the video segments in each storage block;
judging the last writing time of each video segment and the target time;
and deleting the video segment with the last writing time of the video segment being smaller than the target time.
In the embodiment of the application, when the storage space of the resource pool is cleaned, the video segment with the last writing time of the video segment smaller than the target time is deleted, and the video segment is taken as a basic unit, so that the release of the storage space in the resource pool can be increased, and the effective utilization rate of storage resources can be improved. The time span of data in the same storage block is reduced, and the time update of the resource pool is facilitated.
Optionally, the processor 601 is configured to execute the program stored in the memory 602, and further implement any one of the storage block data deletion methods.
Optionally, the electronic device according to the embodiment of the present application further includes a communication interface and a communication bus, where the processor 601, the communication interface, and the memory 602 complete communication with each other through the communication bus.
Alternatively, the electronic device in the embodiment of the present application may be a CVM.
The communication bus mentioned above for the electronic devices may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the electronic device and other devices.
The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processing, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
An embodiment of the present application provides a cloud storage system, referring to fig. 7, including: a management node 701 and a storage node 702, wherein the storage node 702 includes a storage block;
the storage block is used for storing data;
the management node 701 is configured to implement any one of the storage block data deletion methods described above with respect to the storage block.
Alternatively, management node 701 may be a CVM and storage node 702 may be a CVS. Management node 701 may also logically divide storage node 702 into multiple resource pools, as shown in FIG. 1.
The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the following steps when being executed by a processor:
acquiring a target moment and a last writing moment of each storage block;
determining a target storage block with the last writing time of the block smaller than the target time, and deleting data in the target storage block;
acquiring the last writing time of the video segments in each storage block;
judging the last writing time of each video segment and the target time;
and deleting the video segment with the last writing time of the video segment being smaller than the target time.
In the embodiment of the application, when the storage space of the resource pool is cleaned, the video segment with the last writing time of the video segment smaller than the target time is deleted, and the video segment is taken as a basic unit, so that the release of the storage space in the resource pool can be increased, and the effective utilization rate of storage resources can be improved. The time span of data in the same storage block is reduced, and the time update of the resource pool is facilitated.
Optionally, when the computer program is executed by the processor, any one of the storage block data deleting methods described above can be implemented.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for embodiments of the apparatus, electronic device, cloud storage system, and storage medium, the description is relatively simple as it is substantially similar to the method embodiments, and reference should be made to the description of portions of the method embodiments for relevant points.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.
Claims (7)
1. A method for deleting storage block data, the method comprising:
acquiring a target moment and a last writing moment of each storage block;
determining a target storage block with the last writing time of the block smaller than the target time, and deleting data in the target storage block;
acquiring the last writing time of video segments of each video segment in other storage blocks except the target storage block;
judging the last writing time of each video segment and the target time;
deleting the video segment of which the last writing time is smaller than the target time;
detecting the storage space utilization rate of each storage block to be sorted, wherein the storage blocks to be sorted are storage blocks in which video segments are deleted in each storage block;
judging the storage space utilization rate of each storage block to be sorted and the size of a preset storage threshold value:
and performing defragmentation on the storage blocks to be defragmented, wherein the storage space utilization rate of the storage blocks is smaller than a preset storage threshold value.
2. The method of claim 1, wherein the obtaining the target time and the last block write time of each memory block comprises:
and when the preset cleaning time is reached, acquiring a target time and the last writing time of each storage block.
3. A memory block data deletion apparatus, the apparatus comprising:
the first time acquisition module is used for acquiring a target time and the last writing time of each storage block;
the first data deleting module is used for determining a target storage block with the last writing time of the block smaller than the target time and deleting data in the target storage block;
the second moment acquisition module is used for acquiring the last writing moment of the video segments of each video segment in other storage blocks except the target storage block;
the time judging module is used for judging the last writing time of each video segment and the target time;
the second data deleting module is used for deleting the video segments with the last writing time of the video segments being smaller than the target time;
the storage space utilization rate detection module is used for detecting the storage space utilization rate of each storage block to be tidied, wherein the storage blocks to be tidied are storage blocks in which video segments are deleted in each storage block;
the threshold judging module is used for judging the storage space utilization rate of each storage block to be sorted and the size of a preset storage threshold:
and the defragmentation module is used for defragmenting the storage blocks to be defragmented, wherein the storage space utilization rate of the storage blocks is smaller than a preset storage threshold value.
4. The apparatus of claim 3, wherein the first time acquisition module is specifically configured to:
and when the preset cleaning time is reached, acquiring a target time and the last writing time of each storage block.
5. An electronic device comprising a processor and a memory;
the memory is used for storing a computer program;
the processor is configured to implement the method steps of any of claims 1-2 when executing a program stored on the memory.
6. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-2.
7. A cloud storage system, the cloud storage system comprising: the system comprises a management node and a storage node, wherein the storage node comprises a storage block;
the storage block is used for storing data;
the management node is configured to implement the method for deleting data of a storage block according to any one of claims 1-2 for the storage block.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811419430.1A CN111221468B (en) | 2018-11-26 | 2018-11-26 | Storage block data deleting method and device, electronic equipment and cloud storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811419430.1A CN111221468B (en) | 2018-11-26 | 2018-11-26 | Storage block data deleting method and device, electronic equipment and cloud storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111221468A CN111221468A (en) | 2020-06-02 |
| CN111221468B true CN111221468B (en) | 2023-10-13 |
Family
ID=70827089
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811419430.1A Active CN111221468B (en) | 2018-11-26 | 2018-11-26 | Storage block data deleting method and device, electronic equipment and cloud storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111221468B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463058B (en) * | 2020-11-27 | 2023-04-07 | 杭州海康威视系统技术有限公司 | Fragmented data sorting method and device and storage node |
| CN116521094B (en) * | 2023-07-03 | 2023-11-14 | 之江实验室 | Metadata storage method and device, computer equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101819603A (en) * | 2010-05-07 | 2010-09-01 | 中兴通讯股份有限公司 | Embedded system based method and device for searching information record based on |
| CN106326439A (en) * | 2016-08-26 | 2017-01-11 | 东方网力科技股份有限公司 | Real-time video storage and retrieval method and device |
| CN106469120A (en) * | 2016-08-30 | 2017-03-01 | 华为技术有限公司 | Scrap cleaning method, device and equipment |
| CN107092437A (en) * | 2016-02-17 | 2017-08-25 | 杭州海康威视数字技术股份有限公司 | Data write-in, read method and device, cloud storage system |
| CN107678684A (en) * | 2017-08-22 | 2018-02-09 | 深圳市硅格半导体股份有限公司 | Invalid data sweep-out method, device and the memory of memory |
| CN107967117A (en) * | 2016-10-20 | 2018-04-27 | 杭州海康威视数字技术股份有限公司 | A kind of data storage, reading, method for cleaning, device and cloud storage system |
| CN108153594A (en) * | 2017-12-25 | 2018-06-12 | 联想(北京)有限公司 | The resource fragmentation method for sorting and electronic equipment of a kind of artificial intelligence cloud platform |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100876084B1 (en) * | 2007-02-13 | 2008-12-26 | 삼성전자주식회사 | Computing system capable of delivering deletion information to flash storage |
| US9658799B2 (en) * | 2013-09-25 | 2017-05-23 | International Business Machines Corporation | Data storage device deferred secure delete |
| CN103677977B (en) * | 2013-12-19 | 2017-05-17 | 贝壳网际（北京）安全技术有限公司 | Method and device for cleaning temporary files in mobile terminal and the mobile terminal |
| US10089319B2 (en) * | 2015-02-20 | 2018-10-02 | International Business Machines Corporation | Policy-based, multi-scheme data reduction for computer memory |
| CN106446155A (en) * | 2016-09-22 | 2017-02-22 | 北京百度网讯科技有限公司 | Method and device for cleansingdata in cloud storage system |
-
2018
- 2018-11-26 CN CN201811419430.1A patent/CN111221468B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101819603A (en) * | 2010-05-07 | 2010-09-01 | 中兴通讯股份有限公司 | Embedded system based method and device for searching information record based on |
| CN107092437A (en) * | 2016-02-17 | 2017-08-25 | 杭州海康威视数字技术股份有限公司 | Data write-in, read method and device, cloud storage system |
| CN106326439A (en) * | 2016-08-26 | 2017-01-11 | 东方网力科技股份有限公司 | Real-time video storage and retrieval method and device |
| CN106469120A (en) * | 2016-08-30 | 2017-03-01 | 华为技术有限公司 | Scrap cleaning method, device and equipment |
| CN107967117A (en) * | 2016-10-20 | 2018-04-27 | 杭州海康威视数字技术股份有限公司 | A kind of data storage, reading, method for cleaning, device and cloud storage system |
| CN107678684A (en) * | 2017-08-22 | 2018-02-09 | 深圳市硅格半导体股份有限公司 | Invalid data sweep-out method, device and the memory of memory |
| CN108153594A (en) * | 2017-12-25 | 2018-06-12 | 联想(北京)有限公司 | The resource fragmentation method for sorting and electronic equipment of a kind of artificial intelligence cloud platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111221468A (en) | 2020-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106294206B (en) | Cache data processing method and device | |
| KR102440370B1 (en) | System and method for identifying hot data and stream in a solid-state drive | |
| WO2019062574A1 (en) | Metadata query method and device | |
| EP3252609A1 (en) | Cache data determination method and device | |
| CN110489405B (en) | Data processing method, device and server | |
| CN109086141B (en) | Memory management method and device and computer readable storage medium | |
| US20140325123A1 (en) | Information processing apparatus, control circuit, and control method | |
| CN111221468B (en) | Storage block data deleting method and device, electronic equipment and cloud storage system | |
| CN111198856A (en) | File management method and device, computer equipment and storage medium | |
| CN112631953A (en) | TRIM method and device for solid state disk data, electronic equipment and storage medium | |
| CN107193498B (en) | Method and device for carrying out de-duplication processing on data | |
| CN106708865B (en) | Method and device for accessing window data in stream processing system | |
| CN111240890B (en) | Data processing method, snapshot processing device and computing equipment | |
| CN112463058B (en) | Fragmented data sorting method and device and storage node | |
| CN109189343B (en) | Metadata disk-dropping method, device, equipment and computer-readable storage medium | |
| CN111143113A (en) | Method, electronic device and computer program product for copying metadata | |
| CN112948363A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN110019870B (en) | Image retrieval method and system based on memory image cluster | |
| CN108121514B (en) | Meta information updating method and device, computing equipment and computer storage medium | |
| GB2585543A (en) | Data migration in a hierarchical storage management system | |
| EP3296878B1 (en) | Electronic device and page merging method therefor | |
| CN111488128B (en) | Method, device, equipment and medium for updating metadata | |
| US11429587B1 (en) | Multiple duration deduplication entries | |
| CN110941597B (en) | Method and device for cleaning decompressed file, computing equipment and computer storage medium | |
| CN112685334A (en) | Method, device and storage medium for block caching of data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |