WO2017206436A1 - 网络硬盘空间的回收方法、装置和系统 - Google Patents
网络硬盘空间的回收方法、装置和系统 Download PDFInfo
- Publication number
- WO2017206436A1 WO2017206436A1 PCT/CN2016/104494 CN2016104494W WO2017206436A1 WO 2017206436 A1 WO2017206436 A1 WO 2017206436A1 CN 2016104494 W CN2016104494 W CN 2016104494W WO 2017206436 A1 WO2017206436 A1 WO 2017206436A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- storage
- data
- strip
- unrecovered
- space
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Definitions
- the present application relates to the field of communication technology applications, and in particular, to a method, device and system for recovering a network hard disk space.
- the storage concept in the conventional sense has also been updated.
- the virtual storage concept such as network hard disk and cloud disk expands the storage type of storage technology.
- Computer local storage that is, a computer-local solid-state hard disk performs data reading and writing functions through I/O and a host.
- the user uploads the local data in the physical machine of the user to the virtual storage provided by the network server by accessing the network server; similarly, in the data reading During the fetching process, the user downloads the corresponding file data from the virtual storage by accessing the corresponding virtual storage in the web server.
- the related art provides a method for garbage collection of a solid state hard disk, which obtains a blank data block by erasing and moving data of the garbage collection data block in the existing garbage collection strip. The logical subbands of blank data blocks are striped.
- the purpose of the embodiment of the present application is to provide a method, a device, and a system for recovering a network hard disk space, so as to at least solve the problem that the data erasing operation is only provided for the garbage data block in the related art, and the data migration process is not proposed.
- the embodiment of the present application provides a network hard disk space recovery system, including: a data server, a storage server, and a network hard disk, wherein the data server is in communication connection with the storage server, and is used in a pre-stored storage list.
- the group querying the storage attribute of the storage strip group to which the data to be reclaimed belongs in the space reclamation request, and grouping the storage strips in the storage strip group according to the unrecovered data, and generating a space reclamation instruction according to the grouping and storage attributes;
- Server The communication connection with the network hard disk is used to store the unrecovered data in the storage strips in the storage strip group to a new storage strip group in the network hard disk according to the space reclamation instruction, and recycle the storage strip group, and according to the storage
- the embodiment of the present application further provides a method for recovering a network hard disk space, which is applied to a data server in a network hard disk space recovery system, and the method includes: the data server receives a space recovery request, wherein the space recovery request, The data server queries the preset storage list group corresponding to the data to be recycled, and obtains the storage strip group to which the data to be collected belongs; the data server is based on the unrecovered data pair in the storage strip group.
- the storage strips in the storage strip group are grouped to obtain a storage strip to which the unrecovered data belongs; the data server generates a space reclamation instruction according to the storage strip and sends a space reclamation instruction; and the data server receives the space reclamation response information.
- the embodiment of the present application further provides another method for recovering a network hard disk space, which is applied to a storage server in a network hard disk space recovery system, and the method includes: the storage server receives a space recovery instruction; and the storage server recovers according to space.
- the instruction stores the unrecovered data in the storage stripe in the storage strip group to a new storage strip group in the network hard disk; the storage server deletes the data to be recycled, and recovers the storage strip group to which the data to be recovered belongs; the storage server is based on Stores the storage attributes of the new storage strip group to which the unrecycled data belongs, generates space reclamation response information, and sends space reclamation response information.
- the embodiment of the present application further provides a network hard disk space recovery device, which is applied to a data server in a network hard disk space recovery system, and the device includes: a first receiving module, configured to receive a space recovery request, wherein a space reclaiming request, which is used to indicate that the storage space to which the data to be reclaimed belongs is reclaimed; the query module is configured to query a preset storage list group corresponding to the data to be reclaimed, and obtain a storage strip group to which the data to be reclaimed belongs; The storage strips in the storage strip group are grouped according to the unrecovered data in the storage strip group, and the storage strip to which the unrecovered data belongs is obtained; the sending module is configured to generate a space reclamation instruction according to the storage strip, and send the space. a recycling instruction; a second receiving module, configured to receive spatial recovery response information.
- the embodiment of the present application further provides another network hard disk space recovery device, which is applied to a storage server in a network hard disk space recovery system, and the device includes: a receiving module, configured to receive a space recovery instruction; and a storage module For storing the unrecovered data in the storage strips in the storage strip group to a new storage strip group in the network hard disk according to the space reclamation instruction; the recycling module, And the information generating module is configured to generate a space collection response information according to a storage attribute of the new storage strip group to which the unrecycled data belongs; the first sending Module for sending space reclamation response information.
- the embodiment of the present application further provides a data server, including: a processor, a memory, a communication interface, and a bus;
- the processor, the memory, and the communication interface are connected by the bus and complete communication with each other;
- the memory stores executable code
- the processor by reading the executable code stored in the memory, runs a program corresponding to the executable code for executing data in a recycling system applied to a network hard disk space provided by an embodiment of the present application. Any method of recycling the network hard disk space of the server.
- the embodiment of the present application further provides a storage medium for storing executable code, which is used to execute data in a recycling system applied to a network hard disk space provided by an embodiment of the present application. Any method of recycling the network hard disk space of the server.
- the embodiment of the present application further provides an executable code for performing a method for recovering any network hard disk space of a data server in a recovery system applied to a network hard disk space provided by an embodiment of the present application. .
- the embodiment of the present application further provides a storage server, including: a processor, a memory, a communication interface, and a bus;
- the processor, the memory, and the communication interface are connected by the bus and complete communication with each other;
- the memory stores executable code
- the processor by reading the executable code stored in the memory, runs a program corresponding to the executable code, for performing the storage in the recycling system applied to the network hard disk space provided by the embodiment of the present application. Any method of recycling the network hard disk space of the server.
- the embodiment of the present application further provides a storage medium for storing executable code, which is used to perform storage in a recycling system applied to a network hard disk space provided by an embodiment of the present application. Any method of recycling the network hard disk space of the server.
- the embodiment of the present application further provides an executable code for performing a method for recovering any network hard disk space of a storage server in a recycling system applied to a network hard disk space provided by an embodiment of the present application. .
- the data server that is in communication with the storage server is configured to query, in the pre-stored storage list group, the storage attribute of the storage strip group to which the data to be recovered belongs in the space reclamation request, and store the data according to the unrecovered data.
- the storage strips in the stripe group are grouped to manage the storage strips; the storage server connected to the network hard disk stores the unrecovered data in the storage strips in the storage stripe group to the new in the network hard disk
- the new storage strip group realizes the overall recovery of the storage strip group to be recycled, thereby realizing the technical effect of improving the data storage management efficiency in the data moving process, thereby solving the problem that only the garbage data block is provided in the related art. Perform data erasing operations without raising the efficiency of data storage management during data movement technical problem.
- FIG. 1 is a schematic structural diagram of a network hard disk space recovery system according to an embodiment of the present application.
- FIG. 2 is another schematic structural diagram of a network hard disk space recovery system according to an embodiment of the present application.
- FIG. 3 is a schematic flowchart of performing a query by a query device in a network hard disk space recovery system according to an embodiment of the present application
- FIG. 4 is a schematic diagram of data interaction between a data server, a storage server, and a network hard disk in a network hard disk space recovery system according to an embodiment of the present application;
- FIG. 5 is a schematic diagram of a recycling process in a network hard disk space recovery system according to an embodiment of the present application
- FIG. 6 is a schematic flowchart of a method for recovering a network hard disk space of a data server in a recovery system applied to a network hard disk space according to an embodiment of the present application;
- FIG. 7 is a schematic flowchart of a method for recovering a network hard disk space of a storage server in a recovery system applied to a network hard disk space according to an embodiment of the present application;
- FIG. 8 is a schematic structural diagram of a network hard disk space reclamation apparatus of a data server applied to a network hard disk space recovery system according to an embodiment of the present application;
- FIG. 9 is a schematic structural diagram of a network hard disk space recovery device of a storage server applied to a network hard disk space recovery system according to an embodiment of the present application.
- the embodiment of the present application provides a network hard disk space recovery system. It should be noted that the steps shown in the flowchart of the drawing may be executed in a computer system such as a set of computer executable instructions, and although in the process The logical order is shown in the figures, but in some cases the steps shown or described may be performed in a different order than the ones described herein.
- FIG. 1 is a schematic structural diagram of a network hard disk space recovery system according to an embodiment of the present disclosure.
- the system includes: a data server 12 , a storage server 14 , and a network hard disk (Internet Protocol hard disk, referred to as an IP hard disk). 16, among them,
- the data server 12 is connected to the storage server 14 for querying, in the pre-stored storage list group, the storage attribute of the storage strip group to which the data to be recovered belongs in the space reclamation request, and storing the strip group according to the unrecovered data.
- the storage strips in the group are grouped, and a space reclamation instruction is generated according to the grouping and storage attributes;
- the storage server 14 is communicatively coupled to the network hard disk 16 for storing unrecovered data in the storage strips in the storage strip group to a new storage strip group in the network hard disk 16 according to the space reclamation instruction, and recycling the storage strip.
- the sexual generation space reclaims the response information, and returns the space reclamation response information to the data server 12.
- the above strip refers to a set of stored data blocks for storing data.
- the new storage strip group created above may be a blank strip group.
- the recovery system of the network hard disk space is composed of the data server 12, the storage server 14, and the network hard disk 16.
- the user can access the data server 12 through the mobile terminal, and the data server 12 Query the storage location of the data required by the user, the data server 12 accesses the storage server 14 according to the obtained storage location, and the storage server 14 reads the data from the network hard disk 16.
- the data server 12 after receiving the space reclamation request, the data server 12 obtains the storage attribute of the storage strip group of the data to be reclaimed by traversing the query in the pre-stored storage list group, where the storage attribute may include: a storage location And the storage state; after querying the storage strip group to which the data to be recovered belongs, the data server 12 obtains the unrecovered data in the storage strip group, and performs the storage strips in the storage strip group according to the unrecovered data.
- the data server 12 generates a space reclamation instruction according to the grouping and storage attributes; on the storage server 14 side, after receiving the space reclamation instruction, the unrecovered data is stored to the blank storage strip group in the network hard disk, and further, The network hard disk 16 deletes the data to be collected, and recovers the storage strip group to which the data to be recovered belongs. Finally, the space recovery response information is generated according to the storage attribute of the blank storage strip group after the unrecovered data is stored, and the space recovery response information is returned.
- the data server 12 can be a metadata server.
- FIG. 2 is another schematic structural diagram of a network hard disk space recovery system according to an embodiment of the present application.
- the information table group that is, the storage list group in the embodiment of the present application
- the information table group may include: an Object information table, a Strip information table, and a network hard disk information table, wherein the user information can be obtained by querying the Object information table. Whether the required data exists, and if present, the storage location of the data.
- a specific storage location of the data in one or more strips can be obtained, for example, the data is stored in In the two storage data blocks of the two strips, the actual storage location can be obtained by querying the offset of the data in the Object information table and the corresponding position of the data in the Strip information table, for example, the data is in the X strip.
- the first storage data module and the second storage data module of the Y strip; in addition, the query can be obtained by querying the Object information table and the Strip information table. Storing state data storage status of data and the associated strip.
- the data server 12 sends a space reclamation command to the storage server. 14.
- the new storage strip identifier generated by the data server 12, the storage server 14 creates a new storage strip corresponding to the new storage strip identifier in the network hard disk 16, and obtains unrecovered data from the network hard disk 16.
- the unrecovered data is moved to the new storage strip, and the space reclamation response information is generated according to the storage attribute of the new storage strip group after the unrecovered data is stored, and the space reclamation response information is returned to the data server 12.
- the data server 12 can quickly locate the storage location to which the unrecovered data belongs according to the new storage state of the unrecovered data, thereby improving data reading and writing efficiency, and Once the operation is performed, the unrecovered data and the storage strip group in which the unrecovered data is stored are more directly deleted by the positioning, so that the network hard disk space recovery system provided by the embodiment of the present application improves the data storage management efficiency. .
- the data server is connected to the storage server, and is configured to query, in the pre-stored storage list group, the storage strip group to which the data to be recovered belongs in the space collection request.
- the storage server is connected with the network hard disk for storing according to the space reclamation instruction.
- the unrecovered data in the storage strips in the stripe group is stored to a new storage stripe group in the network hard disk, the storage stripe group is reclaimed, and generated based on the storage attributes of the new storage stripe group after storing the unrecovered data
- the space recovers the response information, returns the space recovery response information to the data server, and achieves the space recovery of the storage strips of the data to be recycled, by transferring the unrecovered data to a new storage strip, and storing the unrecovered data.
- the new storage strip is recorded, thereby improving the data
- the storage list group includes: a data storage information table, a storage strip information table, and a network hard disk information table, where the data storage information table includes: data stored in the network hard disk, location of the data in the storage strip, The size of the data and the storage state of the data; the storage stripe information table includes: an identifier of each storage stripe, an attribute of the storage stripe, an Erasure Coding (EC) type, and a disk identifier of a disk to which the storage strip belongs And the storage state of the disk; the network hard disk information table includes: the disk identifier of each disk and the address information of each disk.
- the data storage information table includes: data stored in the network hard disk, location of the data in the storage strip, The size of the data and the storage state of the data
- the storage stripe information table includes: an identifier of each storage stripe, an attribute of the storage stripe, an Erasure Coding (EC) type, and a disk identifier of a disk to which the storage strip belongs
- a data storage Object (object) information table a storage strip strip information table, a network hard disk IP hard disk information table are created in the data server 12, and an Object_key is recorded in the Object information table.
- object key and Strip_id written by Object data and its offset, Strip_id and its corresponding IP hard disk wwn (world wide name) and unit_key are recorded in the Strip information table, and IP hard disk wwn is recorded in the IP hard disk information table.
- IP address; data storage Object (object) information table as shown in Table 1:
- the data server 12 can obtain the Strip to which the Object_key belongs by querying the Object_key corresponding to the data to be recovered, and then obtain the storage stripe identifier Strip_id of the storage stripe to which the Object_key belongs, and the data size Object_size of the Object by the Object_key.
- the data storage state status and the offset of the Object_key in the Object and the Strip, wherein the Strip information table is queried according to the offset of the Object_key in the Object and the Strip, and the actual storage location of the Object_key can be obtained. See Table 2 for details.
- the offset in the Object is used to indicate the sequence number of the Object_key corresponding to the data to be recovered in the Object, that is, the location of the Object_key in the Object can be obtained by querying the offset within the Object;
- the offset within the Strip is used to indicate the location of the Object_key within the Strip.
- the storage strip strip information table is shown in Table 2:
- wwn in Table 2 is used to indicate the identifier of the corresponding disk in the network hard disk 16 to which the storage stripe belongs.
- the network hard disk IP hard disk information table is shown in Table 3:
- the Strip_id corresponding to the Object_key can be obtained by querying Table 1, and the disk wwn corresponding to the Strip_id is obtained by querying Table 2. Finally, the IP address of the disk to which the Object_key belongs can be located through the query table 3.
- the data server 12 includes: a query device, a classification device, and an instruction generation device, where
- the Querying Device is configured to traverse the storage list group, query the storage strip group to which the data to be reclaimed belongs, change the storage state of the storage strip group according to the query result, and check the storage strips in the storage strip group. Storing a storage data block to which the data to be recycled belongs and a storage data block to which the unrecovered data belongs;
- a sorting device configured to be in communication with the querying device, configured to group the storage strips in the storage strip group according to the unrecovered data, to obtain a storage strip group to which the unrecovered data belongs;
- the command generating device is configured to communicate with the sorting device, and is configured to change a storage state of the storage strip group to which the unrecovered data belongs to a storage state to be reclaimed, and generate a storage strip group according to the storage state to be recycled and the unrecovered data. Space recycling instructions.
- the querying device includes: a first query module, a second query module, a status update module, and a determining module, where
- the first query module is configured to query whether the data to be collected exists according to the data storage information table, and if the query result is yes, query the storage strip group corresponding to the data to be recovered in the data storage information table;
- the second query module is in communication with the first query module, and is configured to extract a location of the data to be collected in the data storage information in the storage strip, and query the storage strip information table according to the location to obtain a storage data block that stores the data to be collected. ;
- a status update module configured to communicate with the second query module, to mark the storage status of the storage data block as to be deleted
- the judging module is connected to the status update module, and is configured to determine whether the number of the storage data blocks marked as to be deleted in one of the storage strips is greater than or equal to a preset threshold, and if the judgment result is yes, the label storage is to be recovered.
- the storage state of the data storage strip is to be reclaimed.
- FIG. 3 is a schematic diagram of a process for performing a query by a query device in a network hard disk space recovery system according to an embodiment of the present application.
- the data server 12 Internally traversing the Object information table, the Strip information table, and the IP hard disk information table shown in Tables 1 to 3,
- the Object information table is queried for the Strip_id corresponding to the Object_key;
- the storage data block unit_key corresponding to the Strip_id is queried in the Strip information table;
- the storage state of the strip is set to "to be recycled"; and it is determined whether the strip of the storage strip is the last strip of the traversal query, and if so, the query flow ends, and if not, Then proceed to the second step to the eighth step.
- the "to be deleted" state in the embodiment of the present application is data (information) that has not been actually deleted in the network hard disk or the server for the deletion operation (for example, deleted in the user interface); "not deleted” The status refers to data (information) that has not been deleted.
- the data server 12 further includes: a data receiving device and a status updating device, where
- the data receiving device is in communication with the storage server 14 and configured to receive the space reclamation response information, where the space reclamation response information includes: a data storage state of the unrecovered data and a data storage state of the data to be reclaimed, and a storage strip to which the unrecovered data belongs.
- the status update device is in communication with the data receiving device for updating the data storage information table according to the received data storage status, and updating the storage strip information table according to the stored storage status of the storage strip.
- the storage server 14 includes: a data cache device, an erasure code calculation device, and a hard disk read/write device, where
- the data cache device is respectively connected to the data server 12 and the network hard disk 16 for transmitting a new storage strip creation request to the data server after receiving the space reclamation instruction, after receiving the new storage strip identifier returned by the data server Sending an unrecovered data feedback request to the network hard disk and buffering the unrecovered data returned by the network hard disk;
- the erasure code calculation device is connected to the data cache device for allocating corresponding storage data blocks for the unrecovered data in the new storage strip according to the preset erasure code rule;
- the hard disk read/write device is communicatively coupled to the network hard disk 16 for storing unrecovered data in the storage data block in the new memory strip.
- the storage server 14 further includes: a state feedback device, where
- State feedback means in communication with the data server 12, for storing data status of unrecovered data stored in a new storage strip, storage state of a new storage strip, data storage status of data to be reclaimed, and to be reclaimed
- the storage state of the storage strip group to which the data belongs generates space reclamation response information, and returns the space reclamation response information to the data server 12.
- the new storage strip created above can be a storage strip in the newly created blank storage stripe group.
- FIG. 4 is a schematic diagram of data interaction between a data server, a storage server, and a network hard disk in a network hard disk space recovery system according to an embodiment of the present application, as shown in FIG. 4 .
- the data server 12 (ie, the metadata server shown in FIG. 4) periodically scans the Strip information table, finds the strip to be recycled, and obtains the corresponding Object_key from the unit_key in the Strip information table; queries whether the Object_key is deleted in the Object information table, and records The Object_key that has not been deleted sorts the strips to be recycled according to Object_key, and sends the undeleted Object and the strip to be recycled and the unit_key in the Strip to the storage server 14. After receiving the ungelecasted object and the to-be-recovered strip and the unit_key in the strip, the storage server 14 feeds back to the data server 12 the information of the successful reception.
- the recycling system can start from the first strip to be recycled, and sequentially read the unremoved Object data therein, and execute the steps in the loop loop box on the unremoved Object data, as shown in the following:
- the storage server 14 sends a new storage strip creation request to the data server 12 (ie, requests a new Strip_id from the metadata server as shown in FIG. 4) and receives a new storage strip identifier returned by the data server 12 (ie, a map)
- the metadata server shown in FIG. 4 returns a new "Strip_id", and the storage server 14 reads the data of the unremoved Object in the strip to be recycled from the network hard disk 16 (ie, the IP hard disk shown in FIG. 4), and receives the network hard disk. After the unremoved Object data returned by 16 is cached, the unremoved Object data is cached.
- the storage server 14 generates unit_key of the undelete Object data according to the preset erasure code rule (EC calculation) by the erasure code calculation means (the EC calculation also generates check data).
- the storage server 14 allocates the corresponding storage data block for the new object in the new storage strip, and the storage server 14 writes the undelete Object data and the verification data to the new network hard disk 16 through the hard disk read/write device. Store strips in Strip_id.
- the storage server 14 After receiving the response information of the successful write returned by the network hard disk 16, the storage server 14 transmits to the data server 12 the data storage state including the unrecovered data stored in the new storage strip, and the storage of the new storage strip.
- the storage status, the data storage status of the data to be recovered, and the space recovery response information of the storage status of the storage strip group to which the data to be recovered belongs ie, the undeleted Object information and the new information sent by the storage server to the metadata server shown in FIG. 4) Strip information).
- the storage server 14 sends the data of the Object to be recovered to the network hard disk 16 and receives the information of the successful deletion of the feedback from the network hard disk 16 , and then sends the data to be deleted in the Object information table to the data server 12 and the object to be recovered is to be recovered.
- the command of the Strip information After deleting the corresponding information in the Object information table, the data server 12 feeds back to the storage server the information that the strip to be recovered is successfully deleted.
- the unremoved Object data of the next to-be-recovered strip is sequentially read, and the steps in the loop loop box are executed for the undelete Object data. This will complete the deletion of the next strip to be recycled.
- the above embodiment is a process of sequentially deleting each strip by a loop loop.
- the movement of the unremoved data and the deletion of the data to be deleted may be performed on all the strips in one storage strip group, and then the loop is ended, and then the entire storage strip group is to be recovered. Strip is deleted at the same time.
- the management of each strip of the storage strip group may be performed in sequence, or may be managed in parallel for multiple strips, and is not limited herein.
- FIG. 5 is a schematic diagram of a recovery process in a network hard disk space recovery system according to an embodiment of the present disclosure; as shown in FIG. 5, the data recovery process is as follows:
- object1 ⁇ object7 are deleted, and strip1 ⁇ strip4 are all strips to be recycled.
- strip1, strip2, and strip3 are assigned to object_123
- strip4 is assigned to object8.
- the object_123 scattered data is aggregated into strip5. That is, the objects 1 to 7 are the objects to be collected in the data server, and the storage locations of the object 1 to the object 7 are obtained by querying the strip information table, that is, the strips 1 to the strip 4 are the strips to be recovered (ie, in the embodiment of the present application).
- Storage strip group since object_123 and object8 are unrecycled data, before recycling strip1 ⁇ strip4, According to FIG.
- strip1, strip2, and strip3 are assigned to the object_123 group, and strip4 is assigned to the object8 group.
- the object_123 distributed data is aggregated into the strip5, that is, the storage server 14 is directed to the data server 12.
- Sending a new storage strip creation request the data server 12 generates a new storage strip identifier according to the new storage strip creation request, and the storage server 14 generates a new storage strip in the network hard disk 16 according to the new storage strip identifier.
- strip5 With strip5, and store unrecovered data object_123 and object8 to strip5.
- the collection system of the network hard disk space provided by the embodiment of the present application groups the strips to be recycled according to the undelete Object.
- the unremoved Object data dispersed in the plurality of strips to be recycled is read out, and the storage is performed one or more.
- Strip you can speed up the reading of Object data and reduce the storage space of Object metadata.
- Object metadata can include the storage location of Object data.
- the new storage strip group created above may be a blank strip group.
- the new storage strip created above can be a storage strip in the newly created blank storage stripe group.
- FIG. 6 is a schematic flowchart of a method for recovering a network hard disk space of a data server in a recovery system applied to a network hard disk space according to an embodiment of the present disclosure. As shown in FIG. 6 , the method includes the following steps:
- Step S602 the data server receives a space reclamation request, where the space reclamation request is used to indicate that the storage space to which the data to be reclaimed belongs is reclaimed;
- the data server receives the space recovery request sent by the user by the mobile terminal, where the space recovery request may include data to be collected.
- Step S604 the data server queries the preset storage list group corresponding to the data to be collected, and obtains a storage strip group to which the data to be recovered belongs.
- the data server locates the location of the data storage to be reclaimed by querying the storage list group, that is, the storage strip of the data to be collected in the embodiment of the present application.
- the storage list group in the embodiment of the present application may include: a data storage information table, a storage strip information table, and a network hard disk information table.
- Step S606 the data server groups the storage strips in the storage strip group according to the unrecovered data in the storage strip group, and obtains a storage strip to which the unrecovered data belongs;
- the data server collects the unrecovered data in the storage strip group, and groups the storage strips in the storage strip group according to the unrecovered data to obtain a storage strip corresponding to the unrecovered data, so that the data is moved during the data transfer process.
- Improve data storage management efficiency by identifying the location of unrecovered data.
- Step S608 the data server generates a space reclamation instruction according to the storage strip, and sends a space reclamation instruction;
- the data server due to the space reclamation request received by the data server, the Object data is not actually deleted, but the data server is required to instruct the storage server to delete the data on the network hard disk storing the Object data, and the Object data is actually deleted. Therefore, the data server generates a space reclamation instruction based on the storage stripe and sends the space reclamation instruction to the storage server.
- Step S610 the data server receives the space reclamation response information.
- the storage server deletes the data to be recycled with the network hard disk, and collects the storage strip group to which the data to be recovered belongs, and after the recovery is completed, the data server will accept the storage.
- the space recovery response information returned by the server wherein the space recovery response information may include: a data storage state of the unrecovered data and a data storage state of the data to be recycled, and a storage state of the storage strip to which the unrecovered data belongs and a data to be recycled Stores the storage state of the stripe. So that the data server updates the data storage information table and the storage strip information table according to the space recovery response information.
- the method for recovering the network hard disk space receives the space reclamation request by the data server, where the space reclamation request is used to indicate that the storage space to which the data to be reclaimed belongs is reclaimed; and the data server queries the preset storage corresponding to the data to be reclaimed.
- the data server groups the storage strips in the storage strip group according to the unrecovered data in the storage strip group, and obtains a storage strip to which the unrecovered data belongs;
- the server generates a space reclamation instruction according to the storage stripe, and sends a space reclamation instruction;
- the data server receives the returned space reclamation response information, and achieves the purpose of spatially reclaiming the storage strips of the data to be reclaimed, thereby improving the data moving process.
- the technical effect of the data storage management efficiency further solves the technical problem that how to improve the data storage management efficiency in the data moving process is only provided because the related art only provides the data erasing operation for the garbage data block.
- the data server queries the preset storage list group corresponding to the data to be collected in step S604, and obtains data to be collected.
- the associated storage strip group includes:
- Step1 The data server queries, in the data storage information table, whether the data to be recovered exists;
- Step 2 in the case that the query result is yes, the data server queries the data storage information table for the storage strip group to which the data to be recycled belongs;
- Step 3 extracting an offset of the data to be recovered in the data storage information table
- Step 4 querying the storage strip information table according to the offset, and obtaining a storage data block corresponding to the data to be recovered.
- the steps Step1 to Step4 correspond to the process of querying the storage list group by the querying device in the data server in FIG. 3 in the first embodiment, wherein the data server determines whether the Object corresponding to the data to be recovered exists by querying the Object information table; In the case of the Object information table, the Strip_id corresponding to the Object_key is searched; after the Strip_id is obtained, the stored data block unit_key corresponding to the Strip_id is queried in the Strip information table; the offset of the Object_key and the Strip length in the Object information table, and the Strip information are combined.
- the stored data block unit_key corresponding to the Strip_id obtained in the table locates the unit_key corresponding to the Object.
- step S606 the data server groups the storage strips in the storage strip group according to the unrecovered data in the storage strip group, and the storage strip to which the unrecovered data belongs is obtained:
- Step 1 changing the storage state of the storage data block to which the data to be recovered belongs to the deletion in the storage strip information table
- Step 2 the number of stored data blocks in which the storage state is deleted is obtained, and the number of stored data blocks corresponding to the unrecovered data is obtained;
- Step 3 According to the number of stored data blocks corresponding to the unrecovered data, a storage strip corresponding to the unrecovered data is obtained.
- the data server changes the storage state of the storage data block corresponding to the data to be collected in the storage strip information table to delete, and counts the number of storage data blocks whose storage status is deleted, and obtains the unrecovered data corresponding to the data storage block.
- the number of stored data blocks that is, the data server changes the storage state of the unit_key in the strip information table, and changes the storage state to "delete”; checks the number of unit_keys whose status is "delete” in the strip_id; the storage that is deleted by statistics
- the number of data blocks is used to obtain the number of stored data blocks corresponding to the unrecovered data, thereby obtaining a storage strip corresponding to the unrecovered data.
- the method for recovering the network hard disk space provided by the embodiment of the present application further includes: before the step 2 in step S606, the number of the storage data blocks corresponding to the unrecovered data is obtained.
- Step A determining whether the number is greater than or equal to a preset threshold
- Step B in the case that the determination result is yes, marking the storage data block corresponding to the data to be recovered The storage state of the storage strip of the genus is to be recovered.
- the N in the preset threshold may be N in the erasure code mode N+M, that is, N is the number of stored data blocks actually storing data.
- the method for recovering the network hard disk space provided by the embodiment of the present application further includes: receiving a new storage stripe creation request; creating according to the new storage stripe Request to create a new storage strip identifier;
- the data server receives a new storage stripe creation request sent by the storage server; the data server is based on the new storage strip.
- the creation request creates a new storage stripe identifier and returns the new storage stripe identifier to the storage server, so that the storage server creates a new storage stripe in the network hard disk according to the new storage stripe identifier.
- the space recovery response information includes: a data storage state of the unrecovered data and a data storage state of the data to be recycled, and a storage state of the storage strip to which the unrecovered data belongs and a storage state of the storage strip to which the data to be recycled belongs.
- the method for recovering the network hard disk space provided by the embodiment of the present application further includes: updating the data storage information table according to the received data storage state; The storage state of the storage stripe is updated by the storage strip information table.
- the data server when the storage server and the network hard disk end the recycling process, the data server receives the space recovery response information returned by the storage server, where the recovery response information includes the data storage status of the unrecovered data and In the case of the data storage state of the data to be recovered, and the storage state of the storage strip to which the unrecovered data belongs and the storage state of the storage strip to which the data to be recycled belongs, the data server updates the data storage information table according to the received data storage state, and The storage stripe information table is updated according to the storage state of the received storage strip.
- the strips to be recycled are grouped according to the undelete Objects on the data server side, and the undelete Object data dispersed in the plurality of strips to be recycled is read out and stored centrally.
- the undelete Object data dispersed in the plurality of strips to be recycled is read out and stored centrally.
- the new storage strip group created above may be a blank strip group.
- the new storage strip created above can be a storage strip in the newly created blank storage stripe group.
- FIG. 7 is a schematic flowchart of a method for recovering a network hard disk space of a storage server applied to a network hard disk space recovery system according to an embodiment of the present disclosure. As shown in FIG. 7 , the method includes the following steps:
- Step S702 the storage server receives a space reclamation instruction
- the storage server of the embodiment 1 in the method for recovering the network hard disk space provided by the embodiment of the present application, on the storage server side, corresponding to the data server side provided by the embodiment 2, the storage server receives the space sent by the data server. Recycling instructions.
- Step S704 the storage server stores the unrecovered data in the storage strips in the storage strip group to a new storage strip group in the network hard disk according to the space reclamation instruction;
- the storage server dumps the unrecovered data in the storage strips in the storage strip group to a new storage strip group in the network hard disk according to the space reclamation instruction.
- Step S706 the storage server deletes the data to be recovered, and recovers the storage strip group to which the data to be recovered belongs;
- the storage server deletes the data to be recovered, and recovers the data to be recycled.
- Store strip groups after the unrecovered data is transferred to a new storage strip group in the network hard disk based on the moving of the unrecovered data in step S704, the storage server deletes the data to be recovered, and recovers the data to be recycled.
- Step S708 The storage server generates space collection response information according to the storage attribute of the new storage strip group to which the unrecovered data belongs, and sends the space collection response information.
- the new storage strip group to which the unrecovered data belongs according to the data storage status of the unrecovered data (representing the new storage strip determined in S704)
- the storage state of the group, the new storage strip group stores the unrecovered data, the data storage state of the data to be reclaimed, and the storage state of the storage strip group to which the data to be reclaimed belongs to generate space reclamation response information.
- the storage server sends space reclamation response information to the data server.
- the storage server may also send a space reclamation response to other nodes, which forwards the space reclamation response to the data server.
- the method for recovering the network hard disk space receives the space through the storage server
- the storage server stores the unrecovered data in the storage strips in the storage strip group to a new storage strip group in the network hard disk according to the space reclamation instruction; the storage server deletes the data to be recycled, and recovers the data to be recycled.
- the storage strip group; the storage server generates the space reclamation response information according to the storage attribute of the new storage strip group to which the unrecycled data belongs; the storage server sends the space reclamation response information, and the storage strip for the group to be recycled data is spaced.
- the network hard disk space provided by the embodiment of the present application
- the recycling method further includes: sending a new storage strip creation request; receiving the returned new storage strip identifier; sending an unrecovered data feedback request to the network hard disk; and buffering the unrecovered data returned by the network hard disk.
- the steps of creating a new storage strip identifier in the data server in Embodiment 1 corresponding to FIG. 4 in Embodiment 1, and on the storage server side, because the unrecovered data needs to be placed the storage server Sending a new storage stripe creation request to the data server; after the data server creates a new storage stripe identifier, receiving the new storage stripe identifier, and sending an unrecovered data feedback request to the network hard disk to cache the unrecovered data.
- step S704 the storage server stores the unrecovered data in the storage stripe in the storage strip group to the new storage strip group in the network hard disk according to the space reclamation instruction, including:
- Step 1 according to the preset erasure code rule, in the new storage stripe, the corresponding storage data block is newly allocated for the unrecovered data;
- Step 2 the unrecovered data is stored in the storage data block in the new storage strip.
- the storage server allocates a corresponding storage data block for the unrecovered data in the new storage strip according to the preset erasure code rule, and Unrecovered data is stored in a block of stored data in a new stripe.
- the storage server generates the space reclamation response information according to the storage attribute of the new storage strip group to which the unrecycled data belongs, and sends the space reclamation response information, including:
- Step 1 generating, according to the data storage state of the unrecovered data stored in the new storage strip, the storage state of the new storage stripe, the data storage state of the data to be reclaimed, and the storage state of the storage strip group to which the data to be reclaimed belongs Space recovery response information.
- the new storage strip group created above may be a blank strip group.
- the new storage strip created above can be a storage strip in the newly created blank storage stripe group.
- FIG. 8 is a schematic structural diagram of a device for recovering a network hard disk space of a data server in a recovery system for a network hard disk space according to an embodiment of the present invention. As shown in FIG. 8 , the method includes: a first receiving module 81 , a query module 82 , a grouping module 83, a sending module 84, and a second receiving module 85, wherein
- the first receiving module 81 is configured to receive a space reclamation request, where the space reclamation request is used to indicate that the storage space to which the data to be reclaimed belongs is reclaimed;
- the querying module 82 is configured to query a preset storage list group corresponding to the data to be collected, and obtain a storage strip group to which the data to be recovered belongs;
- a grouping module 83 configured to group the storage strips in the storage strip group according to the unrecovered data in the storage strip group, to obtain a storage strip to which the unrecovered data belongs;
- the sending module 84 is configured to generate a space reclamation instruction according to the storage strip, and send a space reclamation instruction;
- the second receiving module 85 is configured to receive space recovery response information.
- the data server may determine that the unrecovered data is transferred according to the storage state of the storage strip group to which the unrecovered data (for example, the video data that has not been instructed to be deleted) included in the space recovery response information belongs. After the storage location.
- the apparatus for recovering the network hard disk space receives the space reclamation request by the data server, wherein the space reclamation request is used to indicate that the storage space to which the data to be reclaimed belongs is reclaimed; and the data server queries the preset storage corresponding to the data to be reclaimed.
- the data server groups the storage strips in the storage strip group according to the unrecovered data in the storage strip group, and obtains a storage strip to which the unrecovered data belongs;
- the server generates a space reclamation instruction according to the storage strip and sends a space reclamation instruction;
- the data server receives the returned space reclamation response information, and achieves the purpose of spatially reclaiming the storage strips of the data to be reclaimed, thereby improving the data moving process.
- the technical effect of the data storage management efficiency further solves the problem that the related art only provides the data erasing operation for the garbage data block, and does not propose the technical problem of how to improve the data storage management efficiency during the data moving process.
- the query module 82 includes:
- a first query unit configured to query, in the data storage information table, whether the data to be recovered exists in the case that the storage list group includes the data storage information table, the storage strip information table, and the network hard disk information table;
- a second query unit configured to query, in the data storage information table, a storage strip group to which the data to be recovered belongs in the case that the query result is YES;
- An extracting unit configured to extract an offset of the data to be recovered in the data storage information table
- the third query unit is configured to perform a query on the storage strip information table according to the offset, and obtain a storage data block corresponding to the data to be recovered.
- the grouping module 83 includes:
- a state change unit configured to change the storage state of the storage data block to which the data to be recovered belongs to be deleted in the storage strip information table
- a first statistic unit configured to count the number of the stored data blocks whose storage state is deleted, and obtain the number of the stored data blocks corresponding to the unrecovered data
- the second statistic unit is configured to obtain a storage strip corresponding to the unrecovered data according to the number of the stored data blocks corresponding to the unrecovered data.
- the apparatus for recovering the network hard disk space further includes:
- a determining unit configured to determine whether the number is greater than or equal to a preset threshold before obtaining the number of stored data blocks corresponding to the unrecovered data
- the marking unit is configured to mark, when the determination result is yes, the storage state of the storage strip to which the storage data block corresponding to the data to be recovered belongs to be to be recovered.
- the device for recovering the network hard disk space further includes:
- a third receiving module configured to receive a new storage strip creation request before the data server receives the returned space reclamation response information
- An identifier creation module configured to create a new storage strip identifier according to a new storage strip creation request
- the identity sending module is configured to send a new storage strip identifier.
- the device for recovering the network hard disk space further includes:
- the first state update module configured to recover the response information in the space includes: a data storage state of the unrecovered data and a data storage state of the data to be recycled, and a storage state of the storage strip to which the unrecovered data belongs and a storage strip to which the data to be recycled belongs Case of storage status, received at the data server After the returned space recovers the response information, the data storage information table is updated according to the received data storage state;
- the second state update module is configured to update the storage stripe information table according to the storage state of the received storage strip.
- the new storage strip group created above may be a blank strip group.
- the new storage strip created above can be a storage strip in the newly created blank storage stripe group.
- FIG. 9 is a schematic structural diagram of a device for recovering a network hard disk space of a storage server in a recovery system for a network hard disk space according to an embodiment of the present invention. As shown in FIG. 9, the method includes: a receiving module 91, a storage module 92, and a recycling module. 93. The information generating module 94 and the first sending module 95, wherein
- the receiving module 91 is configured to receive a space reclamation instruction
- the storage module 92 is configured to store the unrecovered data in the storage strips in the storage strip group to a new storage strip group in the network hard disk according to the space reclamation instruction;
- a recycling module 93 configured to delete data to be recovered, and recover a storage strip group to which the data to be recovered belongs;
- the information generating module 94 is configured to generate space reclamation response information according to the storage attribute of the new storage strip group to which the unrecovered data belongs;
- the first sending module 95 is configured to send space recovery response information.
- the device for recovering the network hard disk space receives the space reclamation instruction through the storage server; the storage server stores the unrecovered data in the storage strip in the storage strip group to the new storage in the network hard disk according to the space reclamation instruction.
- the storage server deletes the data to be reclaimed, and reclaims the storage strip group to which the data to be reclaimed belongs; the storage server generates the space reclamation response information according to the storage attribute of the new storage strip group to which the unrecovered data belongs; the storage server sends
- the spatial recovery response information achieves the purpose of spatially recycling the storage strips of the data to be recycled, thereby realizing the technical effect of improving the data storage management efficiency during the data moving process, thereby solving the problem that only the garbage is provided in the related art.
- the data block performs the data erasing operation, and does not propose a technical problem of how to improve the efficiency of data storage management during the data moving process.
- the device for recovering the network hard disk space further includes:
- a second sending module configured to store the storage strip group in the storage server according to the space reclamation instruction Sending a new storage strip creation request before the unrecovered data in the strip is stored to the new storage strip group in the network hard disk;
- An identifier receiving module configured to receive a new storage strip identifier
- a request sending module configured to send an unrecovered data feedback request to the network hard disk
- a cache module that caches unrecycled data returned by the network's hard disk.
- the storage module 92 includes:
- An allocating unit configured to allocate a corresponding storage data block for the unrecovered data in the new storage strip according to the preset erasure code rule
- a storage unit for storing unrecovered data into a storage data block in a new storage stripe.
- the information generating module 94 includes:
- the information generating unit is configured to: according to the data storage state of the unrecovered data stored in the new storage strip, the storage state of the new storage stripe, the data storage state of the data to be reclaimed, and the storage strip group to which the data to be reclaimed belongs The storage state generates space reclamation response information.
- the new storage strip group created above may be a blank strip group.
- the new storage strip created above can be a storage strip in the newly created blank storage stripe group.
- the embodiment of the present application further provides a data server, which may include: a processor, a memory, a communication interface, and a bus;
- the processor, the memory, and the communication interface are connected by the bus and complete communication with each other;
- the memory stores executable code
- the processor by reading the executable code stored in the memory, runs a program corresponding to the executable code for executing data in a recycling system applied to a network hard disk space provided by an embodiment of the present application.
- the method for recovering the network hard disk space of the server in the recovery system of the network hard disk space provided by the embodiment of the present application may include:
- the storage list group includes a data storage information table, a storage strip information table, and a network hard disk information table
- the storage strip group to which the data belongs including:
- the data server queries the data storage information table for the storage strip group to which the data to be recovered belongs;
- the storage strip corresponding to the unrecovered data is obtained according to the number of the stored data blocks corresponding to the unrecovered data.
- the method further includes: before the obtaining the number of the storage data blocks corresponding to the unrecovered data, the method further includes:
- the storage state of the storage strip to which the storage data block corresponding to the data to be recovered belongs is marked as to be recovered.
- the method further includes: before the data server receives the returned space recovery response information, the method further includes:
- a new storage stripe identifier is created according to the new storage stripe creation request, and the new storage stripe identifier is sent.
- the collecting the response information in the space includes: a data storage state of the unrecovered data and a data storage state of the to-be-recovered data, and a storage strip of the unrecovered data
- the method further includes:
- the storage strip information table is updated according to the storage state of the received storage strip.
- the embodiment of the present application further provides a storage medium for storing executable code, which is used to execute a network of a data server in a recycling system applied to a network hard disk space provided by an embodiment of the present application.
- the method for recovering the hard disk space of the data server in the recycling system of the network hard disk space provided by the embodiment of the present application may include:
- the storage list group includes a data storage information table, a storage strip information table, and a network hard disk information table
- the storage strip group to which the data belongs including:
- the data server queries the data storage information table for the storage strip group to which the data to be recovered belongs;
- the storage strip corresponding to the unrecovered data is obtained according to the number of the stored data blocks corresponding to the unrecovered data.
- the method further includes: before the obtaining the number of the storage data blocks corresponding to the unrecovered data, the method further includes:
- the storage state of the storage strip to which the storage data block corresponding to the data to be recovered belongs is marked as to be recovered.
- the method further includes: before the data server receives the returned space recovery response information, the method further includes:
- a new storage stripe identifier is created according to the new storage stripe creation request, and the new storage stripe identifier is sent.
- the method further includes:
- the storage strip information table is updated according to the storage state of the received storage strip.
- the embodiment of the present application further provides an executable code for performing a method for recovering a network hard disk space of a data server in a recovery system applied to a network hard disk space provided by the embodiment of the present application;
- the method for recovering the network hard disk space of the data server in the recycling system applied to the network hard disk space provided by the example may include:
- the storage list group includes a data storage information table, a storage strip information table, and a network hard disk information table
- the storage strip group to which the data belongs including:
- the data server queries the data storage information table for the storage strip group to which the data to be recovered belongs;
- the storage strip corresponding to the unrecovered data is obtained according to the number of the stored data blocks corresponding to the unrecovered data.
- the method further includes: before the obtaining the number of the storage data blocks corresponding to the unrecovered data, the method further includes:
- the storage state of the storage strip to which the storage data block corresponding to the data to be recovered belongs is marked as to be recovered.
- the method further includes: before the data server receives the returned space recovery response information, the method further includes:
- a new storage stripe identifier is created according to the new storage stripe creation request, and the new storage stripe identifier is sent.
- the method further includes:
- the storage strip information table is updated according to the storage state of the received storage strip.
- the embodiment of the present application further provides a storage server, which may include: a processor, a memory, a communication interface, and a bus;
- the processor, the memory, and the communication interface are connected by the bus and complete communication with each other;
- the memory stores executable code
- the processor by reading the executable code stored in the memory, runs a program corresponding to the executable code, for performing the storage in the recycling system applied to the network hard disk space provided by the embodiment of the present application.
- the method for recovering the network hard disk space of the server in the recovery system of the network hard disk space provided by the embodiment of the present application may include:
- the method further includes: before the storing the unrecovered data in the storage strip in the storage strip group to the new storage strip group in the network hard disk according to the space reclamation instruction, the method further includes:
- the space recovery instruction stores the unrecovered data in the storage strips in the storage strip group to a new storage strip group in the network hard disk, including:
- the unrecovered data is stored into a block of stored data in the new memory strip.
- the storage state of the band group generates the space reclamation response information.
- the embodiment of the present application further provides a storage medium for storing executable code, which is used to execute a network of a storage server in a recycling system applied to a network hard disk space provided by an embodiment of the present application.
- the method for recovering the hard disk space of the storage server in the recycling system of the network hard disk space provided by the embodiment of the present application may include:
- the method further includes: before the storing the unrecovered data in the storage strip in the storage strip group to the new storage strip group in the network hard disk according to the space reclamation instruction, the method further includes:
- the space recovery instruction stores the unrecovered data in the storage strips in the storage strip group to a new storage strip group in the network hard disk, including:
- the unrecovered data is stored into a block of stored data in the new memory strip.
- the storage state of the band group generates the space reclamation response information.
- the embodiment of the present application further provides an executable code for executing the embodiment of the present application.
- the method for recovering the network hard disk space of the storage server in the recycling system of the network hard disk space provided by the embodiment of the present invention; wherein the network hard disk space of the storage server in the recycling system applied to the network hard disk space provided by the embodiment of the present application is recovered Methods can include:
- the method further includes: before the storing the unrecovered data in the storage strip in the storage strip group to the new storage strip group in the network hard disk according to the space reclamation instruction, the method further includes:
- the space recovery instruction stores the unrecovered data in the storage strips in the storage strip group to a new storage strip group in the network hard disk, including:
- the unrecovered data is stored into a block of stored data in the new memory strip.
- the storage state of the band group generates the space reclamation response information.
- the disclosed technical contents may be implemented in other manners.
- the device embodiments described above are only schematic, for example, the division of the unit may be a logical function division, and may be additionally implemented in actual implementation. Sub-systems, such as multiple units or components, may be combined or integrated into another system, or some features may be omitted or not implemented.
- the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, unit or module, and may be electrical or otherwise.
- the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
- each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
- the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
- the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
- a computer readable storage medium A number of instructions are included to cause a computer device (which may be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present application.
- the foregoing storage medium includes: a U disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and the like. .
Abstract
Description
字段 | Key | 备注 |
wwn | Primary | 主键,表示每个磁盘 |
IP | 磁盘IP地址 |
Claims (33)
- 一种网络硬盘空间的回收系统,其特征在于,包括:数据服务器、存储服务器和网络硬盘,其中,所述数据服务器,与所述存储服务器通信连接,用于在预先存储的存储列表组中,查询空间回收请求中待回收数据所属的存储条带组的存储属性,并依据未回收数据对所述存储条带组中的存储条带进行分组,根据所述分组和所述存储属性生成空间回收指令;所述存储服务器,与所述网络硬盘通信连接,用于依据所述空间回收指令,将所述存储条带组中的存储条带中的未回收数据存储至所述网络硬盘中的新的存储条带组,回收所述存储条带组,并依据存储所述未回收数据后的所述新的存储条带组的存储属性,生成空间回收响应信息,将所述空间回收响应信息返回所述数据服务器。
- 根据权利要求1所述的系统,其特征在于,在所述存储属性包括存储地址和存储状态的情况下,所述数据服务器包括:查询装置、分类装置和指令生成装置,其中,所述查询装置,用于遍历所述存储列表组,查询所述待回收数据所属的存储条带组,依据查询结果更改所述存储条带组的存储状态,并在所述存储条带组中的存储条带查询存储所述待回收数据所属的存储数据块以及未回收数据所属的存储数据块;所述分类装置,与所述查询装置通信连接,用于依据所述未回收数据对所述存储条带组中的存储条带进行分组,得到所述未回收数据所属的存储条带组;所述指令生成装置,与所述分类装置通信连接,用于将所述未回收数据所属的存储条带组的存储状态更改为待回收存储状态,并依据所述待回收存储状态和所述未回收数据所属的存储条带组,生成所述空间回收指令。
- 根据权利要求2所述的系统,其特征在于,所述存储列表组包括:数据存储信息表、存储条带信息表和网络硬盘信息表,其中,所述数据存储信息表包括:存储于所述网络硬盘中的数据、所述数据在所述存储条带中的位置、所述数据的大小和所述数据的存储状态;所述存储条带信息表包括:每个存储条带的标识、所述存储条带的属性、所述存储条带所属磁盘的磁盘标识和所述磁盘的存储状态;所述网络硬盘信息表包括:每个磁盘的磁盘标识 和每个磁盘的地址信息。
- 根据权利要求3所述的系统,其特征在于,所述查询装置包括:第一查询模块、第二查询模块、状态更新模块和判断模块,其中,所述第一查询模块,用于依据所述数据存储信息表查询所述待回收数据是否存在,在查询结果为是的情况下,查询所述数据存储信息表中所述待回收数据对应的存储条带组;所述第二查询模块,与所述第一查询模块通信连接,用于提取所述数据存储信息中所述待回收数据在所述存储条带中的位置,并依据所述位置查询所述存储条带信息表,得到存储所述待回收数据的存储数据块;所述状态更新模块,与所述第二查询模块通信连接,用于将所述存储数据块的存储状态标记为待删除;所述判断模块,与所述状态更新模块通信连接,用于判断在一个存储条带中标记为待删除的存储数据块的数量是否大于或等于预设阈值,在判断结果为是的情况下,标记存储所述待回收数据的存储条带的存储状态为待回收。
- 根据权利要求3所述的系统,其特征在于,所述数据服务器还包括:数据接收装置和状态更新装置,其中,所述数据接收装置,与所述存储服务器通信连接,用于接收所述空间回收响应信息,其中,所述空间回收响应信息包括:所述未回收数据的数据存储状态和所述待回收数据的数据存储状态,以及所述未回收数据所属存储条带的存储状态和所述待回收数据所属存储条带的存储状态;所述状态更新装置,与所述数据接收装置通信连接,用于依据接收到的数据存储状态更新所述数据存储信息表,以及依据接收到的存储条带的存储状态更新所述存储条带信息表。
- 根据权利要求3所述的系统,其特征在于,所述存储服务器包括:数据缓存装置、纠删码计算装置和硬盘读写装置,其中,所述数据缓存装置,分别与所述数据服务器和所述网络硬盘通信连接,用于在接收所述空间回收指令之后,向所述数据服务器发送新的存储条带创建请求,在接收所述数据服务器返回的所述新的存储条带标识后,向所述网络硬盘发送未回收数据反馈请求,并缓存所述网络硬盘返回的未回收数据;所述纠删码计算装置,与所述数据缓存装置通信连接,用于依据预设纠删码规则,在所述新的存储条带中,为所述未回收数据分配对应的存储数据 块;所述硬盘读写装置,与所述网络硬盘通信连接,用于将所述未回收数据存储至所述新的存储条带中的存储数据块中。
- 根据权利要求6所述的系统,其特征在于,所述存储服务器还包括:状态反馈装置,其中,所述状态反馈装置,与所述数据服务器通信连接,用于依据存储于所述新的存储条带中的未回收数据的数据存储状态、所述新的存储条带的存储状态、所述待回收数据的数据存储状态和所述待回收数据所属的存储条带组的存储状态生成所述空间回收响应信息,并将所述空间回收响应信息返回所述数据服务器。
- 一种网络硬盘空间的回收方法,其特征在于,应用于网络硬盘空间的回收系统中的数据服务器,所述方法包括:接收空间回收请求,其中,所述空间回收请求,用于指示回收待回收数据所属的存储空间;查询所述待回收数据对应的预设存储列表组,得到所述待回收数据所属的存储条带组;依据所述存储条带组中的未回收数据对所述存储条带组中的存储条带进行分组,得到所述未回收数据所属的存储条带;依据所述存储条带生成空间回收指令,并发送所述空间回收指令;接收空间回收响应信息。
- 根据权利要求8所述的方法,其特征在于,在所述存储列表组包括数据存储信息表、存储条带信息表和网络硬盘信息表的情况下,所述查询所述待回收数据对应的预设的存储列表组,得到所述待回收数据所属的存储条带组,包括:在所述数据存储信息表中查询所述待回收数据是否存在;在查询结果为是的情况下,所述数据服务器在所述数据存储信息表中查询所述待回收数据所属的存储条带组;提取所述待回收数据在所述数据存储信息表中的偏移量;依据所述偏移量在所述存储条带信息表进行查询,得到所述待回收数据对应的存储数据块。
- 根据权利要求9所述的方法,其特征在于,所述依据所述存储条带 组中的未回收数据对所述存储条带组中的存储条带进行分组,得到所述未回收数据所属的存储条带,包括:将所述待回收数据所属的存储数据块的存储状态,在所述存储条带信息表中更改为删除;统计所述存储状态为删除的存储数据块的个数,得到所述未回收数据对应的存储数据块的个数;依据所述未回收数据对应的存储数据块的个数,得到所述未回收数据对应的存储条带。
- 根据权利要求10所述的方法,其特征在于,在所述得到所述未回收数据对应的存储数据块个数之前,所述方法还包括:判断所述个数是否大于或等于预设阈值;在判断结果为是的情况下,标记所述待回收数据对应的存储数据块所属的存储条带的存储状态为待回收。
- 根据权利要求9所述的方法,其特征在于,在所述数据服务器接收返回的空间回收响应信息之前,所述方法还包括:接收新的存储条带创建请求;依据所述新的存储条带创建请求,创建新的存储条带标识,并发送所述新的存储条带标识。
- 根据权利要求9所述的方法,其特征在于,在所述空间回收响应信息包括:所述未回收数据的数据存储状态和所述待回收数据的数据存储状态,以及所述未回收数据所属存储条带的存储状态和所述待回收数据所属存储条带的存储状态的情况下,在所述接收返回的空间回收响应信息之后,所述方法还包括:依据接收到的数据存储状态,更新所述数据存储信息表;依据接收到的存储条带的存储状态,更新所述存储条带信息表。
- 一种网络硬盘空间的回收方法,其特征在于,应用于网络硬盘空间的回收系统中的存储服务器,所述方法包括:接收空间回收指令;依据空间回收指令将存储条带组中存储条带中的未回收数据存储至网络硬盘中的新的存储条带组;删除待回收数据,并回收所述待回收数据所属的存储条带组;依据存储所述未回收数据所属的所述新的存储条带组的存储属性,生成空间回收响应信息,并发送所述空间回收响应信息。
- 根据权利要求14所述的方法,其特征在于,在所述依据空间回收指令将存储条带组中存储条带中的未回收数据存储至网络硬盘中的新的存储条带组之前,所述方法还包括:发送新的存储条带创建请求;接收返回的新的存储条带标识;向所述网络硬盘发送未回收数据反馈请求;缓存所述网络硬盘返回的未回收数据。
- 根据权利要求15所述的方法,其特征在于,所述依据空间回收指令将存储条带组中存储条带中的未回收数据存储至网络硬盘中的新的存储条带组,包括:依据预设纠删码规则,在所述新的存储条带中,为所述未回收数据新的分配对应的存储数据块;将所述未回收数据存储至所述新的存储条带中的存储数据块中。
- 根据权利要求16所述的方法,其特征在于,所述依据存储所述未回收数据所属的所述新的存储条带组的存储属性生成空间回收响应信息,包括:依据存储于所述新的存储条带中的未回收数据的数据存储状态、所述新的存储条带的存储状态、所述待回收数据的数据存储状态和所述待回收数据所属的存储条带组的存储状态生成所述空间回收响应信息。
- 一种网络硬盘空间的回收装置,其特征在于,应用于网络硬盘空间的回收系统中的数据服务器,包括:第一接收模块,用于接收空间回收请求,其中,所述空间回收请求,用于指示回收待回收数据所属的存储空间;查询模块,用于查询所述待回收数据对应的预设存储列表组,得到所述待回收数据所属的存储条带组;分组模块,用于依据所述存储条带组中的未回收数据对所述存储条带组中的存储条带进行分组,得到所述未回收数据所属的存储条带;发送模块,用于依据所述存储条带生成空间回收指令,并发送所述空间回收指令;第二接收模块,用于接收空间回收响应信息。
- 根据权利要求18所述的装置,其特征在于,所述查询模块包括:第一查询单元,用于在所述存储列表组包括数据存储信息表、存储条带信息表和网络硬盘信息表的情况下,在所述数据存储信息表中查询所述待回收数据是否存在;第二查询单元,用于在查询结果为是的情况下,在所述数据存储信息表中查询所述待回收数据所属的存储条带组;提取单元,用于提取所述待回收数据在所述数据存储信息表中的偏移量;第三查询单元,用于依据所述偏移量在所述存储条带信息表进行查询,得到所述待回收数据对应的存储数据块。
- 根据权利要求19所述的装置,其特征在于,所述分组模块包括:状态更改单元,用于将所述待回收数据所属的存储数据块的存储状态,在所述存储条带信息表中更改为删除;第一统计单元,用于统计所述存储状态为删除的存储数据块的个数,得到所述未回收数据对应的存储数据块的个数;第二统计单元,用于依据所述未回收数据对应的存储数据块的个数,得到所述未回收数据对应的存储条带。
- 根据权利要求20所述的装置,其特征在于,所述装置还包括:判断单元,用于在所述得到所述未回收数据对应的存储数据块个数之前,判断所述个数是否大于或等于预设阈值;标记单元,用于在判断结果为是的情况下,标记所述待回收数据对应的存储数据块所属的存储条带的存储状态为待回收。
- 根据权利要求19所述的装置,其特征在于,所述装置还包括:第三接收模块,用于在所述数据服务器接收返回的空间回收响应信息之前,接收新的存储条带创建请求;标识创建模块,用于依据所述新的存储条带创建请求,创建新的存储条带标识;标识发送模块,用于发送所述新的存储条带标识。
- 根据权利要求19所述的装置,其特征在于,所述装置还包括:第一状态更新模块,用于在所述空间回收响应信息包括:所述未回收数据的数据存储状态和所述待回收数据的数据存储状态,以及所述未回收数据所属存储条带的存储状态和所述待回收数据所属存储条带的存储状态的情况 下,在所述数据服务器接收返回的空间回收响应信息之后,依据接收到的数据存储状态,更新所述数据存储信息表;第二状态更新模块,用于依据接收到的存储条带的存储状态,更新所述存储条带信息表。
- 一种网络硬盘空间的回收装置,其特征在于,应用于网络硬盘空间的回收系统中的存储服务器,包括:接收模块,用于接收空间回收指令;存储模块,用于依据空间回收指令将存储条带组中存储条带中的未回收数据存储至网络硬盘中的新的存储条带组;回收模块,用于删除待回收数据,并回收所述待回收数据所属的存储条带组;信息生成模块,用于依据存储所述未回收数据所属的所述新的存储条带组的存储属性生成空间回收响应信息;第一发送模块,用于发送所述空间回收响应信息。
- 根据权利要求24所述的装置,其特征在于,所述装置还包括:第二发送模块,用于在所述存储服务器依据空间回收指令将存储条带组中存储条带中的未回收数据存储至网络硬盘中的新的存储条带组之前,发送新的存储条带创建请求;标识接收模块,用于接收新的存储条带标识;请求发送模块,用于向所述网络硬盘发送未回收数据反馈请求;缓存模块,用于缓存所述网络硬盘返回的未回收数据。
- 根据权利要求25所述的装置,其特征在于,所述存储模块包括:分配单元,用于依据预设纠删码规则,在所述新的存储条带中,为所述未回收数据新的分配对应的存储数据块;存储单元,用于将所述未回收数据存储至所述新的存储条带中的存储数据块中。
- 根据权利要求26所述的装置,其特征在于,所述信息生成模块包括:信息生成单元,用于依据存储于所述新的存储条带中的未回收数据的数据存储状态、所述新的存储条带的存储状态、所述待回收数据的数据存储状态和所述待回收数据所属的存储条带组的存储状态生成所述空间回收响应信息。
- 一种数据服务器,可以包括:处理器、存储器、通信接口和总线;所述处理器、所述存储器和所述通信接口通过所述总线连接并完成相互间的通信;所述存储器存储可执行代码;所述处理器通过读取所述存储器中存储的可执行代码来运行与所述可执行代码对应的程序,以用于执行权利要求8-13任意一项所述的网络硬盘空间的回收方法。
- 一种存储介质,其特征在于,用于存储可执行代码,所述可执行代码用于执行权利要求8-13任意一项所述的网络硬盘空间的回收方法。
- 一种可执行代码,其特征在于,用于执行权利要求8-13任意一项所述的网络硬盘空间的回收方法。
- 一种存储服务器,可以包括:处理器、存储器、通信接口和总线;所述处理器、所述存储器和所述通信接口通过所述总线连接并完成相互间的通信;所述存储器存储可执行代码;所述处理器通过读取所述存储器中存储的可执行代码来运行与所述可执行代码对应的程序,以用于执行权利要求14-17任意一项所述的网络硬盘空间的回收方法。
- 一种存储介质,其特征在于,用于存储可执行代码,所述可执行代码用于执行权利要求14-17任意一项所述的网络硬盘空间的回收方法。
- 一种可执行代码,其特征在于,用于执行权利要求14-17任意一项所述的网络硬盘空间的回收方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610392634.5 | 2016-06-03 | ||
CN201610392634.5A CN107463333B (zh) | 2016-06-03 | 2016-06-03 | 网络硬盘空间的回收方法、装置和系统 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2017206436A1 true WO2017206436A1 (zh) | 2017-12-07 |
Family
ID=60478466
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2016/104494 WO2017206436A1 (zh) | 2016-06-03 | 2016-11-04 | 网络硬盘空间的回收方法、装置和系统 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN107463333B (zh) |
WO (1) | WO2017206436A1 (zh) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110568993A (zh) * | 2019-08-06 | 2019-12-13 | 新华三技术有限公司成都分公司 | 一种数据更新方法及相关装置 |
CN110874182A (zh) * | 2018-08-31 | 2020-03-10 | 杭州海康威视系统技术有限公司 | 一种条带索引的处理方法、装置及设备 |
CN111949434A (zh) * | 2019-05-17 | 2020-11-17 | 华为技术有限公司 | 磁盘冗余阵列raid管理方法、raid控制器和系统 |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111767010B (zh) * | 2020-06-30 | 2023-03-14 | 杭州海康威视系统技术有限公司 | 数据处理方法、装置、设备及存储介质 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103514098A (zh) * | 2012-06-29 | 2014-01-15 | 伊姆西公司 | 用于回收存储空间的方法和系统 |
CN103744798A (zh) * | 2013-12-17 | 2014-04-23 | 记忆科技(深圳)有限公司 | 固态硬盘的垃圾回收方法 |
US8768898B1 (en) * | 2007-04-26 | 2014-07-01 | Netapp, Inc. | Performing direct data manipulation on a storage device |
CN103973731A (zh) * | 2013-01-29 | 2014-08-06 | 中兴通讯股份有限公司 | 一种云存储的回收站管理方法和装置 |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103902465B (zh) * | 2014-03-19 | 2017-02-08 | 华为技术有限公司 | 一种固态硬盘垃圾回收的方法、系统和固态硬盘控制器 |
-
2016
- 2016-06-03 CN CN201610392634.5A patent/CN107463333B/zh active Active
- 2016-11-04 WO PCT/CN2016/104494 patent/WO2017206436A1/zh active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8768898B1 (en) * | 2007-04-26 | 2014-07-01 | Netapp, Inc. | Performing direct data manipulation on a storage device |
CN103514098A (zh) * | 2012-06-29 | 2014-01-15 | 伊姆西公司 | 用于回收存储空间的方法和系统 |
CN103973731A (zh) * | 2013-01-29 | 2014-08-06 | 中兴通讯股份有限公司 | 一种云存储的回收站管理方法和装置 |
CN103744798A (zh) * | 2013-12-17 | 2014-04-23 | 记忆科技(深圳)有限公司 | 固态硬盘的垃圾回收方法 |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110874182A (zh) * | 2018-08-31 | 2020-03-10 | 杭州海康威视系统技术有限公司 | 一种条带索引的处理方法、装置及设备 |
CN110874182B (zh) * | 2018-08-31 | 2023-12-26 | 杭州海康威视系统技术有限公司 | 一种条带索引的处理方法、装置及设备 |
CN111949434A (zh) * | 2019-05-17 | 2020-11-17 | 华为技术有限公司 | 磁盘冗余阵列raid管理方法、raid控制器和系统 |
CN110568993A (zh) * | 2019-08-06 | 2019-12-13 | 新华三技术有限公司成都分公司 | 一种数据更新方法及相关装置 |
CN110568993B (zh) * | 2019-08-06 | 2022-04-12 | 新华三技术有限公司成都分公司 | 一种数据更新方法及相关装置 |
Also Published As
Publication number | Publication date |
---|---|
CN107463333B (zh) | 2019-12-20 |
CN107463333A (zh) | 2017-12-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
TWI702506B (zh) | 用於合併樹廢棄項目指標之系統、機器可讀媒體及機器實施之方法 | |
TWI719281B (zh) | 用於串流選擇之系統、機器可讀媒體、及機器實施之方法 | |
CN103064639B (zh) | 数据存储方法及装置 | |
US9448927B1 (en) | System and methods for removing obsolete data in a distributed system of hybrid storage and compute nodes | |
WO2017206436A1 (zh) | 网络硬盘空间的回收方法、装置和系统 | |
WO2016086819A1 (zh) | 将数据写入叠瓦状磁记录smr硬盘的方法及装置 | |
CN102511044B (zh) | 一种数据删除方法及装置 | |
CN102541968B (zh) | 一种索引方法 | |
CN103870514B (zh) | 重复数据删除方法和装置 | |
CN103995855B (zh) | 存储数据的方法和装置 | |
CN110399310B (zh) | 一种存储空间的回收方法及装置 | |
CN107092439A (zh) | 一种数据存储的方法及设备 | |
CN106776375A (zh) | 一种磁盘内部的数据缓存方法及装置 | |
CN107786638A (zh) | 一种数据处理方法、装置及系统 | |
CN103608866A (zh) | 用于闪存存储器的数据擦除方法及装置 | |
CN109918352B (zh) | 存储器系统和存储数据的方法 | |
CN103678337A (zh) | 数据清除方法、装置及系统 | |
CN105677238A (zh) | 一种基于分布式存储重复数据删除的虚拟机系统盘的方法 | |
CN105493080B (zh) | 基于上下文感知的重复数据删除的方法和装置 | |
CN106446044A (zh) | 存储空间回收方法及装置 | |
CN104050057B (zh) | 一种历史感知的数据去重碎片消除方法与系统 | |
US10289345B1 (en) | Contention and metadata write amplification reduction in log structured data storage mapping | |
CN106528703A (zh) | 一种重复删除的模式切换方法及装置 | |
CN112052193B (zh) | 一种垃圾回收方法、装置、可读存储介质及电子设备 | |
US11397706B2 (en) | System and method for reducing read amplification of archival storage using proactive consolidation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16903818 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 16903818 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 100519) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 16903818 Country of ref document: EP Kind code of ref document: A1 |