CN113656208B - Data processing method, device, equipment and storage medium of distributed storage system - Google Patents
Data processing method, device, equipment and storage medium of distributed storage system Download PDFInfo
- Publication number
- CN113656208B CN113656208B CN202110945150.XA CN202110945150A CN113656208B CN 113656208 B CN113656208 B CN 113656208B CN 202110945150 A CN202110945150 A CN 202110945150A CN 113656208 B CN113656208 B CN 113656208B
- Authority
- CN
- China
- Prior art keywords
- disk
- disks
- failed
- preset
- data processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0793—Remedial or corrective actions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
- G06F11/0727—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a storage system, e.g. in a DASD or network based storage system
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the disclosure discloses a data processing method, a device, equipment and a storage medium of a distributed storage system, wherein the method comprises the following steps: in response to detecting a disk failure, repairing the failed disk with a repair tool; obtaining disc attributes of M failed discs failed to repair, and obtaining N discs from a preset disc set according to the disc attributes; wherein, the disk attribute at least comprises: disk type, disk capacity; m, N is a natural number; creating mounting points which are matched with the N disks in number, and mounting the N disks on the mounting points; and sending prompt information to the client so that the client processes data. According to the technical scheme, the operation that the operation and maintenance personnel need to repeatedly modify uuid and mount catalogs in the mount script after replacing the disk is saved, the risk caused by misoperation is reduced, automatic disk management is realized, and the convenience and the high efficiency of data processing of the distributed storage system are greatly improved.
Description
Technical Field
The disclosure relates to the technical field of computer application, and in particular relates to a data processing method, device and equipment of a distributed storage system and a storage medium.
Background
With the rapid development of computer technology, distributed storage systems are becoming more and more widely used. The distributed storage system consists of a plurality of storage servers, each storage server is provided with dozens of magnetic disks, and each magnetic disk is provided with a local file system as a data storage unit. In the distributed operating system, because the read-write load of the disk is high for a long time, the failure of the disk device frequently occurs, so that the failed disk needs to be frequently identified and repaired, and the synchronism of the processing of the client is ensured. However, because the number of failed disks is often large, when the repair fails and the replacement of the disks is needed, the operation and maintenance personnel needs to repeatedly execute the mount command to mount the disks on the corresponding mounting points one by one, and the efficiency of the mode is low, so that the distributed storage system is influenced to normally provide services to the outside.
Disclosure of Invention
In order to solve the problems in the related art, embodiments of the present disclosure provide a distributed storage system data processing method, apparatus, device, and storage medium.
In a first aspect, an embodiment of the present disclosure provides a data processing method of a distributed storage system.
Specifically, the data processing method of the distributed storage system comprises the following steps:
in response to detecting a disk failure, repairing the failed disk with a repair tool;
obtaining disc attributes of M failed discs failed to repair, and obtaining N discs from a preset disc set according to the disc attributes; wherein, the disk attribute at least comprises: disk type, disk capacity; m, N is a natural number;
creating mounting points which are matched with the N disks in number, and mounting the N disks on the mounting points;
and sending prompt information to the client so that the client processes data.
With reference to the first aspect, in a first implementation manner of the first aspect, the matching, according to the disc attribute, the N discs from a preset disc set includes:
classifying the magnetic disks in a preset magnetic disk set, and sorting according to the capacity of the magnetic disks;
and matching the disk types of the M failed disks based on the classification result, and selecting a proper number of disks in the preset disk set based on the ordering result to match the disk capacities of the M failed disks.
With reference to the first implementation manner of the first aspect, in a second implementation manner of the first aspect, the matching, based on the classification result, the disk types of the M failed disks includes:
if the data cannot be matched with the same type of disk, adapting other types of disks according to the storage data and the storage risk level of the M failed disks.
With reference to the second implementation manner of the first aspect, in a third implementation manner of the first aspect, the storage risk level is determined by detecting an environmental parameter of the server by using a sensor.
With reference to the first aspect, the first implementation manner of the first aspect, the second implementation manner of the first aspect, and the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the method further includes:
when detecting that P disks in M failed disks are not loaded, deleting mounting points corresponding to the P disks;
and when the P disks are detected to be reloaded, adding a preset disk set.
With reference to the first aspect, in a fifth implementation manner of the first aspect, the client uses a DMA mechanism to perform data processing.
In a second aspect, in an embodiment of the present disclosure, a distributed storage system data processing apparatus is provided.
Specifically, the distributed storage system data processing apparatus includes:
a repair module configured to repair a failed disk with a repair tool in response to detecting a disk failure;
the matching module is configured to acquire disc attributes of M failed discs failed to repair, and N discs are obtained from the preset disc set in a matching mode according to the disc attributes; wherein, the disk attribute at least comprises: disk type, disk capacity; m, N is a natural number;
the mounting module is configured to create mounting points with the number of N magnetic discs being matched with the number of the N magnetic discs, and mount the N magnetic discs on the mounting points;
and the sending module is configured to send prompt information to the client so as to enable the client to perform data processing.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including a memory and a processor, wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method of any one of the first aspects.
In a fourth aspect, in an embodiment of the present disclosure, there is provided a readable storage medium having stored thereon computer instructions which, when executed by a processor, implement a method according to any of the first aspects.
In a fifth aspect, embodiments of the present disclosure provide a computer program product comprising computer instructions which, when executed by a processor, implement the method steps as described in any of the first aspects.
According to the technical scheme provided by the embodiment of the disclosure, in response to detecting the disk failure, the failed disk is repaired by using a repairing tool; obtaining disc attributes of M failed discs failed to repair, and obtaining N discs from a preset disc set according to the disc attributes; wherein, the disk attribute at least comprises: disk type, disk capacity; m, N is a natural number; creating mounting points which are matched with the N disks in number, and mounting the N disks on the mounting points; and sending prompt information to the client so that the client processes data. According to the technical scheme, the N disks are selected from the preset disk set to match M failed disks failed to repair, and then the mounting points with the number being adapted are created to realize batch disk replacement, so that a client can execute disk read-write operation, the operation that an operation and maintenance person needs to repeatedly modify uuid (Universally Unique Identifier, universal unique identification code) in a mount script and mount catalogue after replacing the disks is saved, risks caused by misoperation are reduced, automatic disk management is realized, and convenience and high efficiency of data processing of a distributed storage system are greatly improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
Other features, objects and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:
FIG. 1 illustrates a flow chart of a distributed storage system data processing method according to an embodiment of the present disclosure;
FIG. 2 illustrates a flowchart of matching N disks from a preset disk set according to the disk attributes according to an embodiment of the present disclosure;
FIG. 3 illustrates a flow chart of a distributed storage system data processing method according to an embodiment of the present disclosure;
FIG. 4 illustrates a block diagram of a distributed storage system data processing apparatus according to an embodiment of the present disclosure;
fig. 5 shows a block diagram of an electronic device according to an embodiment of the disclosure;
FIG. 6 illustrates a schematic diagram of a computer system suitable for use in implementing a distributed storage system data processing method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, portions irrelevant to description of the exemplary embodiments are omitted in the drawings.
In this disclosure, it should be understood that terms such as "comprises" or "comprising," etc., are intended to indicate the presence of features, numbers, steps, acts, components, portions, or combinations thereof disclosed in this specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, acts, components, portions, or combinations thereof are present or added.
In addition, it should be noted that, without conflict, the embodiments of the present disclosure and features of the embodiments may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
In the present disclosure, the acquisition of user information or user data is an operation that is authorized, confirmed, or actively selected by the user.
In the distributed operating system, because the read-write load of the disk is high for a long time, the failure of the disk device frequently occurs, so that the failed disk needs to be frequently identified and repaired, and the synchronism of the processing of the client is ensured. However, because the number of failed disks is often large, when the repair fails and the replacement of the disks is needed, the operation and maintenance personnel needs to repeatedly execute the mount command to mount the disks on the corresponding mounting points one by one, and the efficiency of the mode is low, so that the distributed storage system is influenced to normally provide services to the outside.
In view of the above technical problems, according to the technical solution provided in the embodiments of the present disclosure, in response to detecting a disk failure, a repair tool is used to repair the failed disk; obtaining disc attributes of M failed discs failed to repair, and obtaining N discs from a preset disc set according to the disc attributes; wherein, the disk attribute at least comprises: disk type, disk capacity; m, N is a natural number; creating mounting points which are matched with the N disks in number, and mounting the N disks on the mounting points; and sending prompt information to the client so that the client processes data. According to the technical scheme, the N disks are selected from the preset disk set to match M failed disks failed to repair, and then the mounting points with the number being adapted are created to realize batch disk replacement, so that a client can execute disk read-write operation, the operation that an operation and maintenance person needs to repeatedly modify uuid (Universally Unique Identifier, universal unique identification code) in a mount script and mount catalogue after replacing the disks is saved, risks caused by misoperation are reduced, automatic disk management is realized, and convenience and high efficiency of data processing of a distributed storage system are greatly improved.
FIG. 1 illustrates a flow chart of a distributed storage system data processing method according to an embodiment of the present disclosure. As shown in fig. 1, the distributed storage system data processing method includes steps S101 to S104.
In step S101, in response to detecting a disk failure, repairing the failed disk with a repair tool;
in step S102, obtaining disc attributes of M failed discs failed to repair, and obtaining N discs from a preset disc set according to the disc attributes; wherein, the disk attribute at least comprises: disk type, disk capacity; m, N is a natural number;
in step S103, a mounting point adapted to the number of the N disks is created, and the N disks are mounted on the mounting point;
in step S104, a prompt message is sent to the client, so that the client performs data processing.
According to the data processing method of the distributed storage system, N disks are selected from the preset disk set to match M failed disks failed to repair, and then the mounting points with the number being matched are created to realize batch disk replacement, so that a client performs disk read-write operation, the operation that an operation and maintenance person needs to repeatedly modify uuid and mounting catalogues in a mount script after replacing the disks is saved, risks caused by misoperation are reduced, automatic disk management is realized, and convenience and high efficiency of data processing of the distributed storage system are greatly improved.
In an embodiment of the present disclosure, the disk failures referred to in the present disclosure may be pure hardware failures such as head damage, bad tracks, etc., or may be software failures such as logical bad tracks caused after abnormal shutdown, etc. The repair tool utilized may be disk eidolon disk to repair a disk defect, or fsck command to detect and repair a file system, or other repair tools known in the art, which is not limiting of the present disclosure.
In an embodiment of the present disclosure, the preset disk set may be a set of free disks in the distributed storage system that do not allocate data to the disks. The preset disk set may include types of mechanical hard disk HDD, solid state hard disk SSD, hybrid hard disk SSHD, and the like, and the hard disk interface may employ IDE, SATA, SCSI, SAS, which is not limited by the present disclosure.
In one embodiment of the present disclosure, the preset number of disks in the disk set is not fixed, but may be dynamically updated. Specifically, a preset capacity threshold may be set, and before the free disk is added to the preset disk set, if the capacity of the free disk is not greater than the preset capacity threshold, the addition is refused.
In an embodiment of the disclosure, for M failed disks failed to repair, the disk type and the disk capacity of the failed disk are obtained, so that N disks of the same type and capacity can be preferentially adapted from a preset disk set to replace the failed disk, so as to maintain consistency of the failed disk before and after the failure, thereby meeting the requirement of client data processing. In some cases, in order to avoid the influence of the performance degradation of the disk on the customer experience, when N disks are matched, the disk with performance and capacity higher than those of the failed disk may be replaced, or a plurality of disks may be used to replace one failed disk to meet the requirements of the client on the capacity and performance of the disk. Those skilled in the art may configure the matching condition according to the specific situation, which is not limited by the present disclosure.
In an embodiment of the present disclosure, the client performs data processing using a direct memory access DMA mechanism in step S104. The DMA mechanism is used without depending on a large amount of interrupt load of the CPU, so that the data processing efficiency is improved. In the method, the client side writes the file into the disk device in a concurrent read-write mode, so that the data processing efficiency is further improved.
FIG. 2 illustrates a flowchart of matching N disks from a preset disk set according to the disk attributes according to an embodiment of the present disclosure.
As shown in fig. 2, in step S102, matching N disks from a preset disk set according to the disk attribute includes steps S201 to S202.
In step S201, classifying the disks in the preset disk set, and sorting according to the size of the disk capacity;
in step S202, the disk types of the M failed disks are matched based on the classification result, and a proper number of disks in the preset disk set are selected to match the disk capacities of the M failed disks based on the sorting result.
In one embodiment of the present disclosure, the preset disk set is composed of disks of different types and capacities. In order to improve the efficiency of matching M failed disks from the preset disk set, the disks in the preset disk set may be classified first, and then sorted according to the capacity of the disks. When matching is performed, the disk types of M failed disks are matched based on the classification result, and then a proper number of disks are selected from the same disk type according to the disk capacity. The number of the selected disks is not smaller than the sum of the disk capacities of the M fault disks, so that the storage requirement of the client is met.
In an embodiment of the present disclosure, the matching the disk types of the M failed disks in step S202 based on the classification result includes:
if the data cannot be matched with the same type of disk, adapting other types of disks according to the storage data and the storage risk level of the M failed disks.
In this embodiment, if the same type of disk as the failed disk does not exist in the preset disk set, other types of disks may be adapted. The adaptation can be specifically performed according to the storage data and the storage risk level of the fault disk. For example, the solid state disk is adapted when the requirement of the storage data on the read-write speed is high, and a plurality of mechanical hard disks are adapted when the requirement of the storage data on the storage capacity is high. The storage risk level is used for measuring disaster tolerance and backup requirements of data, if the storage risk level is higher, a plurality of different types of disks can be configured for backup so as to reduce risk, and conversely, 1-2 types of disks can be configured for backup. It should be noted that, the foregoing is a schematic description of the disk adaptation type, and in actual adaptation, those skilled in the art may also combine other factors including the type of disk, the number of various types of disks, the capacity of the disk, the interface of the disk, and the like, which are included in the preset disk set, and are not described herein.
In one embodiment of the disclosure, the storage risk level is determined by detecting an environmental parameter of the server using a sensor. For example, vibration sensors are used to detect vibration parameters of the environment in which the disks are located, and to determine whether the environment is susceptible to disk failure, so as to adapt to a proper number and type of disks to replace the failed disk.
FIG. 3 illustrates a flow chart of a distributed storage system data processing method according to an embodiment of the present disclosure. As shown in fig. 3, the distributed storage system data processing method further includes steps S301 to S306.
In step S301, in response to detecting a disk failure, repairing the failed disk with a repair tool;
in step S302, obtaining disc attributes of M failed discs failed to repair, and obtaining N discs from a preset disc set according to the disc attributes; wherein, the disk attribute at least comprises: disk type, disk capacity; m, N is a natural number;
in step S303, when it is detected that P disks are not loaded in M failed disks, mounting points corresponding to the P disks are deleted;
in step S304, when it is detected that the P disks are reloaded, adding a preset disk set;
in step S305, a mounting point adapted to the number of the N disks is created, and the N disks are mounted on the mounting point;
in step S306, a prompt message is sent to the client, so that the client performs data processing.
The related art content related to steps S301, S302, S305, S306 in the embodiments of the present disclosure may be referred to the above steps S101-S104, and the disclosure is not repeated here.
The difference between the embodiment of the present disclosure and the embodiment of steps S101 to S104 described in fig. 1 is that, in the embodiment of the present disclosure, it is considered that in M failed disks failed to repair, P disks may not be loaded due to a unplugged disk or offline reason, and when P disks are reloaded, the P disks are not actually failed disks (no hardware failure is occurred), so that mounting point information corresponding to the P disks may be deleted, and a preset disk set is added to update the disks in the preset disk set, and further the P disks may also participate in a subsequent adaptation link, thereby effectively utilizing disk resources.
FIG. 4 illustrates a block diagram of a distributed storage system data processing apparatus according to an embodiment of the present disclosure. The apparatus may be implemented as part or all of an electronic device by software, hardware, or a combination of both.
As shown in fig. 4, the distributed storage system data processing apparatus includes a repair module 410, a matching module 420, a mounting module 430, and a transmitting module 440.
The repair module 410 is configured to repair a failed disk with a repair tool in response to detecting a disk failure;
the matching module 420 is configured to obtain disc attributes of M failed discs failed to repair, and obtain N discs from a preset disc set according to the disc attributes; wherein, the disk attribute at least comprises: disk type, disk capacity; m, N is a natural number;
the mounting module 430 is configured to create mounting points that are adapted to the number of the N disks, and mount the N disks onto the mounting points;
the sending module 440 is configured to send a prompt message to a client, so that the client performs data processing.
According to the data processing device of the distributed storage system, N disks are selected from the preset disk set to match M failed disks failed to repair, and then the mounting points with the number being matched are created to realize batch disk replacement, so that a client executes disk read-write operation, the operation that operation and maintenance personnel need to repeatedly modify uuid and mounting catalogues in a mount script after replacing the disks is saved, risks caused by misoperation are reduced, automatic disk management is realized, and convenience and high efficiency of data processing of the distributed storage system are greatly improved.
In an embodiment of the present disclosure, the matching module 420 includes:
the sorting unit is configured to sort the disks in the preset disk set and sort the disks according to the capacity of the disks;
and the matching subunit is configured to match the disk types of the M failed disks based on the classification result, and select a proper number of disks in the preset disk set based on the ordering result so as to match the disk capacities of the M failed disks.
In an embodiment of the present disclosure, the matching subunit, based on the classification result, matches the disk type portions of the M failed disks, is configured to:
if the data cannot be matched with the same type of disk, adapting other types of disks according to the storage data and the storage risk level of the M failed disks.
In one embodiment of the disclosure, the storage risk level is determined by detecting an environmental parameter of the server using a sensor.
In an embodiment of the present disclosure, the distributed storage system data processing apparatus further includes:
the deleting module is configured to delete mounting points corresponding to P disks when detecting that the P disks in the M failed disks are not loaded;
and the updating module is configured to add a preset disk set when the P disks are detected to be reloaded.
In one embodiment of the disclosure, the client uses a DMA mechanism to process data.
The present disclosure also discloses an electronic device, and fig. 5 shows a block diagram of the electronic device according to an embodiment of the present disclosure.
As shown in fig. 5, the electronic device 500 includes a memory 501 and a processor 502; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory 501 is used to store one or more computer instructions that are executed by the processor 502 to implement a method in accordance with embodiments of the present disclosure.
In response to detecting a disk failure, repairing the failed disk with a repair tool;
obtaining disc attributes of M failed discs failed to repair, and obtaining N discs from a preset disc set according to the disc attributes; wherein, the disk attribute at least comprises: disk type, disk capacity; m, N is a natural number;
creating mounting points which are matched with the N disks in number, and mounting the N disks on the mounting points;
and sending prompt information to the client so that the client processes data.
In an embodiment of the present disclosure, the matching, according to the disc attribute, the N discs from a preset disc set includes:
classifying the magnetic disks in a preset magnetic disk set, and sorting according to the capacity of the magnetic disks;
and matching the disk types of the M failed disks based on the classification result, and selecting a proper number of disks in the preset disk set based on the ordering result to match the disk capacities of the M failed disks.
In an embodiment of the present disclosure, the matching the disk types of the M failed disks based on the classification result includes:
if the data cannot be matched with the same type of disk, adapting other types of disks according to the storage data and the storage risk level of the M failed disks.
In one embodiment of the disclosure, the storage risk level is determined by detecting an environmental parameter of the server using a sensor.
In an embodiment of the present disclosure, further comprising:
when detecting that P disks in M failed disks are not loaded, deleting mounting points corresponding to the P disks;
and when the P disks are detected to be reloaded, adding a preset disk set.
In one embodiment of the disclosure, the client uses a DMA mechanism to process data.
FIG. 6 illustrates a schematic diagram of a computer system suitable for use in implementing a distributed storage system data processing method according to an embodiment of the present disclosure.
As shown in fig. 6, the computer system 600 includes a processing unit 601, which can execute various processes in the above-described embodiments according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the system 600 are also stored. The CPU601, ROM602, and RAM603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608. The processing unit 601 may be implemented as a processing unit such as CPU, GPU, TPU, FPGA, NPU.
In particular, according to embodiments of the present disclosure, the methods described above may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising computer instructions which, when executed by a processor, implement the method steps described above. In such embodiments, the computer program product may be downloaded and installed from a network via the communication portion 609, and/or installed from the removable medium 611.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules referred to in the embodiments of the present disclosure may be implemented in software or in programmable hardware. The units or modules described may also be provided in a processor, the names of which in some cases do not constitute a limitation of the unit or module itself.
As another aspect, the present disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or the computer system in the above-described embodiments; or may be a computer-readable storage medium, alone, that is not assembled into a device. The computer-readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the present disclosure.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention referred to in this disclosure is not limited to the specific combination of features described above, but encompasses other embodiments in which any combination of features described above or their equivalents is contemplated without departing from the inventive concepts described. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).
Claims (9)
1. A distributed storage system data processing method, comprising:
in response to detecting a disk failure, repairing the failed disk with a repair tool;
obtaining disc attributes of M failed discs failed to repair, and obtaining N discs from a preset disc set according to the disc attributes; wherein, the disk attribute at least comprises: disk type, disk capacity; m, N is a natural number;
when detecting that P disks in M failed disks are not loaded, deleting mounting points corresponding to the P disks;
when the P disks are detected to be reloaded, adding the P disks into the preset disk set;
creating mounting points which are matched with the N disks in number, and mounting the N disks on the mounting points;
and sending prompt information to the client so that the client processes data.
2. The method of claim 1, wherein the matching the N disks from the preset disk set according to the disk attribute includes:
classifying the magnetic disks in a preset magnetic disk set, and sorting according to the capacity of the magnetic disks;
and matching the disk types of the M failed disks based on the classification result, and selecting a proper number of disks in the preset disk set based on the ordering result to match the disk capacities of the M failed disks.
3. The method of claim 2, the matching disk types of the M failed disks based on the classification result, comprising:
if the data cannot be matched with the same type of disk, adapting other types of disks according to the storage data and the storage risk level of the M failed disks.
4. A method according to claim 3, wherein the storage risk level is determined by detecting an environmental parameter of the server using a sensor.
5. The method of claim 1, wherein the client uses a DMA mechanism for data processing.
6. A distributed storage system data processing apparatus comprising:
a repair module configured to repair a failed disk with a repair tool in response to detecting a disk failure;
the matching module is configured to acquire disc attributes of M failed discs failed to repair, and N discs are obtained from the preset disc set in a matching mode according to the disc attributes; wherein, the disk attribute at least comprises: disk type, disk capacity; m, N is a natural number;
the deleting module is configured to delete mounting points corresponding to P disks when detecting that the P disks in the M failed disks are not loaded;
the updating module is configured to add the preset disk set when the P disks are detected to be reloaded;
the mounting module is configured to create mounting points with the number of N magnetic discs being matched with the number of the N magnetic discs, and mount the N magnetic discs on the mounting points;
and the sending module is configured to send prompt information to the client so as to enable the client to perform data processing.
7. An electronic device includes a memory and a processor; wherein the memory is for storing one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps of any of claims 1-5.
8. A readable storage medium having stored thereon computer instructions, which when executed by a processor, implement the method steps of any of claims 1-5.
9. A computer program product comprising computer instructions which, when executed by a processor, implement the method steps of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110945150.XA CN113656208B (en) | 2021-08-17 | 2021-08-17 | Data processing method, device, equipment and storage medium of distributed storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110945150.XA CN113656208B (en) | 2021-08-17 | 2021-08-17 | Data processing method, device, equipment and storage medium of distributed storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113656208A CN113656208A (en) | 2021-11-16 |
| CN113656208B true CN113656208B (en) | 2023-06-16 |
Family
ID=78480719
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110945150.XA Active CN113656208B (en) | 2021-08-17 | 2021-08-17 | Data processing method, device, equipment and storage medium of distributed storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113656208B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102419995A (en) * | 2010-09-27 | 2012-04-18 | 株式会社日立制作所 | Information access device and information access method |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104731915B (en) * | 2015-03-24 | 2019-03-22 | 上海爱数信息技术股份有限公司 | Disk unit mapping method in a kind of distributed memory system |
| US10101913B2 (en) * | 2015-09-02 | 2018-10-16 | Commvault Systems, Inc. | Migrating data to disk without interrupting running backup operations |
| CN108153622B (en) * | 2016-12-06 | 2021-08-31 | 华为技术有限公司 | Fault processing method, device and equipment |
| CN108733518B (en) * | 2017-04-17 | 2021-07-09 | 伊姆西Ip控股有限责任公司 | Method, apparatus, and computer-readable medium for managing a storage system |
| US11176001B2 (en) * | 2018-06-08 | 2021-11-16 | Google Llc | Automated backup and restore of a disk group |
| US10936442B2 (en) * | 2018-07-06 | 2021-03-02 | EMC IP Holding Company LLC | Simultaneous file level recovery from multiple backups using a proxy virtual machine |
| CN109254730A (en) * | 2018-08-29 | 2019-01-22 | 郑州云海信息技术有限公司 | A kind of batch realizes the method and device of automatized magnetic disk management under numerous linux systems |
| CN110045929B (en) * | 2019-04-19 | 2020-06-19 | 苏州浪潮智能科技有限公司 | Mounting point management method and related device |
-
2021
- 2021-08-17 CN CN202110945150.XA patent/CN113656208B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102419995A (en) * | 2010-09-27 | 2012-04-18 | 株式会社日立制作所 | Information access device and information access method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113656208A (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10725692B2 (en) | Data storage method and apparatus | |
| CN102460398B (en) | Source classification for performing deduplication in a backup operation | |
| US10204019B1 (en) | Systems and methods for instantiation of virtual machines from backups | |
| US9104574B2 (en) | System and method for software application remediation | |
| US7644301B2 (en) | Fault management system in multistage copy configuration | |
| CN106557266B (en) | Method and apparatus for redundant array of independent disks RAID | |
| CN109558260B (en) | Kubernetes fault elimination system, method, equipment and medium | |
| US9317383B2 (en) | Communication of conditions at a primary storage controller to a host | |
| CN105354102B (en) | A kind of method and apparatus of file system maintenance and reparation | |
| CN114816820A (en) | Method, device, equipment and storage medium for repairing chproxy cluster fault | |
| CN113157491A (en) | Data backup method and device, communication equipment and storage medium | |
| CN113656208B (en) | Data processing method, device, equipment and storage medium of distributed storage system | |
| CN110968456B (en) | Method and device for processing fault disk in distributed storage system | |
| CN113268206B (en) | Network target range resource hot plug implementation method and system | |
| CN115757270A (en) | File version management system and method and computer equipment | |
| US20130179415A1 (en) | Reducing Duplicate Information when Reporting System Incidents | |
| CN114969206A (en) | Data processing method, device, equipment and storage medium | |
| CN108196979B (en) | Data backup method and device | |
| CN106648985A (en) | Disaster-tolerant repair method and device of text database | |
| CN117614973B (en) | File storage method based on multi-cloud architecture | |
| US8935494B2 (en) | Backing up an image in a computing system | |
| KR101449202B1 (en) | System and method for data copy and restoration in steel process middleware | |
| US11475159B2 (en) | System and method for efficient user-level based deletions of backup data | |
| US20240070031A1 (en) | Method and system for data recovery by a hardware-independent approach involving prioritization and data recovery selectivity | |
| CN116302696A (en) | Archive log generation method of database system, storage medium and computer device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |