CN117851304A - Hard disk replacement method, device, equipment and medium - Google Patents
Hard disk replacement method, device, equipment and medium Download PDFInfo
- Publication number
- CN117851304A CN117851304A CN202410257475.2A CN202410257475A CN117851304A CN 117851304 A CN117851304 A CN 117851304A CN 202410257475 A CN202410257475 A CN 202410257475A CN 117851304 A CN117851304 A CN 117851304A
- Authority
- CN
- China
- Prior art keywords
- hard disk
- data
- replaced
- new
- pcie
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 230000004044 response Effects 0.000 claims abstract description 55
- 230000001360 synchronised effect Effects 0.000 claims abstract description 18
- 230000008569 process Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 10
- 230000005856 abnormality Effects 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 2
- 230000006872 improvement Effects 0.000 abstract description 7
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 230000005540 biological transmission Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000002829 reductive effect Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000002411 adverse Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Abstract
The invention relates to the technical field of computers, and provides a hard disk replacement method, a device, equipment and a medium, wherein the method comprises the following steps: in response to detecting a hard disk to be replaced and a new hard disk newly inserted into the host, dividing a PCIe bus through a PCIe switch, and setting PCIE links from the hard disk to be replaced and the new hard disk to a data processor; the data processor reads the data of the hard disk to be replaced based on the PCIE link, writes the data into a new hard disk, and temporarily stores the writing request received by the hard disk to be replaced into a corresponding log record; in response to all data in the hard disk to be replaced being synchronized to a new hard disk, the data processor synchronizes the write request based on the log record and removes the hard disk to be replaced from the host. The scheme of the invention reduces the consumption of the CPU and the memory, and is beneficial to the improvement of the overall performance and the stability of the cluster.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a medium for replacing a hard disk.
Background
Along with the increase of the demand of data storage, internet manufacturers and data support manufacturers gradually turn to efficient and intelligent distributed storage, and the distributed storage has the characteristics of high performance, high reliability, high expansibility, transparency and autonomy. With long-term use of distributed storage, the hard disk of NVME (Non-Volatile Memory Express, non-volatile memory host controller interface specification) wears with the pressure of IO (Input/Output), and the problem of insufficient life of the hard disk may occur. Therefore, the hard disk needs to be replaced in a scene of meeting business requirements.
In the prior art, a mode of directly pulling out a hard disk can lead to a large amount of data in a cluster to be recovered, so that the performance of the whole cluster is attenuated; or the plug-in disk is used for IO replacement of the hard disk, the hard disk to be replaced is isolated from the whole cluster, so that consumption of CPU (Central Processing Unit ) and memory information of the host is increased, adverse effects are generated on the CPU and IO, and improvement of cluster performance is not facilitated.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a method, apparatus, device and medium for replacing a hard disk.
According to a first aspect of the present invention, there is provided a hard disk replacement method comprising:
in response to detecting a hard disk to be replaced and a new hard disk newly inserted into the host, dividing a PCIe bus through a PCIe switch, and setting PCIE links from the hard disk to be replaced and the new hard disk to a data processor;
the data processor of the host reads the data of the hard disk to be replaced based on the PCIE link, writes the data into the new hard disk, and temporarily stores the writing request received by the hard disk to be replaced into a corresponding log record;
in response to all data in the hard disk to be replaced being synchronized to the new hard disk, the data processor synchronizes the write request based on the log record and removes the hard disk to be replaced from the host.
In some embodiments, the step of dividing the PCIe bus by the PCIe switch includes:
and distributing corresponding data bandwidths for the hard disk to be replaced, the new hard disk and the data processor from the PCIe bus through a PCIe switch.
In some embodiments, the step of the data processor of the host reading the data of the hard disk to be replaced and writing it to the new hard disk based on the PCIE link includes:
and the data processor of the host forwards the NVME instruction output by the hard disk to be replaced to the new hard disk through the PCIe bus distributed by the PCIe switch so as to forward the data of the hard disk to be replaced to the new hard disk and write the data based on the NVME instruction.
In some embodiments, the step of the data processor reading the data of the hard disk to be replaced and writing it to the new hard disk based on the PCIE link includes:
and the data processor reads the data of the hard disk to be replaced based on the PCIE link, and further merges and drops the data written into the new hard disk based on the object, the offset, the length and the numerical value corresponding to the data so as to write into the new hard disk.
In some embodiments, after the step of merging the data written into the new hard disk based on the object, the offset, the length and the numerical value corresponding to the data, the method further includes:
and refreshing the dirty data in the new hard disk into the latest written data based on the dropped data.
In some embodiments, after the step of reading the data of the hard disk to be replaced and writing the data to the new hard disk based on the PCIE link, the data processor further includes: and performing log playback on the new data written in the new hard disk.
In some embodiments, the method further comprises:
and in response to the fact that the hard disk to be replaced receives a reading request in the data synchronization process, judging whether the data of the hard disk to be replaced has copy data, and in response to the copy data, reading corresponding data from the copy data based on the reading request.
In some embodiments, the method further comprises:
and judging whether the hard disk to be replaced has unsynchronized data or not based on the log record in response to the fact that the copy data is not available, and reading the data from the hard disk to be replaced based on the reading request in response to the fact that the unsynchronized data is not available.
In some embodiments, the method further comprises:
and synchronizing the unsynchronized data into the hard disk to be replaced based on the log record in response to the unsynchronized data, and reading the data from the hard disk to be replaced based on the read request in response to synchronization completion.
In some embodiments, the step of synchronizing the unsynchronized data into the hard disk to be replaced based on the log record in response to there being unsynchronized data comprises:
and responding to the unsynchronized data, and synchronizing the data in the hard disk to be replaced into the latest data based on the data in the log record.
In some embodiments, the step of the data processor synchronizing the write request based on the log record and removing the hard disk to be replaced from the host comprises:
and the data processor analyzes the log record to obtain the write-in request, sequentially issues the write-in request to the new hard disk according to the analysis sequence, and removes the replacement hard disk from the host in response to completion of issuing all the write-in requests.
In some embodiments, the detecting that the hard disk is to be replaced on the host comprises:
and in response to detecting that the starting time of the hard disk on the host exceeds the preset time or the data reading/writing abnormality of the hard disk, confirming that the hard disk is to be replaced on the host.
According to a second aspect of the present invention, there is provided a hard disk replacement apparatus, the apparatus comprising:
the first module is used for responding to detection of a hard disk to be replaced on a host and a new hard disk newly inserted into the host, dividing a PCIe bus through a PCIe switch and setting PCIE links from the hard disk to be replaced and the new hard disk to a data processor;
the second module is used for reading the data of the hard disk to be replaced based on the PCIE link by the data processor of the host, writing the data into the new hard disk, and temporarily storing the writing request received by the hard disk to be replaced into a corresponding log record;
and a third module, configured to synchronize the write request based on the log record and remove the hard disk to be replaced from the host in response to all data in the hard disk to be replaced being synchronized to the new hard disk.
In some embodiments, the first module is further to:
and distributing corresponding data bandwidths for the hard disk to be replaced, the new hard disk and the data processor from the PCIe bus through a PCIe switch.
In some embodiments, the second module is further to:
and the data processor of the host forwards the NVME instruction output by the hard disk to be replaced to the new hard disk through the PCIe bus distributed by the PCIe switch so as to forward the data of the hard disk to be replaced to the new hard disk and write the data based on the NVME instruction.
In some embodiments, the second module is further to: and the data processor reads the data of the hard disk to be replaced based on the PCIE link, and further merges and drops the data written into the new hard disk based on the object, the offset, the length and the numerical value corresponding to the data so as to write into the new hard disk.
In some embodiments, the second module is further to: and refreshing the dirty data in the new hard disk into the latest written data based on the dropped data.
In some embodiments, the second module is further to: and performing log playback on the new data written in the new hard disk.
In some embodiments, the hard disk replacement device further comprises means for:
and in response to the fact that the hard disk to be replaced receives a reading request in the data synchronization process, judging whether the data of the hard disk to be replaced has copy data, and in response to the copy data, reading corresponding data from the copy data based on the reading request.
In some embodiments, the hard disk replacement device further comprises means for:
and judging whether the hard disk to be replaced has unsynchronized data or not based on the log record in response to the fact that the copy data is not available, and reading the data from the hard disk to be replaced based on the reading request in response to the fact that the unsynchronized data is not available.
In some embodiments, the hard disk replacement device further comprises means for:
and synchronizing the unsynchronized data into the hard disk to be replaced based on the log record in response to the unsynchronized data, and reading the data from the hard disk to be replaced based on the read request in response to synchronization completion.
In some embodiments, the hard disk replacement device further comprises means for: and responding to the unsynchronized data, and synchronizing the data in the hard disk to be replaced into the latest data based on the data in the log record.
In some embodiments, the third module is further to:
and the data processor analyzes the log record to obtain the write-in request, sequentially issues the write-in request to the new hard disk according to the analysis sequence, and removes the replacement hard disk from the host in response to completion of issuing all the write-in requests.
In some embodiments, the first module is further to:
and in response to detecting that the starting time of the hard disk on the host exceeds the preset time or the data reading/writing abnormality of the hard disk, confirming that the hard disk is to be replaced on the host.
According to a third aspect of the present invention, there is also provided an electronic device including:
at least one processor; and
and a memory storing a computer program executable on a processor, the processor executing the hard disk replacement method described above when executing the program.
According to a fourth aspect of the present invention, there is also provided a computer readable storage medium storing a computer program which when executed by a processor performs the aforementioned hard disk replacement method.
According to the hard disk replacement method, when the hard disk to be replaced on the host is detected, a new hard disk is inserted into the host and a PCIe bus is divided through a PCIe switch, the hard disk to be replaced and a PCIE link from the new hard disk to a data processor are arranged, the data processor directly reads the data of the hard disk to be replaced and writes the data into the new hard disk without a CPU, the writing request received by the hard disk to be replaced is temporarily stored in a log record, all the data of the hard disk to be replaced are synchronized, the writing request is synchronized based on the log record, and the hard disk to be replaced is removed from the host. The performance of the PCIe bus and the performance of the hard disk are fully exerted, the IO is unloaded in the hard disk replacement process through the data processor, the CPU is not needed to participate, the consumption of the CPU and the memory is reduced, and the improvement of the overall performance and the stability of the cluster are facilitated.
In addition, the invention also provides a hard disk replacement device, an electronic device and a computer readable storage medium, which can also achieve the technical effects, and are not repeated here.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention and that other embodiments may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for replacing a hard disk according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a network architecture based on a data processor and NVME hard disk according to another embodiment of the present invention;
FIG. 3 is a schematic diagram of a mechanism for data synchronization and LOG storage in a hard disk space according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of a hard disk replacement device according to another embodiment of the present invention;
FIG. 5 is an internal block diagram of an electronic device in accordance with another embodiment of the present invention;
fig. 6 is a block diagram of a computer readable storage medium according to another embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention will be further described in detail with reference to the accompanying drawings.
It should be noted that, in the embodiments of the present invention, all the expressions "first" and "second" are used to distinguish two entities with the same name but different entities or different parameters, and it is noted that the "first" and "second" are only used for convenience of expression, and should not be construed as limiting the embodiments of the present invention, and the following embodiments are not described one by one.
In one embodiment, referring to fig. 1, the present invention provides a method 100 for replacing a hard disk, specifically, the method for replacing a hard disk includes the following steps:
step 101, in response to detecting a hard disk to be replaced and a new hard disk newly inserted into the host, dividing a PCIe bus through a PCIe switch, and setting PCIE links from the hard disk to be replaced and the new hard disk to a data processor;
step 102, the data processor reads the data of the hard disk to be replaced based on the PCIE link and writes the data into the new hard disk, and temporarily stores the writing request received by the hard disk to be replaced as a corresponding log record;
step 103, in response to all data in the hard disk to be replaced being synchronized to the new hard disk, the data processor synchronizes the write request based on the log record and removes the hard disk to be replaced from the host.
According to the hard disk replacement method, when the hard disk to be replaced on the host is detected, a new hard disk is inserted into the host and a PCIe bus is divided through a PCIe switch, the hard disk to be replaced and a PCIE link from the new hard disk to a data processor are arranged, the data processor directly reads the data of the hard disk to be replaced and writes the data into the new hard disk without a CPU, the writing request received by the hard disk to be replaced is temporarily stored in a log record, all the data of the hard disk to be replaced are synchronized, the writing request is synchronized based on the log record, and the hard disk to be replaced is removed from the host. The performance of the PCIe bus and the performance of the hard disk are fully exerted, the IO is unloaded in the hard disk replacement process through the data processor, the CPU is not needed to participate, the consumption of the CPU and the memory is reduced, and the improvement of the overall performance and the stability of the cluster are facilitated.
For further understanding of the hard disk replacement method of the present invention, fig. 2 shows a network architecture schematic based on a data processor (Data Processing Unit, DPU) and an NVME hard disk, and as shown in fig. 2, in this embodiment, there are two NVME hard disks, hard disk 1 (hard disk to be replaced) and hard disk 2 (new hard disk), the PCIe bus is divided by a PCIe (Peripheral Component Interconnect express) switch, PCIe links from hard disk 1 and hard disk 2 to the DPU are set, that is, data transmission from hard disk 1 to hard disk 2 is performed by the DPU, the DPU can control data of the NVME hard disk by the NVME protocol, that is, the DPU can directly access the NVME hard disk by the PCIe link, and read data from hard disk 1 and directly write data to hard disk 2.
In some embodiments, with continued reference to fig. 2, intermediate forwarding is performed through the DPU, so that the hard disk 1 and the hard disk 2 are connected to a bus of the DPU, and data transmission is performed through a PCIe channel link, so that a data transmission mode requiring participation of a CPU is abandoned, performance of an NVME hard disk and performance of a PCIe bus are exerted, synchronization speed of data of the NVME hard disk is accelerated, time for replacing the hard disk is reduced, participation of the CPU is reduced, the whole IO stack is unloaded, and resource utilization rate of nodes is improved for a distributed storage cluster.
With continued reference to fig. 2, in the prior art, the PCIe bandwidth needs to be divided by Root Complex for data transmission between the hard disk 1 and the hard disk 2, where the Root Complex is located at the Root of the I/O hierarchy between the CPU/memory subsystem and the I/O, so that space isolation and conversion between the CPU address space and the PCIe system address may be implemented, and in the data transmission process, the CPU is required to be used through the central processor. The consumption of CPU and memory information of the host is increased, adverse effects are generated on the CPU and the IO, and the improvement of cluster performance is not facilitated.
According to several embodiments of the present invention, the step of dividing the PCIe bus by the PCIe switch includes:
and distributing corresponding data bandwidths for the hard disk to be replaced, the new hard disk and the data processor from the PCIe bus through a PCIe switch.
The PCIe buses of the host are divided again through the PCIe switch, enough data bandwidth is allocated for the hard disk to be replaced, the new hard disk and the data processor, the through performance of the PCIe buses and the performance of the hard disk can be fully exerted, and the cluster performance is improved.
And the data transmission mode between the hard disk to be replaced and the new hard disk is not used for the central processing unit of the host. The method realizes that data exchange between the hard disk to be replaced and the new hard disk is not carried out by the CPU, reduces occupation of the CPU and the memory, and is beneficial to reducing the performance loss of the CPU.
According to several embodiments of the present invention, the step of the data processor of the host reading the data of the hard disk to be replaced and writing it to the new hard disk based on the PCIE link includes:
and the data processor of the host forwards the NVME instruction output by the hard disk to be replaced to the new hard disk through the PCIe bus distributed by the PCIe switch so as to forward the data of the hard disk to be replaced to the new hard disk and write the data based on the NVME instruction.
The data processor forwards the instruction set, further forwards the data of the hard disk to be replaced and the data of the new hard disk, synchronizes the data of the hard disk, releases the resources on the host, and improves the efficiency of hard disk replacement more quickly.
In some embodiments, please continue with fig. 2, the dpu forwards the NVME instruction output by the hard disk 1 to the hard disk 2 through the PCIe bus, and forwards the data of the hard disk 1 to the hard disk 2 and writes based on the NVME instruction. Specifically, through DPU to NVME protocol simulation, DPU sends the NVME instruction of reading to waiting to renew the hard disk, waits to renew the hard disk and receives NVME instruction and pass through PCIe bus and transmit data to DPU, DPU merges data into NVME instruction set and sends to new hard disk, and new hard disk carries out IO after receiving NVME instruction. The whole process only updates the data message header, namely, no data copy exists, data reading and writing are carried out on the PCIe bus through the NVMe instruction set, CPU participation is avoided in the whole process, the DPU does instruction set conversion, the data of the whole hard disk are synchronized, and the data are dropped into the new hard disk through the PCIe bus.
According to several embodiments of the present invention, the step of the data processor reading the data of the hard disk to be replaced and writing it to the new hard disk based on the PCIE link includes:
and the data processor reads the data of the hard disk to be replaced based on the PCIE link, and further merges and drops the data written into the new hard disk based on the object, the offset, the length and the numerical value corresponding to the data so as to write into the new hard disk.
According to several embodiments of the present invention, after the step of merging the data written into the new hard disk based on the object, the offset, the length and the numerical value corresponding to the data, the method further includes:
and refreshing the dirty data in the new hard disk into the latest written data based on the dropped data.
According to several embodiments of the present invention, after the step of reading the data of the hard disk to be replaced and writing the data to the new hard disk based on the PCIE link, the data processor further includes: and performing log playback on the new data written in the new hard disk.
According to several embodiments of the present invention, the step of temporarily storing the write request received by the hard disk to be replaced as a corresponding log record includes:
and responding to the write-in request received by the hard disk to be replaced, recording the write-in request in a log mode by the data processor of the host computer, and temporarily storing the write-in request as corresponding log records according to the received time sequence.
The write-in request of the disk to be replaced is temporarily stored in a log recording mode, the write-in request can be temporarily stored in sequence according to the time sequence, the consistency and the order of IO requests in the process of replacing the hard disk are ensured, and the occurrence of abnormality is avoided.
In some embodiments, in the process of DPU transfer, the write data cannot be normally issued to the hard disk to be replaced, so in the process of data synchronization, the write request of the hard disk to be replaced needs to be integrated and written into the free space in LOG (LOG) mode for temporary storage, waiting for the DPU to complete the synchronization of all the data of the hard disk to be replaced, and issuing the write request to the new hard disk through LOG replay mechanism. The LOG mode writing replacement is a design mode of a file system or a storage system, data is organized into a series of LOG entries, data fragments and metadata operations are reduced, and the utilization rate of storage space and the performance of writing operation are improved.
According to several embodiments of the present invention, the step of logging the write request by the data processor of the host includes:
the data processor of the host stores the information of the object, the data offset, the data length and the corresponding data corresponding to the writing request as TLV format information and records the TLV format information in a log mode.
The write-in request is recorded according to a unified format, which is beneficial to ensuring the consistency of data.
In some embodiments, please refer to fig. 3, fig. 3 illustrates a schematic diagram of a data synchronization and LOG storage mechanism of a hard disk space provided for another embodiment of the present invention, as shown in fig. 3, a writing service writes a to-be-replaced reel in a LOG manner, solidifies an OBJ (object), an offset (data offset), a length (data length) and data (data) corresponding to the whole IO into TLV format information, directly drops a disk, performs LOG playback after waiting for the end of data synchronization of hard disk replacement, and drops the data to a position actually needed to be written. The TLV format information is data encoding related information of one input/output operation (IO) into a TLV (Type-Length-Value) format, and standardized and structured data representation can be realized by encoding related information of the IO operation into the TLV format, so that data processing is more flexible and efficient.
According to several embodiments of the present invention, the step of temporarily storing the received time sequence as a corresponding log record includes:
and writing the writing request into the pre-allocated free space according to the received time sequence so as to temporarily store the writing request as a corresponding log record.
And the write request is written into the free space in a LOG mode, so that the utilization rate of the storage space and the performance and efficiency of the storage system can be improved.
According to several embodiments of the invention, the method further comprises:
and in response to the fact that the hard disk to be replaced receives a reading request in the data synchronization process, judging whether the data of the hard disk to be replaced has copy data, and in response to the copy data, reading corresponding data from the copy data based on the reading request.
In the data synchronization process, the method ensures that no new data fall in as much as possible, avoids the need of re-data copying request during data playback, and performs new IO playback until the data copying is completed, thereby ensuring the continuity and speed of data synchronization and reducing the problem of data inconsistency in the data synchronization process.
According to several embodiments of the invention, the method further comprises:
and judging whether the hard disk to be replaced has unsynchronized data or not based on the log record in response to the fact that the copy data is not available, and reading the data from the hard disk to be replaced based on the reading request in response to the fact that the unsynchronized data is not available.
According to several embodiments of the invention, the method further comprises:
and synchronizing the unsynchronized data into the hard disk to be replaced based on the log record in response to the unsynchronized data, and reading the data from the hard disk to be replaced based on the read request in response to synchronization completion.
According to several embodiments of the present invention, the step of synchronizing the unsynchronized data into the hard disk to be replaced based on the log record in response to there being unsynchronized data, includes:
and responding to the unsynchronized data, and synchronizing the data in the hard disk to be replaced into the latest data based on the data in the log record.
In some embodiments, in the distributed storage cluster, the read request should be read from other copies as far as possible, if the read request cannot be read from other copies, the node can provide a data read operation, and the write request LOGs in a LOG mode, so that all data of the current node are the latest data, other node faults caused during hard disk replacement are avoided, and damage to data integrity is avoided. The distributed storage still provides data reading service and writing service to the outside, so that the continuity of the distributed storage service and normal operation of user service in the process of hardware replacement are ensured.
In some embodiments, the read request occurs in the hard disk to be replaced, whether the data is changed is judged through the written LOG, if not, the data is directly read from the hard disk to be replaced, and if so, the data in the LOG and the data on the real physical disk are synchronized to be the latest data, and the latest data is provided for the customer service for reading operation.
According to several embodiments of the present invention, the step of the data processor synchronizing the write request based on the log record and removing the hard disk to be replaced from the host includes:
and the data processor analyzes the log record to obtain the write-in request, sequentially issues the write-in request to the new hard disk according to the analysis sequence, and removes the replacement hard disk from the host in response to completion of issuing all the write-in requests.
In some embodiments, if the DPU completes data synchronization, LOG playback is performed, and the written data is merged and dropped based on an Object Identifier (OID), an offset (data offset), a length (data length) and data (data) corresponding to the written data, so that dirty data is refreshed to be the latest data, thereby ensuring the consistency of the data. In the LOG playback process, the latest data are synchronized step by step, so that no new data fall in as much as possible, the data copying is avoided in the data playback process, when all LOG playback is completed, all IO directly fall into a new hard disk, the hard disk to be replaced is removed, and the consistency of the data is guaranteed.
In some embodiments, after LOG data is played back to a certain time point, the remaining number is smaller than a preset number threshold, for example, 20, then the writing request converted into LOG is converted into a normal data writing request, and the normal data writing request directly falls into a new hard disk, and when the new hard disk is dropped, according to LOG corresponding data, the problem of data inconsistency is avoided. Meanwhile, LOG is converted into a data disk, so that all data is ensured to truly fall into a new hard disk, and the problem of inconsistent data in the process is avoided.
According to several embodiments of the present invention, the detecting that the hard disk is to be replaced on the host includes:
and in response to detecting that the starting time of the hard disk on the host exceeds the preset time or the data reading/writing abnormality of the hard disk, confirming that the hard disk is to be replaced on the host.
The method comprises the steps that a hard disk installed on a host computer has preset starting time, and if the starting time of a certain hard disk exceeds the preset starting time, the hard disk possibly has problems; meanwhile, if the read/write of a certain hard disk is abnormal, the damage of the hard disk is indicated.
According to several embodiments of the invention, the method further comprises:
and connecting the new hard disk and the hard disk to be replaced with a data processor of the host computer through the PCIe bus.
According to a second aspect of the present invention, as shown in fig. 4, fig. 4 shows a hard disk replacement apparatus 200 according to another embodiment of the present invention, the apparatus comprising:
a first module 201, configured to divide a PCIe bus through a PCIe switch in response to detecting that a hard disk to be replaced and a new hard disk newly inserted into the host are located on the host, and set PCIe links from the hard disk to be replaced and the new hard disk to a data processor;
a second module 202, configured to read the data of the hard disk to be replaced based on the PCIE link and write the data into the new hard disk, and temporarily store a write request received by the hard disk to be replaced as a corresponding log record;
and a third module 203, configured to synchronize the write request based on the log record and remove the hard disk to be replaced from the host in response to all data in the hard disk to be replaced being synchronized to the new hard disk.
In some embodiments, the first module 201 is further configured to:
and distributing corresponding data bandwidths for the hard disk to be replaced, the new hard disk and the data processor from the PCIe bus through a PCIe switch.
In some embodiments, the second module 202 is further configured to:
and the data processor of the host forwards the NVME instruction output by the hard disk to be replaced to the new hard disk through the PCIe bus distributed by the PCIe switch so as to forward the data of the hard disk to be replaced to the new hard disk and write the data based on the NVME instruction.
In some embodiments, the second module 202 is further configured to: and the data processor reads the data of the hard disk to be replaced based on the PCIE link, and further merges and drops the data written into the new hard disk based on the object, the offset, the length and the numerical value corresponding to the data so as to write into the new hard disk.
In some embodiments, the second module 202 is further configured to: and refreshing the dirty data in the new hard disk into the latest written data based on the dropped data.
In some embodiments, the second module 202 is further configured to: and performing log playback on the new data written in the new hard disk.
In some embodiments, the hard disk replacement device 200 further comprises modules for:
and in response to the fact that the hard disk to be replaced receives a reading request in the data synchronization process, judging whether the data of the hard disk to be replaced has copy data, and in response to the copy data, reading corresponding data from the copy data based on the reading request.
In some embodiments, the hard disk replacement device 200 further comprises modules for:
and judging whether the hard disk to be replaced has unsynchronized data or not based on the log record in response to the fact that the copy data is not available, and reading the data from the hard disk to be replaced based on the reading request in response to the fact that the unsynchronized data is not available.
In some embodiments, the hard disk replacement device 200 further comprises modules for:
and synchronizing the unsynchronized data into the hard disk to be replaced based on the log record in response to the unsynchronized data, and reading the data from the hard disk to be replaced based on the read request in response to synchronization completion.
In some embodiments, the hard disk replacement device 200 further comprises modules for: and responding to the unsynchronized data, and synchronizing the data in the hard disk to be replaced into the latest data based on the data in the log record.
In some embodiments, the third module 203 is further configured to:
and the data processor analyzes the log record to obtain the write-in request, sequentially issues the write-in request to the new hard disk according to the analysis sequence, and removes the replacement hard disk from the host in response to completion of issuing all the write-in requests.
In some embodiments, the first module 201 is further configured to:
and in response to detecting that the starting time of the hard disk on the host exceeds the preset time or the data reading/writing abnormality of the hard disk, confirming that the hard disk is to be replaced on the host.
According to the hard disk replacement device, when the hard disk to be replaced on the host is detected, a new hard disk is inserted into the host and a PCIe bus is divided through a PCIe switch, the hard disk to be replaced and a PCIE link from the new hard disk to a data processor are arranged, the data processor of the host directly reads the data of the hard disk to be replaced and writes the data into the new hard disk, the data is not passed through a CPU, a writing request received by the hard disk to be replaced is temporarily stored in a log record, all the data of the hard disk to be replaced are synchronized, the writing request is synchronized based on the log record, and the hard disk to be replaced is removed from the host. The performance of the PCIe bus and the performance of the hard disk are fully exerted, the IO is unloaded in the hard disk replacement process through the data processor, the CPU is not needed to participate, the consumption of the CPU and the memory is reduced, and the improvement of the overall performance and the stability of the cluster are facilitated.
It should be noted that, the specific limitation of the hard disk replacement device may be referred to the limitation of the hard disk replacement method, and the description thereof is omitted herein. The above-described respective modules in the hard disk replacement apparatus may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or independent of a processor in the electronic device, or may be stored in software in a memory in the electronic device, so that the processor may call and execute operations corresponding to the above modules.
According to another aspect of the present invention, there is provided an electronic device, which may be a server, and an internal structure thereof is shown in fig. 5. The electronic device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the electronic device is for storing data. The network interface of the electronic device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements the hard disk replacement method described above.
According to still another aspect of the present invention, a computer readable storage medium is provided, as shown in fig. 6, on which a computer program is stored, which when executed by a processor, implements the hard disk replacement method described above.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (15)
1. A hard disk replacement method, characterized in that the hard disk replacement method comprises:
in response to detecting a hard disk to be replaced and a new hard disk newly inserted into the host, dividing a PCIe bus through a PCIe switch, and setting PCIE links from the hard disk to be replaced and the new hard disk to a data processor;
the data processor reads the data of the hard disk to be replaced based on the PCIE link, writes the data into the new hard disk, and temporarily stores a writing request received by the hard disk to be replaced into a corresponding log record;
in response to all data in the hard disk to be replaced being synchronized to the new hard disk, the data processor synchronizes the write request based on the log record and removes the hard disk to be replaced from the host.
2. The hard disk replacement method according to claim 1, wherein the step of dividing the PCIe bus by the PCIe switch comprises:
and distributing corresponding data bandwidths for the hard disk to be replaced, the new hard disk and the data processor from the PCIe bus through a PCIe switch.
3. The hard disk replacement method according to claim 1, wherein the step of the data processor reading the data of the hard disk to be replaced based on the PCIE link and writing it to the new hard disk includes:
and the data processor forwards the NVME instruction output by the hard disk to be replaced to the new hard disk through a PCIe bus distributed by the PCIe switch, so that the data of the hard disk to be replaced is forwarded to the new hard disk and written based on the NVME instruction.
4. The hard disk replacement method according to claim 1, wherein the data processor reads the data of the hard disk to be replaced based on the PCIE link and writes it to the new hard disk, comprising:
and the data processor reads the data of the hard disk to be replaced based on the PCIE link, and further merges and drops the data written into the new hard disk based on the object, the offset, the length and the numerical value corresponding to the data so as to write into the new hard disk.
5. The method for replacing a hard disk according to claim 4, wherein after the step of merging and dropping the data written into the new hard disk based on the object, the offset, the length and the numerical value corresponding to the data, the method further comprises:
and refreshing the dirty data in the new hard disk into the latest written data based on the dropped data.
6. The hard disk replacement method according to claim 1, wherein after the step of the data processor reading the data of the hard disk to be replaced based on the PCIE link and writing it to the new hard disk, further comprising: and performing log playback on the new data written in the new hard disk.
7. The hard disk replacement method according to claim 1, wherein the method further comprises:
and in response to the fact that the hard disk to be replaced receives a reading request in the data synchronization process, judging whether the data of the hard disk to be replaced has copy data, and in response to the copy data, reading corresponding data from the copy data based on the reading request.
8. The hard disk replacement method according to claim 7, further comprising:
and judging whether the hard disk to be replaced has unsynchronized data or not based on the log record in response to the fact that the copy data is not available, and reading the data from the hard disk to be replaced based on the reading request in response to the fact that the unsynchronized data is not available.
9. The hard disk replacement method according to claim 8, further comprising:
and synchronizing the unsynchronized data into the hard disk to be replaced based on the log record in response to the unsynchronized data, and reading the data from the hard disk to be replaced based on the read request in response to synchronization completion.
10. The hard disk replacement method according to claim 9, wherein the step of synchronizing the unsynchronized data into the hard disk to be replaced based on the log record in response to the unsynchronized data being present, comprises:
and responding to the unsynchronized data, and synchronizing the data in the hard disk to be replaced into the latest data based on the data in the log record.
11. The hard disk replacement method according to claim 1, wherein the step of the data processor synchronizing the write request based on the log record and removing the hard disk to be replaced from the host comprises:
and the data processor analyzes the log record to obtain the write-in request, sequentially issues the write-in request to the new hard disk according to the analysis sequence, and removes the replacement hard disk from the host in response to completion of issuing all the write-in requests.
12. The method of claim 1, wherein detecting that the hard disk is to be replaced on the host comprises:
and in response to detecting that the starting time of the hard disk on the host exceeds the preset time or the data reading/writing abnormality of the hard disk, confirming that the hard disk is to be replaced on the host.
13. A hard disk replacement apparatus, the apparatus comprising:
the first module is used for responding to detection of a hard disk to be replaced on a host and a new hard disk newly inserted into the host, dividing a PCIe bus through a PCIe switch and setting PCIE links from the hard disk to be replaced and the new hard disk to a data processor;
the second module is used for reading the data of the hard disk to be replaced based on the PCIE link, writing the data into the new hard disk, and temporarily storing the writing request received by the hard disk to be replaced into a corresponding log record;
and a third module, configured to synchronize the write request based on the log record and remove the hard disk to be replaced from the host in response to all data in the hard disk to be replaced being synchronized to the new hard disk.
14. An electronic device, comprising:
at least one processor; and
a memory storing a computer program executable in the processor, the processor executing the method of hard disk replacement according to any one of claims 1 to 12 when the program is executed.
15. A computer readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, performs the hard disk replacement method of any one of claims 1-12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410257475.2A CN117851304A (en) | 2024-03-07 | 2024-03-07 | Hard disk replacement method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410257475.2A CN117851304A (en) | 2024-03-07 | 2024-03-07 | Hard disk replacement method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117851304A true CN117851304A (en) | 2024-04-09 |
Family
ID=90542204
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410257475.2A Pending CN117851304A (en) | 2024-03-07 | 2024-03-07 | Hard disk replacement method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117851304A (en) |
-
2024
- 2024-03-07 CN CN202410257475.2A patent/CN117851304A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6859888B2 (en) | Data storage array apparatus storing error information without delay in data access, and method, program recording medium, and program for the same | |
| US11307776B2 (en) | Method for accessing distributed storage system, related apparatus, and related system | |
| JP6264666B2 (en) | Data storage method, data storage device, and storage device | |
| US20050073887A1 (en) | Storage system | |
| CN111880751B (en) | Hard disk migration method, distributed storage cluster system and storage medium | |
| CN104935654A (en) | Caching method, write point client and read client in server cluster system | |
| CN111708738B (en) | Method and system for realizing interaction of hadoop file system hdfs and object storage s3 data | |
| WO2023015866A1 (en) | Data writing method, apparatus and system, and electronic device and storage medium | |
| CN111581020A (en) | Method and device for data recovery in distributed block storage system | |
| CN110442646B (en) | Write performance optimization system and method for master end of ceph data synchronization module | |
| US20220334733A1 (en) | Data restoration method and related device | |
| CN113885809B (en) | Data management system and method | |
| CN110147203B (en) | File management method and device, electronic equipment and storage medium | |
| CN115167782A (en) | Temporary storage copy management method, system, device and storage medium | |
| CN111966531B (en) | Data snapshot method and device, computer equipment and storage medium | |
| JP2019204527A (en) | Processing method and processing device of data position of storage equipment, computer equipment, and computer-readable storage medium | |
| CN116450418B (en) | Breakpoint difference synchronization method and system based on volume backup and volume real-time backup method | |
| CN117851304A (en) | Hard disk replacement method, device, equipment and medium | |
| WO2022033269A1 (en) | Data processing method, device and system | |
| JP6376626B2 (en) | Data storage method, data storage device, and storage device | |
| CN106933707B (en) | Data recovery method and system of data storage device based on raid technology | |
| CN114003612A (en) | Processing method and processing system for abnormal conditions of database | |
| JP7073737B2 (en) | Communication log recording device, communication log recording method, and communication log recording program | |
| CN111522883A (en) | Backup method, device and equipment of object data and storage medium | |
| CN110134548B (en) | Quick restarting method of distributed memory big data processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |