CN101859265A - Disk array reconstruction method and device - Google Patents
Disk array reconstruction method and device Download PDFInfo
- Publication number
- CN101859265A CN101859265A CN 201010189640 CN201010189640A CN101859265A CN 101859265 A CN101859265 A CN 101859265A CN 201010189640 CN201010189640 CN 201010189640 CN 201010189640 A CN201010189640 A CN 201010189640A CN 101859265 A CN101859265 A CN 101859265A
- Authority
- CN
- China
- Prior art keywords
- storage
- rebuild
- array
- data
- memory device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a disk array reconstruction method and a device. By utilizing the technical scheme, during the reconstruction process of the array, the pressure of data stream can be shared by other memory devices with surplus capacity so as to accelerate the reconstruction process, so the monitored data is free from being lost caused by the decrease of the array reconstruction performance, the reading and writing reliability of the data of the system can be improved, and the processing efficiency of the array reconstruction in the system can be improved.
Description
Technical field
The present invention relates to communication technical field, particularly a kind of disk array reconstruction method and equipment.
Background technology
In the video monitoring service, vision signal enters EC (Video Codec) by camera, by coding module with 1 second data compilation become an I frame group (forming) of specifying the code check size by an I frame and a plurality of P frame.Every I frame that compiles out or P frame all will be placed in the buffer area of EC in this process.EC gives out a contract for a project module at every turn from a buffer area taking-up I frame group (1 second data), sends message by TCP (TransmissionControl Protocol, transmission control protocol) agreement to memory device.Memory device returns a feedback information notice EC continuation transmission after storing message data into data disks, and the module of giving out a contract for a project is obtained an I frame group again from buffer area, the forwarding step before continuing.
The EC buffer size is limited, generally can only several seconds to tens seconds data of buffer memory.If storage device processes I/O is (Input/Output, I/O) performance is lower, the time of finishing data storage is greater than 1 second, makes that the EC module of giving out a contract for a project is also corresponding elongated from the time that buffer area obtains I frame group, and the data formation speed causes the water level of EC buffer area to go up greater than take-off speed.When water level surpassed cache size, follow-up I frame of coming from collector (or P frame) will cover original data buffer memory, causes packet loss.
In the application of reality, memory property descends and to mainly contain two reasons and cause during array rebuild:
First reason, as shown in Figure 1, when disk 3 damages, when rebuilding behind the RAID5 array degradation, need from disk 1,2,4,5, strip data and check bit to be read respectively, make each disk be in frequent read data status, so disk will increase greatly to response time of other operations, and also write the dish performance and sharply descend with regard to the array that makes.
Second reason after strip data and verification and reading from disk, will be transferred to verification and chip and handle, calculate the corresponding data lost on the disk 3 or verification and, the data that calculate are write direct in the disk of rebuilding.This process RAID (Redundant Array of Independent Disk, raid-array) the consistent busy state that is in of the verification on the memory device with chip, verification after new data write is corresponding slowly, also is the reason that causes whole storage device performance to reduce.
Existing memory device generally can adopt the array that has redundant ability to store for the safety of data, but when redundant array generation data disks is lost and is rebuild, the performance of memory device will have more significantly and descend, packet drop need be considered to rebuild during deployment, matching E C ability can't be arrived when causing under the normal condition configure storage devices.
Present existing scheme, in order to guarantee that packet loss does not take place the array rebuild performance back EC that descends to occur at memory device, the EC storage plan quantity of distributing in the storage when plan implementation is that the performance when rebuilding simultaneously by array is implemented.
Such as in the scheme of an enforcement, 4 memory storage devices of a DM control, 4 RAID5 arrays (each array is made up of 15+1 disk) are arranged under each memory device, when the array under the memory device all is in the normal state, each memory device can be handled 220 road EC 4Mb storage flow at most packet loss not take place, and four memory devices then are 880 the tunnel.When 1 array rebuild was arranged under certain memory device, packet loss can not take place when being no more than 160 the tunnel in the monitor data of this memory device, stores stream maximum.
And in the memory device array to take place to rebuild probability be very large, so when monitoring scheme is disposed for fear of the generation of EC end packet loss phenomenon, the 4Mb storage flow that every memory device distributes down is no more than 160 the tunnel at most! This is the 160x4=640 road by the maximum storable 4Mb storage flow of the embodiment that a DM manages, 4 memory devices are formed so in the above.
In realizing process of the present invention, the inventor finds that there is following problem at least in prior art:
Processing power when existing technology is used array rebuild is disposed embodiment, wasted the handling property under a part of memory device normal condition, in the time of quite most of storage array normal condition, the storage capacity of storage system can't fully be used.
Summary of the invention
The invention provides a kind of disk array reconstruction method and equipment, solved that redundant array generation data disks is lost and when rebuilding, the problem that the performance of memory device descends and causes monitor data to be lost.
For achieving the above object, one aspect of the present invention provides a kind of disk array reconstruction method, be applied to comprise a data management server, in the system of a plurality of memory devices and a plurality of encoding devices, described encoding device is storing data information in corresponding memory device, described data management server is used for described a plurality of memory devices are managed, comprise one or more storage arrays in the described memory device, described data management server is that described a plurality of memory device is provided with storage flow batching counter and storage flow threshold respectively, and described method specifically may further comprise the steps:
When described data management server receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, described data management server judges that the current numerical value of storage flow batching counter that whether exists does not reach other memory devices of storing flow threshold;
If exist, described data management server is not created interim storage resources in the numerical value of one or more storage flow batching counters reaches the memory device of storage flow threshold, and will to need to the described storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array be the address of described interim storage resources;
When described data management apparatus will need when described storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array all is set at the address of described interim storage resources, described data management apparatus sends the reconstruction acknowledge message of waiting to rebuild array to described memory device, so that storage array described to be rebuild begins to rebuild;
Wait to rebuild storage array and rebuild when finishing when described, described data management server will need the address of the storage array after the storage destination address of the encoding device of storage data in described interim storage resources reverts to reconstruction.
Preferably, when described data management server receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, described data management server judges that the current numerical value of storage flow batching counter that whether exists does not reach after other memory devices of storage flow threshold, also comprises:
If there is no, described data management server is created interim storage resources in sending the normal storage array that memory device comprised of described reconstruction notification message, and will to need to the described storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array be the address of described interim storage resources;
The address that will need all to be set at described interim storage resources when described data management apparatus to described storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array, or the numerical value of the storage flow batching counter in the memory device of described transmission reconstruction notification message reaches when storing flow threshold, described data management apparatus sends the reconstruction acknowledge message of waiting to rebuild array to described memory device, so that storage array described to be rebuild begins to rebuild;
When the described reconstruction of waiting to rebuild storage array is finished, described data management server will need the address of the storage array after the storage destination address of the encoding device of storage data in described interim storage resources reverts to reconstruction.
Preferably, it is the address of described interim storage resources that described data management server will need to the described storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array, also comprises:
Described data management server is set up the storage relation mapping table, and described storage relationship map table record is described waits to rebuild storage resources in the storage array and the corresponding relation between the described interim storage resources.
Preferably, described data management server will need the address of the storage array after the storage destination address of the encoding device of storage data in described interim storage resources reverts to reconstruction, specifically comprise:
Described data management server is according to described storage relation mapping table, according to the corresponding relation of described storage relationship map table record, the address of the storage array after the storage destination address that need store the encoding device of data in each described interim storage resources reverts to reconstruction.
Preferably, described method also comprises:
Described data management server writes down the zero-time and the termination time of each described encoding device storing data information in each described interim storage resources in described storage relation mapping table.
On the other hand, the present invention also provides a kind of data management server, be applied to comprise in the system of a data management server, a plurality of memory device and a plurality of encoding devices, described encoding device is storing data information in corresponding memory device, comprise one or more storage arrays in the described memory device, comprising:
Module is set, is used to described a plurality of memory device that storage flow batching counter and storage flow threshold are set respectively;
Communication module is used to receive the reconstruction notification message of waiting to rebuild storage array that memory device sends;
Enquiry module, be connected with described communication module with the described module that is provided with, be used for when described communication module receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, inquire about and currently whether exists the described numerical value that the set storage flow batching counter of module is set not reach the memory device of storing flow threshold;
Scheduler module, be connected with described enquiry module, be used at the Query Result of described enquiry module when existing, numerical value at one or more storage flow batching counters does not reach the interim storage resources of establishment in the memory device of storing flow threshold, and will to need to the described storage setting recipient of waiting to rebuild in the storage array encoding device of storage data be the address of described interim storage resources, and wait to rebuild storage array and rebuild when finishing described, the address of the storage array after the storage destination address that need store the encoding device of data in described interim storage resources reverts to reconstruction;
Judge module, be connected with described communication module with described scheduler module, be used for judging whether described scheduler module will need all to be set to the described storage destination address of waiting to rebuild whole encoding devices of storage array storage data the address of described interim storage resources, in judged result when being, indicate described communication module to send the reconstruction acknowledge message of waiting to rebuild array to described memory device, so that storage array described to be rebuild begins to rebuild, and judge and describedly wait to rebuild storage array and whether finish reconstruction.
Preferably, described scheduler module, also be used at the Query Result of described enquiry module when not existing, in sending the normal storage array that memory device comprised of described reconstruction notification message, create interim storage resources, and will to need to the described storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array be the address of described interim storage resources;
Judge described scheduler module when described judge module and will need all to be set at the address of described interim storage resources to described storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array, or described enquiry module inquires described transmission when rebuilding the numerical value of the storage flow batching counter in the memory device of notification message and reaching the storage flow threshold, described notification module indicates described communication module to send the reconstruction acknowledge message of waiting to rebuild array to described memory device, so that storage array described to be rebuild begins to rebuild;
When described judge module judges that storage array described to be rebuild is finished reconstruction, the address of the storage array after described scheduler module also is used for reverting to reconstruction to the storage destination address that described interim storage resources is stored the encoding device of data.
Preferably, described data management server also comprises:
Logging modle, be connected with described scheduler module, be used for setting up storage relation mapping table, storage resources in the storage array that the needs that the described scheduler module of described storage relationship map table record is generated are rebuild and the corresponding relation between the described interim storage resources according to the scheduling situation of described scheduler module.
Preferably,
The storage relation mapping table that described scheduler module is set up according to described logging modle, according to the corresponding relation of described storage relationship map table record, the address of the storage array after the storage destination address that need store the encoding device of data in each described interim storage resources reverts to reconstruction.
Preferably,
Described logging modle also is used for zero-time and termination time in described storage relation mapping table each described encoding device of record storing data information in each described interim storage resources.
Compared with prior art, the present invention has the following advantages:
By using technical scheme of the present invention, in the array rebuild process, data stream pressure can be the memory device of free complementary energy power to share by other, accelerate process of reconstruction, monitor data can not descended because of the array rebuild performance cause loss of data, thereby improved the reliability of the reading and writing data of system, and improved the treatment effeciency of array rebuild in the system.
Description of drawings
Fig. 1 is a RAID5 array degradation synoptic diagram in the prior art;
Fig. 2 is the schematic flow sheet of a kind of disk array reconstruction method proposed by the invention;
Fig. 3 is the schematic flow sheet of disk array reconstruction method under a kind of concrete application scenarios proposed by the invention;
Fig. 4 is a kind of synoptic diagram of monitoring the file storage ultimate principle proposed by the invention;
Fig. 5 is the structural representation of a kind of data management server proposed by the invention.
Embodiment
At deficiency of the prior art, need in the process that storage array is rebuild, store for the data message in the storage array that should be stored in reconstruction by corresponding storage resources temporarily, in order to guarantee the continuity of storage service, storage array rebuild finish after, also need corresponding operational data storage is returned in the storage array after the reconstruction.
Based on such technical thought, the present invention proposes a kind of disk array reconstruction method, be applied to comprise in the system of a data management server, a plurality of memory device and a plurality of encoding devices, encoding device is storing data information in corresponding memory device, data management server is used for a plurality of memory devices are managed, and comprises one or more storage arrays in the memory device.
Based on above-mentioned purpose, as shown in Figure 2, the schematic flow sheet for a kind of disk array reconstruction method proposed by the invention specifically may further comprise the steps:
Step S201, data management server are that a plurality of memory devices are provided with storage flow batching counter and storage flow threshold respectively.
Step S202, when data management server receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, data management server judges that the current numerical value of storage flow batching counter that whether exists does not reach other memory devices of storing flow threshold.
If there is execution in step S203;
If there is no, execution in step S206.
Step S203, data management server do not create interim storage resources in the numerical value of one or more storage flow batching counters reaches the memory device of storage flow threshold, and will to need to the storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array be the address of interim storage resources.
When data management apparatus will need when the storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array all is set at the address of interim storage resources execution in step S204.
Step S204, data management apparatus send the reconstruction acknowledge message of waiting to rebuild array to memory device, so that storage array to be rebuild begins to rebuild.
Step S205, rebuild when finishing when waiting to rebuild storage array, data management server will need the address of the storage array after the storage destination address of the encoding device of storage data in interim storage resources reverts to reconstruction.
Step S206, data management server create interim storage resources in sending the normal storage array that memory device comprised of rebuilding notification message, and will to need to the storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array be the address of interim storage resources.
The address that will need all to be set at interim storage resources when data management apparatus to the storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array, or the numerical value that sends the storage flow batching counter in the memory device of rebuilding notification message reaches when storing flow threshold execution in step S204.
In step S203 and step S206, the data information memory that data management server will need the encoding device of storage data in the storage array that the pairing needs of array rebuild notification message are rebuild in interim storage resources time, also comprise:
Data management server is set up the storage relation mapping table, and storage relationship map table record waits to rebuild the storage resources and the corresponding relation between the storage resources temporarily in the storage array.
In concrete application scenarios, data management server is the zero-time and the termination time of each encoding device of record storing data information in each interim storage resources in the storage relation mapping table further.
In such cases, the realization flow of step S205 specifically comprises:
Data management server is according to the storage relation mapping table, according to the corresponding relation of storage relationship map table record, and the address of the storage array after the storage destination address that need store the encoding device of data in each interim storage resources reverts to reconstruction.
Compared with prior art, the present invention has the following advantages:
By using technical scheme of the present invention, in the array rebuild process, data stream pressure can be the memory device of free complementary energy power to share by other, accelerate process of reconstruction, monitor data can not descended because of the array rebuild performance cause loss of data, thereby improved the reliability of the reading and writing data of system, and improved the treatment effeciency of array rebuild in the system.
In order further to set forth technological thought of the present invention, existing in conjunction with concrete application scenarios, technical scheme of the present invention is described.
Technical scheme proposed by the invention is primarily aimed at that redundant array generation data disks is lost and when rebuilding, the problem that the performance of memory device descends and causes monitor data to be lost, in carrying out the array rebuild process, fully use the storage capacity of memory device normal condition as far as possible, guarantee that the monitor data in the array rebuild process is not lost.
Based on above-mentioned technological thought, the present invention has created a kind of mechanism, if in the memory device of DM management a plurality of arrays are arranged, so, when wherein certain array was prepared to begin to rebuild, corresponding memory device at first needed to report notice DM, this array to be rebuild is in degrading state simultaneously, waits for that DM replys.
Wherein, degradation (Degrade) state, claim critical conditions (Critical) again, refer to have in the redundant array state after one (such as the RAID5 array) or a plurality of (such as the RAID6 array) disk failure, but still can provide normal read-write service to the external world, but this moment if a disk failure being arranged again, then whole array then is in disabled state.
After DM receives the notice that memory device reports, in other idle relatively arrays, create interim Tmp_SAN resource, and notify corresponding EC will plan to write the monitor data stream of rebuilding array originally to write among these Tmp_SAN, notify memory device to begin array rebuild at last.In process of reconstruction, DM safeguards the storage time of the monitor data stream that corresponding D M data map table record is redirected.
After reconstruction was finished, corresponding memory device reported DM, and data stream redirects in the array (former SAN resource) of finishing reconstruction.And the data that write Tmp_SAN before in the process of reconstruction are not moved back, and can need select the resource reading of data of carry according to the time of writing down in the DM data mapping tables when carrying out video playback, carry out playback.
After surpassing retention time and reconstruction and finishing, deletion Tmp_SAN resource Free up Memory.
As shown in Figure 3, the schematic flow sheet for the disk array reconstruction method under a kind of concrete application scenarios of the present invention's proposition specifically may further comprise the steps:
Monitoring storage scheme in step S301, the current system of DM initialization.
In the monitoring storage scheme, each circuit-switched data stream can be created a SAN resource that calculates by the storage fate on the array at memory device when creating storage plan, can create a file system simultaneously.File system uses the SUPER DATA of a 16K to come identification document.
SUPER DATA back is the one-level index block, and size is 128K.One-level index block back is the data cell of a series of 256M size, is used to preserve data, the 256M data cell that act as the retrieval back of one-level index block.
Each secondary data piece is 256MB, comprises the data cell of secondary index (256KB) and 16368 16KB.The data block of each 256M is all with the secondary index BOB(beginning of block), secondary index act as I frame group in the current data block.
One-level index in the file and secondary index are all with time keeping.
Above-mentioned monitoring file storage principle specifically as shown in Figure 4.
Step S302, DM are that every memory device is provided with a counter, and a threshold value are set for this counter.
In concrete application scenarios, the account form of numerical value that can further set counter is as follows:
The numerical value of counter=∑ code stream value * way is such as existing 2 road 4Mb, 3 road 2Mb, the storage flow of 4 road 1Mb, the then numerical value=4*2+2*3+1*4=18 of counter.
The account form of the numerical value of said counting device can be adjusted according to the needs of concrete application scenarios, and such variation does not influence protection scope of the present invention.
When current numerical value reaches threshold value in the counter on the memory device, can't continue in this memory device, to add new storage plan again.
In the array rebuild process, data stream pressure is the controller of free complementary energy power to share by other, accelerates process of reconstruction, monitor data can not descended because of the array rebuild performance cause loss of data.During with the plan implementation in original option A, utilize this programme, can be with 3 full specification 220 road memorying data flows wherein, reserve one 160 road memorying data flow, when in three controllers in front a controller generation array rebuild being arranged, 60 circuit-switched data wherein temporarily can be stored on the equipment of backstage by present technique, avoid the data packet loss.
Step S303, when the some arrays of memory device because disk failure need rebuild array the time, memory device notice DM, and make this array be in degrading state waits for the affirmation instruction of DM.
Step S304, DM inquire about whether have idle storage capacity in other memory devices of its management.
After DM receives the announcement information that some arrays that a memory device reports need rebuild, all memory devices of searching and managing, whether inquiry exists the memory device different with the memory device that reports above-mentioned notification message, and the numerical value of the counter on this memory device does not reach threshold value, if exist, then have idle storage capacity in this memory device, otherwise, idle storage capacity then do not had.
If have idle storage capacity, then execution in step S305;
If do not have idle storage capacity, then execution in step S308
Step S305, DM add interim storage plan on a memory device with idle storage capacity, and create interim Tmp_SAN resource, the Counter Value on this memory device of corresponding increase simultaneously.
Step S306, DM judge whether will be to have been rebuild array in all data traffics share by interim storage plan and finishing.
If, execution in step S310 then;
If not, execution in step S307 then.
Step S307, DM judge whether the numerical value of the counter of the memory device that this sets up interim storage plan reaches threshold value.
If do not reach, return step S305, continue in this memory device, to add interim storage plan;
If reach, then stop on this memory device and to create interim Tmp_SAN, return step S304, whether the memory device of inquiring about other has idle storage capacity, and proceeds to create on other have the memory device of idle storage capacity.
Step S308, DM add interim storage plan in the regular array of the memory device that reports notification message, and create interim Tmp_SAN resource.
Step S309, DM judge the interim storage resources of continuation establishment that whether also has living space on this memory device.
If have, execution in step S306 then;
If no, then execution in step S310 directly rebuilds.
Step S310, DM notice EC writes data among the Tmp_SAN.
DM has a data mapping presentation, in order to write down the corresponding relation of former SAN resource and Tmp_SAN resource, also records zero-time and concluding time that monitor data is redirected in the table.After the Tmp_SAN establishment was finished, DM notice EC write data among the Tmp_SAN.
Step S311, DM notify corresponding memory device to begin array rebuild, and write down corresponding zero-time.
When all EC be connected with Tmp_SAN set up finish after, DM notice memory device becomes the reconstruction state by waiting status, begins to rebuild.
DM data map presentation is as shown in table 1:
| The EC title | Server address | The SAN resource name | The Tmp server address | Zero-time | Concluding time |
| ??EC2004_25101 | ??148.100.0.148 | ?SANDisk：5 | ??148.200.0.156 | ??12:56 | |
| ??EC2004_25102 | ??148.100.0.148 | ?SANDisk：6 | ??148.200.0.156 | ??12:56 |
| The EC title | Server address | The SAN resource name | The Tmp server address | Zero-time | Concluding time |
| ??EC2004_25103 | ??148.100.0.148 | ?SANDisk：7 | ??148.200.0.156 | ??12:56 | |
| ??EC2004_25104 | ??148.100.0.148 | ?SANDisk：8 | ??148.200.0.156 | ??12:56 | |
| ??EC2004_25201 | ??148.100.0.148 | ?SANDisk：9 | ??148.200.0.156 | ??12:56 | |
| ??EC2004_25202 | ??148.100.0.148 | ?SANDisk：10 | ??148.200.0.156 | ??12:56 | |
| ??EC2004_25203 | ??148.100.0.148 | ?SANDisk：11 | ??148.200.0.156 | ??12:56 |
Table 1DM data mapping tables
The name of Tmp_SAN when creating first is called: the Tmp+SAN resource name is Tmp_SANDisk:5 as the SAN resource name of EC2004_25101 correspondence in the table 1.
The Tmp_SAN resource only comprises the one-level index block of SUPER DATA, 128KB of 16KB and the second-level index block of a 256MB.When the monitor data of storage surpasses the second-level index block capacity, be that unit expands with 256MB, and in the one-level index block, write down the first address of expansion area.Can not expand if the place memory device has had the space, monitor data stream is directed in the original SAN resource, and will insert the current time in the concluding time hurdle of DM data mapping tables.
After finishing array rebuild in step S312, the memory device, memory device notice DM, the monitor data stream of corresponding stored equipment redirected on original memory device during DM will show, and will insert the current time in the concluding time hurdle of DM data mapping tables record strip simultaneously.
All data are recovered normal writing mode.
Deduct the mapping table record strip concluding time when being equal to, or greater than the maximum data retention time when the DM current time, then delete this record, notify memory device to delete this Tmp_SAN resource Free up Memory simultaneously, and corresponding adjustment Counter Value.
When needing the playback monitor data, at first relatively whether the playback duration section comprises the time period in the record strip of DM data mapping tables, if comprise, comprises that then partial data will read from the Tmp_SAN resource.
By using technical scheme of the present invention, in the array rebuild process, data stream pressure can be the memory device of free complementary energy power to share by other, accelerate process of reconstruction, monitor data can not descended because of the array rebuild performance cause loss of data, thereby improved the reliability of the reading and writing data of system, and improved the treatment effeciency of array rebuild in the system.
In order to realize technical scheme of the present invention, the invention allows for a kind of data management server, be applied to comprise in the system of a data management server, a plurality of memory device and a plurality of encoding devices, encoding device is storing data information in corresponding memory device, comprise one or more storage arrays in the memory device, its structural representation comprises as shown in Figure 5:
Wherein, scheduler module 54, also be used at the Query Result of enquiry module 53 when not existing, in send rebuilding the normal storage array that memory device comprised of notification message, create interim storage resources, and will to need to the storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array be the address of interim storage resources;
Judge storage array to be rebuild when judge module 55 and finish when rebuilding, the address of the storage array after scheduler module 54 also is used for reverting to reconstruction to the storage destination address of the encoding device of interim storage resources storage data.
Further, this data management server also comprises:
The storage relation mapping table that scheduler module 54 is set up according to logging modle 56, according to the corresponding relation of storage relationship map table record, the address of the storage array after the storage destination address that need store the encoding device of data in each interim storage resources reverts to reconstruction.
In the concrete application scenarios, logging modle 56 also is used for zero-time and termination time in storage relation mapping table each encoding device of record storing data information in each interim storage resources.
Compared with prior art, the present invention has the following advantages:
By using technical scheme of the present invention, in the array rebuild process, data stream pressure can be the memory device of free complementary energy power to share by other, accelerate process of reconstruction, monitor data can not descended because of the array rebuild performance cause loss of data, thereby improved the reliability of the reading and writing data of system, and improved the treatment effeciency of array rebuild in the system.
Through the above description of the embodiments, those skilled in the art can be well understood to the present invention and can realize by hardware, also can realize by the mode that software adds necessary general hardware platform.Based on such understanding, technical scheme of the present invention can embody with the form of software product, it (can be CD-ROM that this software product can be stored in a non-volatile memory medium, USB flash disk, portable hard drive etc.) in, comprise some instructions with so that computer equipment (can be personal computer, server, the perhaps network equipment etc.) each implements the described method of scene to carry out the present invention.
It will be appreciated by those skilled in the art that accompanying drawing is a preferred synoptic diagram of implementing scene, module in the accompanying drawing or flow process might not be that enforcement the present invention is necessary.
It will be appreciated by those skilled in the art that the module in the device of implementing in the scene can be distributed in the device of implementing scene according to implementing scene description, also can carry out respective change and be arranged in the one or more devices that are different from this enforcement scene.The module of above-mentioned enforcement scene can be merged into a module, also can further split into a plurality of submodules.
The invention described above sequence number is not represented the quality of implementing scene just to description.
More than disclosed only be several concrete enforcement scene of the present invention, still, the present invention is not limited thereto, any those skilled in the art can think variation all should fall into protection scope of the present invention.
Claims (10)
1. disk array reconstruction method, be applied to comprise in the system of a data management server, a plurality of memory device and a plurality of encoding devices, described encoding device is storing data information in corresponding memory device, described data management server is used for described a plurality of memory devices are managed, comprise one or more storage arrays in the described memory device, it is characterized in that, described data management server is that described a plurality of memory device is provided with storage flow batching counter and storage flow threshold respectively, and described method specifically may further comprise the steps:
When described data management server receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, described data management server judges that the current numerical value of storage flow batching counter that whether exists does not reach other memory devices of storing flow threshold;
If exist, described data management server is not created interim storage resources in the numerical value of one or more storage flow batching counters reaches the memory device of storage flow threshold, and will to need to the described storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array be the address of described interim storage resources;
When described data management apparatus will need when described storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array all is set at the address of described interim storage resources, described data management apparatus sends the reconstruction acknowledge message of waiting to rebuild array to described memory device, so that storage array described to be rebuild begins to rebuild;
Wait to rebuild storage array and rebuild when finishing when described, described data management server will need the address of the storage array after the storage destination address of the encoding device of storage data in described interim storage resources reverts to reconstruction.
2. the method for claim 1, it is characterized in that, when described data management server receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, described data management server judges that the current numerical value of storage flow batching counter that whether exists does not reach after other memory devices of storage flow threshold, also comprises:
If there is no, described data management server is created interim storage resources in sending the normal storage array that memory device comprised of described reconstruction notification message, and will to need to the described storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array be the address of described interim storage resources;
The address that will need all to be set at described interim storage resources when described data management apparatus to described storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array, or the numerical value of the storage flow batching counter in the memory device of described transmission reconstruction notification message reaches when storing flow threshold, described data management apparatus sends the reconstruction acknowledge message of waiting to rebuild array to described memory device, so that storage array described to be rebuild begins to rebuild;
When the described reconstruction of waiting to rebuild storage array is finished, described data management server will need the address of the storage array after the storage destination address of the encoding device of storage data in described interim storage resources reverts to reconstruction.
3. method as claimed in claim 1 or 2 is characterized in that, it is the address of described interim storage resources that described data management server will need to the described storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array, also comprises:
Described data management server is set up the storage relation mapping table, and described storage relationship map table record is described waits to rebuild storage resources in the storage array and the corresponding relation between the described interim storage resources.
4. method as claimed in claim 3 is characterized in that, described data management server will need the address of the storage array after the storage destination address of the encoding device of storage data in described interim storage resources reverts to reconstruction, specifically comprise:
Described data management server is according to described storage relation mapping table, according to the corresponding relation of described storage relationship map table record, the address of the storage array after the storage destination address that need store the encoding device of data in each described interim storage resources reverts to reconstruction.
5. method as claimed in claim 4 is characterized in that, also comprises:
Described data management server writes down the zero-time and the termination time of each described encoding device storing data information in each described interim storage resources in described storage relation mapping table.
6. data management server, be applied to comprise in the system of a data management server, a plurality of memory device and a plurality of encoding devices, described encoding device is storing data information in corresponding memory device, comprise one or more storage arrays in the described memory device, it is characterized in that, comprising:
Module is set, is used to described a plurality of memory device that storage flow batching counter and storage flow threshold are set respectively;
Communication module is used to receive the reconstruction notification message of waiting to rebuild storage array that memory device sends;
Enquiry module, be connected with described communication module with the described module that is provided with, be used for when described communication module receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, inquire about and currently whether exists the described numerical value that the set storage flow batching counter of module is set not reach the memory device of storing flow threshold;
Scheduler module, be connected with described enquiry module, be used at the Query Result of described enquiry module when existing, numerical value at one or more storage flow batching counters does not reach the interim storage resources of establishment in the memory device of storing flow threshold, and will to need to the described storage setting recipient of waiting to rebuild in the storage array encoding device of storage data be the address of described interim storage resources, and wait to rebuild storage array and rebuild when finishing described, the address of the storage array after the storage destination address that need store the encoding device of data in described interim storage resources reverts to reconstruction;
Judge module, be connected with described communication module with described scheduler module, be used for judging whether described scheduler module will need all to be set to the described storage destination address of waiting to rebuild whole encoding devices of storage array storage data the address of described interim storage resources, in judged result when being, indicate described communication module to send the reconstruction acknowledge message of waiting to rebuild array to described memory device, so that storage array described to be rebuild begins to rebuild, and judge and describedly wait to rebuild storage array and whether finish reconstruction.
7. data management server as claimed in claim 6 is characterized in that,
Described scheduler module, also be used at the Query Result of described enquiry module when not existing, in sending the normal storage array that memory device comprised of described reconstruction notification message, create interim storage resources, and will to need to the described storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array be the address of described interim storage resources;
Judge described scheduler module when described judge module and will need all to be set at the address of described interim storage resources to described storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array, or described enquiry module inquires described transmission when rebuilding the numerical value of the storage flow batching counter in the memory device of notification message and reaching the storage flow threshold, described notification module indicates described communication module to send the reconstruction acknowledge message of waiting to rebuild array to described memory device, so that storage array described to be rebuild begins to rebuild;
When described judge module judges that storage array described to be rebuild is finished reconstruction, the address of the storage array after described scheduler module also is used for reverting to reconstruction to the storage destination address that described interim storage resources is stored the encoding device of data.
8. as claim 6 or 7 described data management servers, it is characterized in that, also comprise:
Logging modle, be connected with described scheduler module, be used for setting up storage relation mapping table, storage resources in the storage array that the needs that the described scheduler module of described storage relationship map table record is generated are rebuild and the corresponding relation between the described interim storage resources according to the scheduling situation of described scheduler module.
9. data management server as claimed in claim 8 is characterized in that,
The storage relation mapping table that described scheduler module is set up according to described logging modle, according to the corresponding relation of described storage relationship map table record, the address of the storage array after the storage destination address that need store the encoding device of data in each described interim storage resources reverts to reconstruction.
10. data management server as claimed in claim 9 is characterized in that,
Described logging modle also is used for zero-time and termination time in described storage relation mapping table each described encoding device of record storing data information in each described interim storage resources.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101896403A CN101859265B (en) | 2010-06-02 | 2010-06-02 | Disk array reconstruction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101896403A CN101859265B (en) | 2010-06-02 | 2010-06-02 | Disk array reconstruction method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101859265A true CN101859265A (en) | 2010-10-13 |
| CN101859265B CN101859265B (en) | 2012-09-26 |
Family
ID=42945186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010101896403A Expired - Fee Related CN101859265B (en) | 2010-06-02 | 2010-06-02 | Disk array reconstruction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101859265B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436355A (en) * | 2011-11-15 | 2012-05-02 | 华为技术有限公司 | Data transmission method, device and system |
| CN107888678A (en) * | 2017-11-08 | 2018-04-06 | 北京奇艺世纪科技有限公司 | A kind of control method and device of file download |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101241453A (en) * | 2008-03-05 | 2008-08-13 | 杭州华三通信技术有限公司 | Magnetic disc redundant array maintenance method and device |
| CN101540173A (en) * | 2009-04-27 | 2009-09-23 | 杭州华三通信技术有限公司 | Method and device for storing data in reconstruction of disk array |
| CN101609420A (en) * | 2009-07-17 | 2009-12-23 | 杭州华三通信技术有限公司 | Realize method and the redundant arrays of inexpensive disks and the controller thereof of rebuilding of disc redundant array |
-
2010
- 2010-06-02 CN CN2010101896403A patent/CN101859265B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101241453A (en) * | 2008-03-05 | 2008-08-13 | 杭州华三通信技术有限公司 | Magnetic disc redundant array maintenance method and device |
| CN101540173A (en) * | 2009-04-27 | 2009-09-23 | 杭州华三通信技术有限公司 | Method and device for storing data in reconstruction of disk array |
| CN101609420A (en) * | 2009-07-17 | 2009-12-23 | 杭州华三通信技术有限公司 | Realize method and the redundant arrays of inexpensive disks and the controller thereof of rebuilding of disc redundant array |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436355A (en) * | 2011-11-15 | 2012-05-02 | 华为技术有限公司 | Data transmission method, device and system |
| CN102436355B (en) * | 2011-11-15 | 2014-06-25 | 华为技术有限公司 | Data transmission method, device and system |
| CN107888678A (en) * | 2017-11-08 | 2018-04-06 | 北京奇艺世纪科技有限公司 | A kind of control method and device of file download |
| CN107888678B (en) * | 2017-11-08 | 2020-11-13 | 北京奇艺世纪科技有限公司 | Method and device for controlling file downloading |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101859265B (en) | 2012-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105653202A (en) | Multi-tier scheme for logical storage management | |
| US7321955B2 (en) | Control device, control method and storage medium recording a control program for controlling write-back schedule of data from cache memory to a plurality of storage devices | |
| CN104317742A (en) | Thin provisioning method for optimizing space management | |
| CN100498678C (en) | Method and system for read-write operation to cheap magnetic disk redundant array | |
| US9158695B2 (en) | System for dynamically adaptive caching | |
| CN103049220A (en) | Storage control method, storage control device and solid-state storage system | |
| CN102682012A (en) | Method and device for reading and writing data in file system | |
| US9569381B2 (en) | Scheduler for memory | |
| CN103514249A (en) | Method and system for automatic data reduction and storage device | |
| CN102541746B (en) | Method and system for array data processing | |
| CN105302497A (en) | Cache management method and cache management system | |
| CN104679668A (en) | Storage system and control method thereof | |
| CN102177496A (en) | System and method for transferring data between different RAID data storage types for current data and replay data | |
| KR102022721B1 (en) | System for dynamically adaptive caching | |
| CN115617742A (en) | Data caching method, system, equipment and storage medium | |
| CN102523410B (en) | Method for writing video data and video data storage equipment | |
| CN109375868B (en) | Data storage method, scheduling device, system, equipment and storage medium | |
| CN101729421A (en) | Storage method and device based on time division multiplex | |
| CN101859265B (en) | Disk array reconstruction method and device | |
| CN102883143B (en) | A kind of method adjusting RAID array buffer memory | |
| CN104503924A (en) | Area distribution method and device of tiered storage system | |
| CN109271293A (en) | Hard disk data reading method and device and related equipment | |
| CN106557277B (en) | The reading method and device of disk array | |
| KR20150127434A (en) | Memory management apparatus and control method thereof | |
| CN103714022A (en) | Mixed storage system based on data block |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 310052 Binjiang District Changhe Road, Zhejiang, China, No. 466, No. Patentee after: Xinhua three Technology Co., Ltd. Address before: 310053 Hangzhou hi tech Industrial Development Zone, Zhejiang province science and Technology Industrial Park, No. 310 and No. six road, HUAWEI, Hangzhou production base Patentee before: Huasan Communication Technology Co., Ltd. |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120926 Termination date: 20200602 |