CN101859265B - Disk array reconstruction method and device - Google Patents
Disk array reconstruction method and device Download PDFInfo
- Publication number
- CN101859265B CN101859265B CN2010101896403A CN201010189640A CN101859265B CN 101859265 B CN101859265 B CN 101859265B CN 2010101896403 A CN2010101896403 A CN 2010101896403A CN 201010189640 A CN201010189640 A CN 201010189640A CN 101859265 B CN101859265 B CN 101859265B
- Authority
- CN
- China
- Prior art keywords
- storage
- rebuild
- array
- data
- reconstruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a disk array reconstruction method and a device. By utilizing the technical scheme, during the reconstruction process of the array, the pressure of data stream can be shared by other memory devices with surplus capacity so as to accelerate the reconstruction process, so the monitored data is free from being lost caused by the decrease of the array reconstruction performance, the reading and writing reliability of the data of the system can be improved, and the processing efficiency of the array reconstruction in the system can be improved.
Description
Technical field
The present invention relates to communication technical field, particularly a kind of disk array reconstruction method and equipment.
Background technology
In the video monitoring service, vision signal gets into EC (Video Codec) through camera, through coding module with 1 second data compilation become an I frame group (forming) of specifying the code check size by an I frame and a plurality of P frame.Every I frame that compiles out or P frame all will be placed in the buffer area of EC in this process.EC gives out a contract for a project module at every turn from a buffer area taking-up I frame group (1 second data), sends message through TCP (TransmissionControl Protocol, transmission control protocol) agreement to memory device.Memory device returns a feedback information notice EC continuation transmission after storing message data into data disks, and the module of giving out a contract for a project is obtained an I frame group again from buffer area, the forwarding step before continuing.
The EC buffer size is limited, generally can only several seconds to tens seconds data of buffer memory.If storage device processes I/O is (Input/Output; I/O) performance is lower; The time of accomplishing data storage is greater than 1 second, makes that the EC module of giving out a contract for a project is also corresponding elongated from the time that buffer area obtains I frame group, and the data formation speed causes the water level of EC buffer area to go up greater than take-off speed.When water level surpassed cache size, follow-up I frame of coming from collector (or P frame) will cover original data buffer memory, causes packet loss.
In the application of reality, memory property descends and to mainly contain two reasons and cause during array rebuild:
First reason; As shown in Figure 1, when disk 3 damages, when rebuilding behind the RAID5 array degradation; Need from disk 1,2,4,5, strip data and check bit to be read respectively; Make each disk be in frequent read data status, so disk will increase greatly to response time of other operations, and also write the dish performance and sharply descend with regard to the array that makes.
Second reason after strip data and verification and from disk, reading, will be transferred to verification and chip and handle, calculate the corresponding data lost on the disk 3 or verification with, the data that calculate are write direct in the disk of rebuilding.This process RAID (Redundant Array of Independent Disk; Raid-array) the consistent busy state that is in of the verification on the memory device with chip; Verification after new data write is corresponding slowly, also is the reason that causes whole storage device performance to reduce.
Existing memory device generally can adopt the array that has redundant ability to store for the safety of data; But when redundant array generation data disks is lost and is rebuild; The performance of memory device will have more significantly and descend; Packet drop need be considered to rebuild during deployment, matching E C ability can't be arrived when causing under the normal condition configure storage devices.
Present existing scheme, in order to guarantee that packet loss does not take place the array rebuild performance back EC that descends to occur at memory device, the EC storage plan quantity of when plan implementation, distributing in the storage is that the performance when rebuilding simultaneously by array is implemented.
Such as in the scheme of an enforcement; 4 memory storage devices of a DM control; 4 RAID5 arrays (each array is made up of 15+1 disk) are arranged under each memory device; When the array under the memory device all was in the normal state, each memory device can be handled 220 road EC 4Mb storage flow at most packet loss not take place, and four memory devices then are 880 the tunnel.When 1 array rebuild was arranged under certain memory device, packet loss can not take place when being no more than 160 the tunnel in the monitor data of this memory device, stores stream maximum.
And in the memory device array to take place to rebuild probability be very large, so when monitoring scheme is disposed for fear of the generation of EC end packet loss phenomenon, the 4Mb storage flow that every memory device distributes down is no more than 160 the tunnel at most! This is the 160x4=640 road by the maximum storable 4Mb storage flow of the embodiment that a DM manages, 4 memory devices are formed so in the above.
In realizing process of the present invention, the inventor finds that there is following problem at least in prior art:
Processing power when existing technology is used array rebuild is disposed embodiment; Wasted the handling property under a part of memory device normal condition; In the time of quite most of storage array normal condition, the storage capacity of storage system can't fully be used.
Summary of the invention
The present invention provides a kind of disk array reconstruction method and equipment, has solved that redundant array generation data disks is lost and when rebuilding, the problem that the performance of memory device descends and causes monitor data to be lost.
For achieving the above object; One aspect of the present invention provides a kind of disk array reconstruction method; Be applied to comprise in the system of a data management server, a plurality of memory device and a plurality of encoding devices; Said encoding device is storing data information in corresponding memory device, and said data management server is used for said a plurality of memory devices are managed, and comprises one or more storage arrays in the said memory device; Said data management server is that said a plurality of memory device is provided with storage flow batching counter and storage flow threshold respectively, and said method specifically may further comprise the steps:
When said data management server receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, said data management server judges that the current numerical value of storage flow batching counter that whether exists does not reach other memory devices of storing flow threshold;
If exist; Said data management server does not reach the interim storage resources of establishment in the memory device of storing flow threshold at the numerical value of one or more storage flow batching counters, and need be the address of said interim storage resources to the said storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array;
When said data management apparatus need be when said storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array all be set at the address of said interim storage resources; Said data management apparatus sends the reconstruction acknowledge message of waiting to rebuild array to said memory device, so that storage array said to be rebuild begins to rebuild;
When said when waiting to rebuild storage array and rebuild accomplishing, the address of the storage array after said data management server need be stored the encoding device of data in said interim storage resources storage destination address reverts to reconstruction.
Preferably; When said data management server receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time; Said data management server judges that the current numerical value of storage flow batching counter that whether exists does not reach after other memory devices of storage flow threshold, also comprises:
If do not exist; Said data management server is created interim storage resources in the normal storage array that memory device comprised that sends said reconstruction notification message, and need be the address of said interim storage resources to the said storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array;
Need all be set at the address of said interim storage resources to said storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array when said data management apparatus; Or the numerical value of the storage flow batching counter in the memory device of said transmission reconstruction notification message reaches when storing flow threshold; Said data management apparatus sends the reconstruction acknowledge message of waiting to rebuild array to said memory device, so that storage array said to be rebuild begins to rebuild;
When the said reconstruction of waiting to rebuild storage array is accomplished, the address of the storage array after said data management server need be stored the encoding device of data in said interim storage resources storage destination address reverts to reconstruction.
Preferably, said data management server need be the address of said interim storage resources to the said storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array, also comprise:
Said data management server is set up the storage relation mapping table, and said storage relationship map table record is said waits to rebuild storage resources and the corresponding relation between the said interim storage resources in the storage array.
Preferably, the address of the storage array after said data management server need be stored the encoding device of data in said interim storage resources storage destination address reverts to reconstruction specifically comprises:
Said data management server is according to said storage relation mapping table; According to said storage relation mapping table recorded corresponding relation, the address of the storage array after the storage destination address that need in each said interim storage resources, store the encoding device of data reverts to reconstruction.
Preferably, said method also comprises:
Said data management server writes down the zero-time and the termination time of each said encoding device storing data information in each said interim storage resources in said storage relation mapping table.
On the other hand; The present invention also provides a kind of data management server; Be applied to comprise in the system of a data management server, a plurality of memory device and a plurality of encoding devices; Said encoding device is storing data information in corresponding memory device, comprises one or more storage arrays in the said memory device, comprising:
Module is set, is used to said a plurality of memory device storage flow batching counter and storage flow threshold are set respectively;
Communication module is used to receive the reconstruction notification message of waiting to rebuild storage array that memory device sends;
Enquiry module; Be connected with said communication module with the said module that is provided with; Be used for when said communication module receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, inquire about and currently whether exists the said numerical value that the set storage flow batching counter of module is set not reach other memory devices of storing flow threshold;
Scheduler module; Be connected with said enquiry module; Be used at the Query Result of said enquiry module when existing; Numerical value at one or more storage flow batching counters does not reach the interim storage resources of establishment in the memory device of storing flow threshold; And need be the address of said interim storage resources to the said storage setting recipient of waiting to rebuild in the storage array encoding device of storage data, and wait to rebuild storage array and rebuild when accomplishing, the address of the storage array after the storage destination address that need in said interim storage resources, store the encoding device of data reverts to reconstruction said;
Judge module; Be connected with said communication module with said scheduler module; Be used for judging that whether said scheduler module need all be set at the address of said interim storage resources to the said storage destination address of waiting to rebuild whole encoding devices of storage array storage data; In judged result when being; Indicate said communication module to send the reconstruction acknowledge message of waiting to rebuild array,, and judge and saidly wait to rebuild storage array and whether accomplish reconstruction so that storage array said to be rebuild begins to rebuild to said memory device.
Preferably; Said scheduler module; Also be used at the Query Result of said enquiry module when not existing; In the normal storage array that memory device comprised that sends said reconstruction notification message, create interim storage resources, and need be the address of said interim storage resources to the said storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array;
Judge said scheduler module when said judge module and need all be set at the address of said interim storage resources to said storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array; Or said enquiry module inquires said transmission when rebuilding the numerical value of the storage flow batching counter in the memory device of notification message and reaching the storage flow threshold; Said judge module indicates said communication module to send the reconstruction acknowledge message of waiting to rebuild array to said memory device, so that storage array said to be rebuild begins to rebuild;
Judge when said judge module and saidly to wait to rebuild storage array and accomplish when rebuilding, the address of the storage array after said scheduler module also is used for reverting to reconstruction to the storage destination address of the encoding device of said interim storage resources storage data.
Preferably, said data management server also comprises:
Logging modle is connected with said scheduler module, is used for setting up the storage relation mapping table, said interim storage resources that the said scheduler module of said storage relationship map table record is generated and the corresponding relation between the said storage resources of waiting to rebuild in the storage array.
Preferably,
Said scheduler module; Also be used for the storage relation mapping table set up according to said logging modle; According to said storage relation mapping table recorded corresponding relation, the address of the storage array after the storage destination address that need in each said interim storage resources, store the encoding device of data reverts to reconstruction.
Preferably,
Said logging modle also is used for zero-time and termination time in said storage relation mapping table each said encoding device of record storing data information in each said interim storage resources.
Compared with prior art, the present invention has the following advantages:
Through using technical scheme of the present invention; In the array rebuild process; Data stream pressure can be the memory device of free complementary energy power to share by other, accelerates process of reconstruction, monitor data can not descended because of the array rebuild performance cause loss of data; Thereby improved the reliability of the reading and writing data of system, and improved the treatment effeciency of array rebuild in the system.
Description of drawings
Fig. 1 is a RAID5 array degradation synoptic diagram in the prior art;
Fig. 2 is the schematic flow sheet of a kind of disk array reconstruction method proposed by the invention;
Fig. 3 is the schematic flow sheet of disk array reconstruction method under a kind of concrete application scenarios proposed by the invention;
Fig. 4 is a kind of synoptic diagram of monitoring the file storage ultimate principle proposed by the invention;
Fig. 5 is the structural representation of a kind of data management server proposed by the invention.
Embodiment
To deficiency of the prior art; Need in the process that storage array is rebuild, store for the data message in the storage array that should be stored in reconstruction through corresponding storage resources temporarily; In order to guarantee the continuity of storage service; After storage array rebuild to be accomplished, also need corresponding operational data storage be returned in the storage array after the reconstruction.
Based on such technical thought; The present invention proposes a kind of disk array reconstruction method; Be applied to comprise in the system of a data management server, a plurality of memory device and a plurality of encoding devices; Encoding device is storing data information in corresponding memory device, and data management server is used for a plurality of memory devices are managed, and comprises one or more storage arrays in the memory device.
Based on above-mentioned purpose, as shown in Figure 2, the schematic flow sheet for a kind of disk array reconstruction method proposed by the invention specifically may further comprise the steps:
Step S201, data management server are that a plurality of memory devices are provided with storage flow batching counter and storage flow threshold respectively.
Step S202, when data management server receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, data management server judges that the current numerical value of storage flow batching counter that whether exists does not reach other memory devices of storing flow threshold.
If there is execution in step S203;
If do not exist, execution in step S206.
Step S203, data management server do not reach the interim storage resources of establishment in the memory device of storing flow threshold at the numerical value of one or more storage flow batching counters, and need be the address of interim storage resources to the storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array.
When data management apparatus need all be set at the address of interim storage resources to the storage destination address of whole encoding devices of waiting to rebuild in the storage array storage data, execution in step S204.
Step S204, data management apparatus send the reconstruction acknowledge message of waiting to rebuild array to memory device, so that storage array to be rebuild begins to rebuild.
Step S205, rebuild when accomplishing the address of the storage array after data management server need be stored the encoding device of data in interim storage resources storage destination address reverts to reconstruction when waiting to rebuild storage array.
Step S206, data management server are created interim storage resources in sending the normal storage array that memory device comprised of rebuilding notification message, and need be the address of interim storage resources to the storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array.
Need all be set at the address of interim storage resources to the storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array when data management apparatus; Or the numerical value that sends the storage flow batching counter in the memory device of rebuilding notification message reaches when storing flow threshold execution in step S204.
In step S203 and step S206, data management server need be in the pairing storage array of need rebuilding of array rebuild notification message the data information memory of the encoding device of storage data in interim storage resources time, also comprise:
Data management server is set up the storage relation mapping table, and storage relationship map table record waits to rebuild the storage resources and the corresponding relation between the storage resources temporarily in the storage array.
In concrete application scenarios, data management server is the zero-time and the termination time of each encoding device of record storing data information in each interim storage resources in the storage relation mapping table further.
In such cases, the realization flow of step S205 specifically comprises:
Data management server is according to the storage relation mapping table, according to storage relation mapping table recorded corresponding relation, and the address of the storage array after the storage destination address that need in each interim storage resources, store the encoding device of data reverts to reconstruction.
Compared with prior art, the present invention has the following advantages:
Through using technical scheme of the present invention; In the array rebuild process; Data stream pressure can be the memory device of free complementary energy power to share by other, accelerates process of reconstruction, monitor data can not descended because of the array rebuild performance cause loss of data; Thereby improved the reliability of the reading and writing data of system, and improved the treatment effeciency of array rebuild in the system.
In order further to set forth technological thought of the present invention, combine concrete application scenarios at present, technical scheme of the present invention is described.
Technical scheme proposed by the invention is primarily aimed at that redundant array generation data disks is lost and when rebuilding; The problem that the performance of memory device descends and causes monitor data to be lost; In carrying out the array rebuild process, fully use the storage capacity of memory device normal condition as far as possible, guarantee that the monitor data in the array rebuild process is not lost.
Based on above-mentioned technological thought, the present invention has created a kind of mechanism, if in the memory device of DM management a plurality of arrays are arranged; So, when wherein certain array was prepared to begin to rebuild, corresponding memory device at first need report notice DM; This array to be rebuild is in degrading state simultaneously, waits for that DM replys.
Wherein, Degradation (Degrade) state; Claim critical conditions (Critical) again, the state after (such as the RAID5 array) or a plurality of (such as the RAID6 array) disk failure that refers to there is in the redundant array, but still can normal read-write service be provided to the external world; But this moment if a disk failure being arranged again, then whole array then is in disabled state.
After DM receives the notice that memory device reports; In other idle relatively arrays, create interim Tmp_SAN resource; And the monitor data stream of notifying corresponding EC will plan originally to write to rebuild array writes among these Tmp_SAN, notifies memory device to begin array rebuild at last.In process of reconstruction, DM safeguards the storage time of the monitor data stream that corresponding D M data map table record is redirected.
After rebuilding completion, corresponding memory device reports DM, and data stream redirects to be accomplished in the array of rebuilding (former SAN resource).And the data that write Tmp_SAN before in the process of reconstruction do not move back, can be when carrying out video playback according to the time of writing down in the DM data mapping tables, and selection needs the resource reading of data of carry, carries out playback.
After surpassing retention time and rebuilding completion, deletion Tmp_SAN resource Free up Memory.
As shown in Figure 3, the schematic flow sheet for the disk array reconstruction method under a kind of concrete application scenarios of the present invention's proposition specifically may further comprise the steps:
Monitoring storage scheme in step S301, the current system of DM initialization.
In the monitoring storage scheme, each circuit-switched data stream can be created a SAN resource that calculates by the storage fate on the array at memory device when creating storage plan, can create a file system simultaneously.File system uses the SUPER DATA of a 16K to come identification document.
SUPER DATA is the one-level index block at the back, and size is 128K.The one-level index block is the data cell of a series of 256M size at the back, is used to preserve data, the 256M data cell that act as the retrieval back of one-level index block.
Each secondary data piece is 256MB, comprises the data cell of secondary index (256KB) and 16368 16KB.The data block of each 256M is all with the secondary index BOB(beginning of block), secondary index act as the I frame group in the current data block.
One-level index in the file and secondary index are all with time keeping.
Above-mentioned monitoring file storage principle is specifically as shown in Figure 4.
Step S302, DM are that every memory device is provided with a counter, and a threshold value are set for this counter.
In concrete application scenarios, the account form of numerical value that can further set counter is following:
The numerical value of counter=∑ code stream value * way is such as existing 2 road 4Mb, 3 road 2Mb, the storage flow of 4 road 1Mb, the then numerical value=4*2+2*3+1*4=18 of counter.
The account form of the numerical value of said counting device can be adjusted according to the needs of concrete application scenarios, and such variation does not influence protection scope of the present invention.
When current numerical value reaches threshold value in the counter on the memory device, can't continue in this memory device, to add new storage plan again.
In the array rebuild process, data stream pressure is the controller of free complementary energy power to share by other, accelerates process of reconstruction, monitor data can not descended because of the array rebuild performance cause loss of data.During with the plan implementation in original option A; Utilize this programme; Can 3 full specification 220 road memorying data flows wherein be reserved one 160 road memorying data flow, when in three controllers in front a controller generation array rebuild being arranged; Can 60 circuit-switched data wherein temporarily be stored on the equipment of backstage through present technique, avoid the data packet loss.
Step S303, when the some arrays of memory device because disk failure need rebuild array the time, memory device notice DM, and make this array be in degrading state waits for the affirmation instruction of DM.
Whether has idle storage capacity in other memory devices of step S304, its management of DM inquiry.
After DM receives the announcement information that some arrays that a memory device reports need rebuild; Whether all memory devices of searching and managing, inquiry exist and the memory device different storage devices that reports above-mentioned notification message, and the numerical value of the counter on this memory device does not reach threshold value; If exist; Then have idle storage capacity in this memory device, otherwise, idle storage capacity then do not had.
If have idle storage capacity, then execution in step S305;
If do not have idle storage capacity, then execution in step S308
Step S305, DM add interim storage plan on a memory device with idle storage capacity, and create interim Tmp_SAN resource, the Counter Value on this memory device of corresponding increase simultaneously.
Step S306, DM judge whether will be to have been rebuild array in all data traffics share through interim storage plan and finishing.
If, execution in step S310 then;
If not, execution in step S307 then.
Step S307, DM judge whether the numerical value of the counter of the memory device that this sets up interim storage plan reaches threshold value.
If do not reach, return step S305, continue in this memory device, to add interim storage plan;
If reach, then stop on this memory device and to create interim Tmp_SAN, return step S304, whether the memory device of inquiring about other has idle storage capacity, and on other have the memory device of idle storage capacity, proceeds to create.
Step S308, DM add interim storage plan in the regular array of the memory device that reports notification message, and create interim Tmp_SAN resource.
Step S309, DM judge the interim storage resources of continuation establishment that whether also has living space on this memory device.
If have, execution in step S306 then;
If no, then execution in step S310 directly rebuilds.
Step S310, DM notice EC write data among the Tmp_SAN.
DM has a data mapping presentation, in order to write down the corresponding relation of former SAN resource and Tmp_SAN resource, also records zero-time and concluding time that monitor data is redirected in the table.After Tmp_SAN created completion, DM notice EC write data among the Tmp_SAN.
Step S311, DM notify corresponding memory device to begin array rebuild, and write down corresponding zero-time.
When all EC were connected the foundation completion with Tmp_SAN after, DM notice memory device became the reconstruction state by waiting status, begins to rebuild.
DM data map presentation is as shown in table 1:
| The EC title | Server address | The SAN resource name | The Tmp server address | Zero-time | Concluding time |
| EC2004_25101 | 148.100.0.148 | ?SANDisk：5 | 148.200.0.156 | 12:56 | ? |
| EC2004_25102 | 148.100.0.148 | ?SANDisk：6 | 148.200.0.156 | 12:56 | ? |
| EC2004_25103 | 148.100.0.148 | ?SANDisk：7 | 148.200.0.156 | 12:56 | ? |
| EC2004_25104 | 148.100.0.148 | ?SANDisk：8 | 148.200.0.156 | 12:56 | ? |
| EC2004_25201 | 148.100.0.148 | ?SANDisk：9 | 148.200.0.156 | 12:56 | ? |
| EC2004_25202 | 148.100.0.148 | ?SANDisk：10 | 148.200.0.156 | 12:56 | ? |
| EC2004_25203 | 148.100.0.148 | ?SANDisk：11 | 148.200.0.156 | 12:56 | ? |
Table 1DM data mapping tables
The name of Tmp_SAN when creating first is called: the Tmp+SAN resource name, the SAN resource name corresponding like EC2004_25101 in the table 1 is Tmp_SANDisk:5.
The Tmp_SAN resource only comprises the SUPER DATA of 16KB, the one-level index block of 128KB and the second-level index block of a 256MB.When the monitor data of storage surpasses the second-level index block capacity, be that unit expands with 256MB, and in the one-level index block, write down the first address of expansion area.Can not expand if the place memory device has had the space, monitor data stream is directed in the original SAN resource, and will insert the current time in the concluding time hurdle of DM data mapping tables.
After accomplishing array rebuild in step S312, the memory device, memory device notice DM, the monitor data of corresponding stored equipment stream redirected on original memory device during DM will show, and will insert the current time in the concluding time hurdle of DM data mapping tables record strip simultaneously.
All data are recovered normal writing mode.
Deduct the mapping table record strip concluding time when being equal to, or greater than the maximum data retention time when the DM current time, then delete this record, notify the memory device deletion this Tmp_SAN resource Free up Memory simultaneously, and corresponding adjustment Counter Value.
When needing the playback monitor data, at first relatively whether the playback duration section comprises the time period in the record strip of DM data mapping tables, if comprise, comprises that then partial data will read from the Tmp_SAN resource.
Through using technical scheme of the present invention; In the array rebuild process; Data stream pressure can be the memory device of free complementary energy power to share by other, accelerates process of reconstruction, monitor data can not descended because of the array rebuild performance cause loss of data; Thereby improved the reliability of the reading and writing data of system, and improved the treatment effeciency of array rebuild in the system.
In order to realize technical scheme of the present invention; The invention allows for a kind of data management server; Be applied to comprise in the system of a data management server, a plurality of memory device and a plurality of encoding devices, encoding device is storing data information in corresponding memory device, comprises one or more storage arrays in the memory device; Its structural representation is as shown in Figure 5, comprising:
Wherein, Scheduler module 54; Also be used at the Query Result of enquiry module 53 when not existing; In sending the normal storage array that memory device comprised of rebuilding notification message, create interim storage resources, and need be the address of interim storage resources to the storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array;
When judge module 55 judges that waiting to rebuild storage array accomplishes when rebuilding, the address of the storage array after scheduler module 54 also is used for reverting to reconstruction to the storage destination address of the encoding device of interim storage resources storage data.
Further, this data management server also comprises:
The storage relation mapping table that scheduler module 54 is set up according to logging modle 56; According to storage relation mapping table recorded corresponding relation, the address of the storage array after the storage destination address that need in each interim storage resources, store the encoding device of data reverts to reconstruction.
In the concrete application scenarios, logging modle 56 also is used for zero-time and termination time in storage relation mapping table each encoding device of record storing data information in each interim storage resources.
Compared with prior art, the present invention has the following advantages:
Through using technical scheme of the present invention; In the array rebuild process; Data stream pressure can be the memory device of free complementary energy power to share by other, accelerates process of reconstruction, monitor data can not descended because of the array rebuild performance cause loss of data; Thereby improved the reliability of the reading and writing data of system, and improved the treatment effeciency of array rebuild in the system.
Through the description of above embodiment, those skilled in the art can be well understood to the present invention and can realize through hardware, also can realize by the mode that software adds necessary general hardware platform.Based on such understanding; Technical scheme of the present invention can be come out with the embodied of software product, this software product can be stored in a non-volatile memory medium (can be CD-ROM, USB flash disk; Portable hard drive etc.) in; Comprise some instructions with so that computer equipment (can be personal computer, server, the perhaps network equipment etc.) each implements the described method of scene to carry out the present invention.
It will be appreciated by those skilled in the art that accompanying drawing is a preferred synoptic diagram of implementing scene, module in the accompanying drawing or flow process might not be that embodiment of the present invention is necessary.
It will be appreciated by those skilled in the art that the module in the device of implementing in the scene can be distributed in the device of implementing scene according to implementing scene description, also can carry out respective change and be arranged in the one or more devices that are different from this enforcement scene.The module of above-mentioned enforcement scene can be merged into a module, also can further split into a plurality of submodules.
The invention described above sequence number is not represented the quality of implementing scene just to description.
More than disclosedly be merely several practical implementation scene of the present invention, still, the present invention is not limited thereto, any those skilled in the art can think variation all should fall into protection scope of the present invention.
Claims (10)
1. disk array reconstruction method; Be applied to comprise in the system of a data management server, a plurality of memory device and a plurality of encoding devices; Said encoding device is storing data information in corresponding memory device; Said data management server is used for said a plurality of memory devices are managed, and comprises one or more storage arrays in the said memory device, it is characterized in that; Said data management server is that said a plurality of memory device is provided with storage flow batching counter and storage flow threshold respectively, and said method specifically may further comprise the steps:
When said data management server receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, said data management server judges that the current numerical value of storage flow batching counter that whether exists does not reach other memory devices of storing flow threshold;
If exist; Said data management server does not reach the interim storage resources of establishment in the memory device of storing flow threshold at the numerical value of one or more storage flow batching counters, and need be the address of said interim storage resources to the said storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array;
When said data management apparatus need be when said storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array all be set at the address of said interim storage resources; Said data management apparatus sends the reconstruction acknowledge message of waiting to rebuild array to said memory device, so that storage array said to be rebuild begins to rebuild;
When said when waiting to rebuild storage array and rebuild accomplishing, the address of the storage array after said data management server need be stored the encoding device of data in said interim storage resources storage destination address reverts to reconstruction.
2. the method for claim 1; It is characterized in that; When said data management server receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time; Said data management server judges that the current numerical value of storage flow batching counter that whether exists does not reach after other memory devices of storage flow threshold, also comprises:
If do not exist; Said data management server is created interim storage resources in the normal storage array that memory device comprised that sends said reconstruction notification message, and need be the address of said interim storage resources to the said storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array;
Need all be set at the address of said interim storage resources to said storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array when said data management apparatus; Or the numerical value of the storage flow batching counter in the memory device of said transmission reconstruction notification message reaches when storing flow threshold; Said data management apparatus sends the reconstruction acknowledge message of waiting to rebuild array to said memory device, so that storage array said to be rebuild begins to rebuild;
When the said reconstruction of waiting to rebuild storage array is accomplished, the address of the storage array after said data management server need be stored the encoding device of data in said interim storage resources storage destination address reverts to reconstruction.
3. according to claim 1 or claim 2 method is characterized in that, said data management server need be the address of said interim storage resources to the said storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array, also comprise:
Said data management server is set up the storage relation mapping table, and said storage relationship map table record is said waits to rebuild storage resources and the corresponding relation between the said interim storage resources in the storage array.
4. method as claimed in claim 3 is characterized in that, the address of the storage array after said data management server need be stored the encoding device of data in said interim storage resources storage destination address reverts to reconstruction specifically comprises:
Said data management server is according to said storage relation mapping table; According to said storage relation mapping table recorded corresponding relation, the address of the storage array after the storage destination address that need in each said interim storage resources, store the encoding device of data reverts to reconstruction.
5. method as claimed in claim 4 is characterized in that, also comprises:
Said data management server writes down the zero-time and the termination time of each said encoding device storing data information in each said interim storage resources in said storage relation mapping table.
6. disk array reconstruction device; Be applied to comprise in the system of a disk array reconstruction device, a plurality of memory device and a plurality of encoding devices; Said encoding device is storing data information in corresponding memory device; Comprise one or more storage arrays in the said memory device, it is characterized in that, comprising:
Module is set, is used to said a plurality of memory device storage flow batching counter and storage flow threshold are set respectively;
Communication module is used to receive the reconstruction notification message of waiting to rebuild storage array that memory device sends;
Enquiry module; Be connected with said communication module with the said module that is provided with; Be used for when said communication module receive that memory device sends wait to rebuild the reconstruction notification message of storage array the time, inquire about and currently whether exists the said numerical value that the set storage flow batching counter of module is set not reach other memory devices of storing flow threshold;
Scheduler module; Be connected with said enquiry module; Be used at the Query Result of said enquiry module when existing; Numerical value at one or more storage flow batching counters does not reach the interim storage resources of establishment in the memory device of storing flow threshold; And need be the address of said interim storage resources to the said storage setting recipient of waiting to rebuild in the storage array encoding device of storage data, and wait to rebuild storage array and rebuild when accomplishing, the address of the storage array after the storage destination address that need in said interim storage resources, store the encoding device of data reverts to reconstruction said;
Judge module; Be connected with said communication module with said scheduler module; Be used for judging that whether said scheduler module need all be set at the address of said interim storage resources to the said storage destination address of waiting to rebuild whole encoding devices of storage array storage data; In judged result when being; Indicate said communication module to send the reconstruction acknowledge message of waiting to rebuild array,, and judge and saidly wait to rebuild storage array and whether accomplish reconstruction so that storage array said to be rebuild begins to rebuild to said memory device.
7. disk array reconstruction device as claimed in claim 6 is characterized in that,
Said scheduler module; Also be used at the Query Result of said enquiry module when not existing; In the normal storage array that memory device comprised that sends said reconstruction notification message, create interim storage resources, and need be the address of said interim storage resources to the said storage setting recipient of waiting to rebuild the encoding device of storage data in the storage array;
Judge said scheduler module when said judge module and need all be set at the address of said interim storage resources to said storage destination address of waiting to rebuild whole encoding devices of storage data in the storage array; Or said enquiry module inquires said transmission when rebuilding the numerical value of the storage flow batching counter in the memory device of notification message and reaching the storage flow threshold; Said judge module indicates said communication module to send the reconstruction acknowledge message of waiting to rebuild array to said memory device, so that storage array said to be rebuild begins to rebuild;
Judge when said judge module and saidly to wait to rebuild storage array and accomplish when rebuilding, the address of the storage array after said scheduler module also is used for reverting to reconstruction to the storage destination address of the encoding device of said interim storage resources storage data.
8. like claim 6 or 7 described disk array reconstruction devices, it is characterized in that, also comprise:
Logging modle is connected with said scheduler module, is used for setting up the storage relation mapping table, said interim storage resources that the said scheduler module of said storage relationship map table record is generated and the corresponding relation between the said storage resources of waiting to rebuild in the storage array.
9. disk array reconstruction device as claimed in claim 8 is characterized in that,
Said scheduler module; Also be used for the storage relation mapping table set up according to said logging modle; According to said storage relation mapping table recorded corresponding relation, the address of the storage array after the storage destination address that need in each said interim storage resources, store the encoding device of data reverts to reconstruction.
10. disk array reconstruction device as claimed in claim 9 is characterized in that,
Said logging modle also is used for zero-time and termination time in said storage relation mapping table each said encoding device of record storing data information in each said interim storage resources.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101896403A CN101859265B (en) | 2010-06-02 | 2010-06-02 | Disk array reconstruction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101896403A CN101859265B (en) | 2010-06-02 | 2010-06-02 | Disk array reconstruction method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101859265A CN101859265A (en) | 2010-10-13 |
| CN101859265B true CN101859265B (en) | 2012-09-26 |
Family
ID=42945186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010101896403A Expired - Fee Related CN101859265B (en) | 2010-06-02 | 2010-06-02 | Disk array reconstruction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101859265B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436355B (en) * | 2011-11-15 | 2014-06-25 | 华为技术有限公司 | Data transmission method, device and system |
| CN107888678B (en) * | 2017-11-08 | 2020-11-13 | 北京奇艺世纪科技有限公司 | Method and device for controlling file downloading |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101241453A (en) * | 2008-03-05 | 2008-08-13 | 杭州华三通信技术有限公司 | Magnetic disc redundant array maintenance method and device |
| CN101540173A (en) * | 2009-04-27 | 2009-09-23 | 杭州华三通信技术有限公司 | Method and device for storing data in reconstruction of disk array |
| CN101609420A (en) * | 2009-07-17 | 2009-12-23 | 杭州华三通信技术有限公司 | Realize method and the redundant arrays of inexpensive disks and the controller thereof of rebuilding of disc redundant array |
-
2010
- 2010-06-02 CN CN2010101896403A patent/CN101859265B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101241453A (en) * | 2008-03-05 | 2008-08-13 | 杭州华三通信技术有限公司 | Magnetic disc redundant array maintenance method and device |
| CN101540173A (en) * | 2009-04-27 | 2009-09-23 | 杭州华三通信技术有限公司 | Method and device for storing data in reconstruction of disk array |
| CN101609420A (en) * | 2009-07-17 | 2009-12-23 | 杭州华三通信技术有限公司 | Realize method and the redundant arrays of inexpensive disks and the controller thereof of rebuilding of disc redundant array |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101859265A (en) | 2010-10-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105653202A (en) | Multi-tier scheme for logical storage management | |
| US7321955B2 (en) | Control device, control method and storage medium recording a control program for controlling write-back schedule of data from cache memory to a plurality of storage devices | |
| CN101727293B (en) | Method, device and system for setting solid state disk (SSD) memory | |
| CN100498678C (en) | Method and system for read-write operation to cheap magnetic disk redundant array | |
| CN103049220A (en) | Storage control method, storage control device and solid-state storage system | |
| CN102682012A (en) | Method and device for reading and writing data in file system | |
| US9158695B2 (en) | System for dynamically adaptive caching | |
| CN104317742A (en) | Thin provisioning method for optimizing space management | |
| CN102982182B (en) | Data storage planning method and device | |
| CN102541746B (en) | Method and system for array data processing | |
| CN105786400A (en) | Heterogeneous hybrid memory module, system and storage method | |
| CN104679668A (en) | Storage system and control method thereof | |
| CN102346653A (en) | Memory array and system and data protection method | |
| CN115617742A (en) | Data caching method, system, equipment and storage medium | |
| KR102022721B1 (en) | System for dynamically adaptive caching | |
| CN101827120A (en) | Cluster storage method and system | |
| CN102523410B (en) | Method for writing video data and video data storage equipment | |
| CN101859265B (en) | Disk array reconstruction method and device | |
| CN109375868B (en) | Data storage method, scheduling device, system, equipment and storage medium | |
| CN101729421A (en) | Storage method and device based on time division multiplex | |
| CN103150268A (en) | Block-level data capture method in CDP (Continuous Data Protection) | |
| CN104811646A (en) | Multi-video stream data concurrent modulation and buffering storage method based on continuous storage model | |
| CN109271293A (en) | Hard disk data reading method and device and related equipment | |
| CN104503924A (en) | Area distribution method and device of tiered storage system | |
| KR20150127434A (en) | Memory management apparatus and control method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 310052 Binjiang District Changhe Road, Zhejiang, China, No. 466, No. Patentee after: Xinhua three Technology Co., Ltd. Address before: 310053 Hangzhou hi tech Industrial Development Zone, Zhejiang province science and Technology Industrial Park, No. 310 and No. six road, HUAWEI, Hangzhou production base Patentee before: Huasan Communication Technology Co., Ltd. |
|
| CP03 | Change of name, title or address | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120926 Termination date: 20200602 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |