CN102542040B - Capacity acquiring method and system - Google Patents

Capacity acquiring method and system Download PDF

Info

Publication number
CN102542040B
CN102542040B CN201110444072.1A CN201110444072A CN102542040B CN 102542040 B CN102542040 B CN 102542040B CN 201110444072 A CN201110444072 A CN 201110444072A CN 102542040 B CN102542040 B CN 102542040B
Authority
CN
China
Prior art keywords
disk
capacity
amount
storage block
capacity information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201110444072.1A
Other languages
Chinese (zh)
Other versions
CN102542040A (en
Inventor
赵健博
洪亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Qihoo Technology Co Ltd
Original Assignee
Beijing Qihoo Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Qihoo Technology Co Ltd filed Critical Beijing Qihoo Technology Co Ltd
Priority to CN201110444072.1A priority Critical patent/CN102542040B/en
Publication of CN102542040A publication Critical patent/CN102542040A/en
Application granted granted Critical
Publication of CN102542040B publication Critical patent/CN102542040B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a capacity acquiring method and system for solving the problem of causing a burden on a system due to overlarge IO (Input/Output) consumption in a capacity acquiring process. In order to solve the problem, the invention discloses a capacity acquiring method which comprises the following steps of: sending a request for calling disk metadata to a file system of a disk; receiving the returned disk metadata which contains the total amount of storage blocks and the idle amount of storage blocks in the disk; and computing the total amount of storage blocks and the idle amount of storage blocks in the disk to acquire the capacity information of the disk. Through the invention, the disk metadata is directly called, and the IO consumption can not be generated, so that the utilization ratio of the disk is always kept in a normal level and the burden on the system can not be caused.

Description

Capacity acquiring method in distributed file system and procurement of reserve capacity system
Technical field
The application relates to computer memory technical, particularly relates to capacity acquiring method and procurement of reserve capacity system in a kind of distributed file system.
background technology
HDFS(Hadoop Distributed File System) be a distributed file system, the feature that has high fault tolerance, and the data that provide high-throughput to visit application program, are applicable to having the application program of super large data set, can solve the problem of mass data storage.
The host node of HDFS (NameNode) is communicated by letter by carrying out periodic heartbeat with memory node (DataNode), obtains the capacity information of each memory node, and then obtains the capacity information of whole cluster.And each memory node comprises a plurality of disks, so memory node adopts du command mode to obtain the space hold amount of each disk, and then obtains local space hold amount.
The specific practice that du command mode is obtained the space hold amount of each disk is, each disk that searching loop memory node comprises, and carry out following operation for each disk: from the root directory of disk storage data (for example, the catalogue of parameter d fs.data.dir configuration) start, take this root directory according to the mode of depth-first, to travel through this directory tree as root node.If run into intermediate node, this node correspondence is sub-directory, continues traversal sub-directory, until run into leaf node, what this node was corresponding is file, obtains corresponding file metadata, obtains the size information of file from file metadata.Then using after the cumulative summation of the size information of all leaf node files corresponding to certain intermediate node as the size of giving intermediate node correspondence sub-directory.Then successively add up until described directory tree has traveled through, obtained the space hold amount of All Files in whole directory tree, acquired the space hold amount of this disk, then return to du order.
In above-mentioned process, need the All Files in scanning disk; therefore conventionally can produce a large amount of IO (Input/Output; I/O) consume; this process causes disk utilization saturated; be that disk utilization has reached 100%; this has caused very large burden to system, can affect even in some cases the performance of HDFS.
summary of the invention
The application provides capacity acquiring method and the procurement of reserve capacity system in a kind of distributed file system, to solve in the capacity of obtaining IO, consumes and excessive system is made to cumbrous problem.
In order to address the above problem, the application discloses the capacity acquiring method in a kind of distributed file system, comprising:
File system to disk sends the request of calling disk metadata;
The disk metadata that reception is returned, described disk metadata comprises the total amount of storage block and the free time of storage block amount in disk;
By the calculating to the total amount of storage block in described disk and the free time of storage block amount, obtain the capacity information of disk;
The capacity information of disk is returned to df order;
Memory node calls df order to obtain the capacity information of each disk, and the capacity information of all disks is added and as the capacity information of this memory node;
Memory node periodically calls df order to obtain the capacity information of this memory node, and described capacity information is periodically transferred to host node.
Preferably, described capacity information comprises space total amount, Spatial Residual amount and space hold amount.
Preferably, the capacity of each storage block is fixed, described by the calculating to the total amount of storage block in described disk and the free time of storage block amount, obtains the capacity information of disk, comprising:
The total amount of storage block in the capacity of described storage block and described disk is multiplied each other, and result of calculation is as the space total amount of disk;
The free time amount of storage block in the capacity of described storage block and described disk is multiplied each other, and result of calculation is as the Spatial Residual amount of disk.
Preferably, the space total amount of disk is deducted to the Spatial Residual amount of disk, the space hold amount of result of calculation disk.
Preferably, class DF calls df order for realizing, and described memory node, by carrying out df order to obtain the space hold amount of each disk, comprising:
Memory node, by calling the getDfsUsed method of FSDataset class, finally calls the getDfsUsed method of FSVolume class;
The getDfsUsed method of described FSVolume class is realized and is called df order by class DF, to obtain the capacity information of disk.
Preferably, it is benchmark that memory node be take the capacity information of each disk, in corresponding disk, distributes and save data.
Accordingly, disclosed herein as well is a kind of procurement of reserve capacity system, comprising:
Send request module, for the file system to disk, send the request of calling disk metadata;
Receive meta data block, for receiving the disk metadata of returning, described disk metadata comprises the total amount of storage block and the free time of storage block amount in disk;
Calculated capacity module, for the calculating of the free time amount by the total amount of described disk storage block and storage block, obtains the capacity information of disk;
Capacity returns to module, for the capacity information of disk is returned to df order;
Obtain capacity module, for memory node, call df order to obtain the capacity information of each disk, and the capacity information of all disks is added and as the capacity information of this memory node;
Transport module, periodically calls df order to obtain the capacity information of this memory node for memory node, and described capacity information is periodically transferred to host node.
Preferably, described capacity information comprises space total amount, Spatial Residual amount and space hold amount.
Preferably, the capacity of each storage block is fixed, and described calculated capacity module, comprising:
Computer memory total amount submodule, for the total amount of the capacity of described storage block and described disk storage block is multiplied each other, result of calculation is as the space total amount of disk;
Computer memory surplus submodule, for the free time amount of the capacity of described storage block and described disk storage block is multiplied each other, result of calculation is as the Spatial Residual amount of disk.
Preferably, described calculated capacity module, also comprises:
Computer memory occupancy submodule, for deducting the space total amount of disk the Spatial Residual amount of disk, the space hold amount of result of calculation disk.
Preferably, class DF calls df order for realizing, described in obtain capacity module and comprise:
Call method submodule, by calling the getDfsUsed method of FSDataset class, finally calls the getDfsUsed method of FSVolume class for memory node;
Call instruction submodule, calls df order for the getDfsUsed method of described FSVolume class by class DF realization, to obtain the capacity information of disk.
Distribute data module, for memory node, take the capacity information of each disk is benchmark, in corresponding disk, distributes and save data.
Compared with prior art, the application comprises following advantage:
First, the application does not need to travel through according to this logical storage structure of directory tree each file of disk, obtain the size information in the metadata of each file, after successively adding up again, obtain the space hold amount of disk, but according to the physical storage structure of disk, file system to disk sends the request of calling disk metadata, the disk metadata that reception is returned, described disk metadata comprises the total amount of storage block and the free time of storage block amount in disk, again by the calculating to the total amount of storage block in described disk and the free time of storage block amount, obtain the capacity information of disk.In this process, directly call disk metadata, can not produce IO and consume, make the utilization factor of disk remain at normal level, can not cause burden to system.
accompanying drawing explanation
Fig. 1 is a kind of capacity acquiring method process flow diagram described in the embodiment of the present application;
Fig. 2 is distributed system architecture figure described in the embodiment of the present application;
Fig. 3 is a kind of procurement of reserve capacity system construction drawing described in the embodiment of the present application.
Embodiment
For the application's above-mentioned purpose, feature and advantage can be become apparent more, below in conjunction with the drawings and specific embodiments, the application is described in further detail.
The application provides a kind of capacity acquiring method, memory node directly calls disk metadata according to the physical arrangement of data storage when obtaining the capacity information of each disk, and then the capacity information that obtains disk can not produce IO consumption, make the utilization factor of disk remain at normal level, can not cause burden to system.
Disk can be divided into logical storage structure and physical storage structure when storage data.
Described logical storage structure be take file as storage cell, sets up a directory tree, and the host node in directory tree is master catalogue, and intermediate node is sub-directory, and leaf node is exactly file.
Described physical storage structure be take storage block as storage cell, a disk is divided into many storage blocks, the capacity of storage block is smaller, and the capacity of storage block is fixed, a common file can be stored in several storage blocks, for example storage block is 1KB, and the size of certain file is 5KB, so just needs 5 storage blocks to store this file.
With reference to Fig. 1, provided a kind of capacity acquiring method process flow diagram described in the embodiment of the present application.
Step 11, the request of calling disk metadata to the file system transmission of disk;
In the file system of disk, there is disk metadata, the data relevant with capacity of described disk meta-data preservation are according to the storage of disk physical storage structure, so disk metadata can comprise the data such as the total amount of storage block and the free time of storage block amount in disk.
The application obtains capacity according to disk physical storage structure, therefore, when obtaining the capacity information of each disk, can directly call the disk metadata in disk file system.Therefore need to send the request of calling disk metadata to the file system of disk.
Described file system, also claims file management system, is the software mechanism of being in charge of in operating system with storage file information.File system is comprised of three parts: software relevant to file management, be managed file and implement file management desired data structure.
Step 12, receives the disk metadata of returning, and described disk metadata comprises the total amount of storage block and the free time of storage block amount in disk;
The file system of disk, after receiving the request of calling disk metadata, can be returned to disk metadata, now receives described disk metadata, just can receive the data such as the total amount of storage block and the free time of storage block amount in disk.
Step 13, measures by calculating the total amount of storage block and the free time of storage block in described disk, obtains the capacity information of disk.
Described capacity information comprises space total amount, Spatial Residual amount and space hold amount, and the capacity of each storage block is fixed, and therefore, according to the total amount of storage block in disk metadata and the free time of storage block amount, can calculate the capacity information that obtains disk.
Concrete calculation procedure is as follows:
Step 131, multiplies each other the total amount of storage block in the capacity of described storage block and described disk, and result of calculation is as the space total amount of disk;
Step 132, multiplies each other the free time amount of storage block in the capacity of described storage block and described disk, and result of calculation is as the Spatial Residual amount of disk.
Step 133, deducts the space total amount of disk the Spatial Residual amount of disk, the space hold amount of result of calculation disk.
For example, in certain disk, the total amount of storage block is 1024, and the free time amount of storage block is 512, and the capacity of each storage block is 1KB.
The space total amount of this disk is 1KB * 1024=1024KB=1M
The Spatial Residual amount of this disk is 1KB * 512=512KB
The space hold amount of this disk is 1024KB-512KB=512KB
By above-mentioned computing method, can obtain the capacity information of this disk.
Prior art is that logical storage structure according to disk is from host node, along intermediate node, finally can traverse leaf node, from master catalogue, start finally to traverse file, and obtain file metadata, cumulative until master catalogue successively again, obtains the space occupancy volume of this disk.This process often can produce a large amount of IO (Input/Output; I/O) consume, this process causes disk utilization saturated, and disk utilization has reached 100%; this has caused very large burden to system, can affect even in some cases the performance of HDFS.
The application is according to the physical storage structure of disk, file system to disk sends the request of calling disk metadata, the disk metadata that reception is returned, described disk metadata comprises the total amount of storage block and the free time of storage block amount in disk, by the calculating to the total amount of storage block in described disk and the free time of storage block amount, obtain the capacity information of disk again.In this process, directly call disk metadata, can not produce IO and consume, make the utilization factor of disk remain at normal level, can not cause burden to system.
Preferably, after obtaining the capacity information of disk, also comprise:
The capacity information of disk is returned to df order.
Df order can perform step 11 processes to step 13 by call function, and this function finally can get the capacity information of disk, then described capacity information is returned to df order.
Preferably, memory node calls df order to obtain the capacity information of each disk, and the capacity information of all disks is added and as the capacity information of this memory node.
In the application, by df command calls function, finally get the capacity information of disk, therefore described memory node can call the capacity information that each disk is obtained in df order, then the capacity information of all disks is added and as the capacity information of this memory node.
For example, a certain memory node comprises 3 disks, by df command calls function, this function by sending request, reception of magnetic disc metadata and calculating, the capacity information that finally returns to disk 1 comprises: space total amount is 100M, Spatial Residual amount 40M and space hold amount 60M.The capacity information of disk 2 comprises: space total amount is 200M, Spatial Residual amount 120M and space hold amount 80M.The capacity information of disk 3 comprises: space total amount is 50M, Spatial Residual amount 40M and space hold amount 10M.
The capacity information of final this memory node is:
Space total amount 100M+200M+50M=350M
Spatial Residual amount 40M+120M+40M=200M
Space hold amount 60M+80M+10M=150M
Preferably, class DF calls df order for realizing, and described memory node, by carrying out df order to obtain the space hold amount of each disk, comprising:
Memory node, by calling the getDfsUsed method of FSDataset class, finally calls the getDfsUsed method of FSVolume class;
The getDfsUsed method of described FSVolume class is realized and is called df order by class DF, to obtain the capacity information of disk.
The getDfsUsed method that memory node (DataNode) calls FSDataset class is obtained the capacity information of disk, then by communicating by letter and transmit this information with the heartbeat of host node (NameNode).And the getDfsUsed method of FSDataset class is finally called the getDfsUsed method of FSVolume class.
Owing to having comprised the class that realizes of df, i.e. class DF in FSVolume.So the realization of getDfsUsed method need to be configured such that to the version by DF class.
The following situation of literary composition is set:
Figure BDA0000125429650000071
Figure BDA0000125429650000081
Wherein, object usage is the example of DF class, call the realization that " usage.getUsed () " is df order, final call function obtains space total amount and the Spatial Residual amount of disk from the management structure of the file system of each piece disk, is calculating the space hold amount that obtains disk.
Preferably, memory node periodically calls df order to obtain corresponding capacity information, and described capacity information is periodically transferred to host node.
In HDFS system, the host node of HDFS (NameNode) is communicated by letter by carrying out periodic heartbeat with memory node (DataNode), obtains the capacity information of each memory node, and then obtains the capacity information of whole cluster.
Therefore memory node can periodically call df order to obtain corresponding capacity information, and then described capacity information is periodically transferred to host node.
The application can be applied in HDFS (Hadoop Distributed File System) distributed file system, and this system can solve the problem of mass data storage.
Follow-up, host node, when storage new data, can, according to the capacity information of each memory node transmission, be determined and be stored in which memory node.
Memory node is after receiving the data of host node transmission, and the capacity information of each disk of take is benchmark, in corresponding disk, distributes and save data.
For example, the size of certain data is 200M, and the capacity information of memory node 1 is: disk A space total amount is 100M, Spatial Residual amount 40M and space hold amount 60M, and disk B space total amount is 300M, Spatial Residual amount 120M and space hold amount 180M.The capacity information of memory node 2 is: disk A space total amount is 500M, Spatial Residual amount 105M and space hold amount 395M, and disk B space total amount is 500M, Spatial Residual amount 320M and space hold amount 180M.
Therefore, host node can store data in memory node 2, and it is benchmark that memory node 2 be take the capacity information of each disk again, and the Spatial Residual amount of disk B is 320M, is greater than the big or small 200M of data, therefore can be by data allocations to storing in disk B.
With reference to Fig. 2, provided distributed system architecture figure described in the embodiment of the present application.
In described distributed system architecture, comprised following module:
Client is client, obtains the memory location of certain file from host node, then carries out write/read operation to corresponding memory node place.
NameNode is the host node in hadoop distributed file system, safeguarded the metadata information of the All Files in hadoop distributed file system, the position of the data storage of the access rights of include file, host and file etc., and these metadata informations finally all can be persisted in disk.
DataNode is the memory node in hadoop distributed file system, has stored the content of the file in hadoop distributed file system.
SecondaryNameNode is the standby host node of hadoop distributed file system, and described standby host node hadoop metadata of distributed type file system regular and that host node keeps carries out synchronously, the data of persistence on host node being merged simultaneously.
Wherein, the file in each memory node of hadoop distributed file system, is finally divided into fixed-size (for example the size of a piece is 128MB, configurable) and deposits.If block size of the big or small deficiency of file, file is stored as a piece, and its size is the actual size of file, does not understand and takes up room more.If file size is not block size alignment, last piece can not take the space of a piece, and all block size summations are the space of file actual size.
In concrete enforcement, each memory node (DataNode) can send to host node (NameNode) being stored in local piece termly.So that NameNode can perception hadoop file system in the data storage location of All Files.
Now, described file is divided into fixed-size to be deposited, and is that this is for hadoop distributed file system, for determining the data storage location of file.
And file discussed above is divided into several storage blocks, deposit, for disk, only can get capacity information.The technology that this two place is discussed is different, can not obscure and look it.
Capacity acquiring method described in the application can be applied to hadoop distributed file system, and the memory node that is specifically applied to this system is obtaining the capacity information of each disk.Memory node obtains the capacity information of each disk by calling df order, and then obtains the capacity information of this memory node, more periodically carries out heartbeat with host node and communicate by letter, and informs the capacity information of oneself.
Follow-up host node, when the deposit position of decision-making new data, can pass through the capacity information of certain memory node, comes this memory node of decision-making whether can also continue to deposit new data.
With reference to Fig. 3, provided a kind of procurement of reserve capacity system construction drawing described in the embodiment of the present application.
Accordingly, the application also provides a kind of procurement of reserve capacity system, comprises and sends request module 11, reception meta data block 12 and calculated capacity module 13, wherein
Send request module 11, for the file system to disk, send the request of calling disk metadata;
Receive meta data block 12, for receiving the disk metadata of returning, described disk metadata comprises the total amount of storage block and the free time of storage block amount in disk;
Calculated capacity module 13, for the calculating of the free time amount by the total amount of described disk storage block and storage block, obtains the capacity information of disk.
Wherein, described capacity information comprises space total amount, Spatial Residual amount and space hold amount.
Preferably, the capacity of each storage block is fixed, and described calculated capacity module 13, comprising:
Computer memory total amount submodule 131, for the total amount of the capacity of described storage block and described disk storage block is multiplied each other, result of calculation is as the space total amount of disk;
Computer memory surplus submodule 132, for the free time amount of the capacity of described storage block and described disk storage block is multiplied each other, result of calculation is as the Spatial Residual amount of disk;
Computer memory occupancy submodule 133, for deducting the space total amount of disk the Spatial Residual amount of disk, the space hold amount of result of calculation disk.
Preferably, described system also comprises:
Capacity returns to module 14, for the capacity information of disk is returned to df order.
Obtain capacity module 15, for memory node, call df order to obtain the capacity information of each disk, and the capacity information of all disks is added and as the capacity information of this memory node.
Transport module 16, periodically calls df order to obtain corresponding capacity information for memory node, and described capacity information is periodically transferred to host node.
Distribute data module 17, for memory node, take the capacity information of each disk is benchmark, in corresponding disk, distributes and save data.
Preferably, class DF calls df order for realizing, described in obtain capacity module 15 and comprise:
Call method submodule 151, by calling the getDfsUsed method of FSDataset class, finally calls the getDfsUsed method of FSVolume class for memory node;
Call instruction submodule 152, calls df order for the getDfsUsed method of described FSVolume class by class DF realization, to obtain the capacity information of disk.
For system embodiment, because it is substantially similar to embodiment of the method, so description is fairly simple, relevant part is referring to the part explanation of embodiment of the method.
Each embodiment in this instructions all adopts the mode of going forward one by one to describe, and each embodiment stresses is the difference with other embodiment, between each embodiment identical similar part mutually referring to.
The application can describe in the general context of the computer executable instructions of being carried out by computing machine, for example program module.Usually, program module comprises the routine carrying out particular task or realize particular abstract data type, program, object, assembly, data structure etc.Also can in distributed computing environment, put into practice the application, in these distributed computing environment, by the teleprocessing equipment being connected by communication network, be executed the task.In distributed computing environment, program module can be arranged in the local and remote computer-readable storage medium that comprises memory device.
Finally, also it should be noted that, in this article, relational terms such as the first and second grades is only used for an entity or operation to separate with another entity or operational zone, and not necessarily requires or imply and between these entities or operation, have the relation of any this reality or sequentially.And, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thereby the process, method, commodity or the equipment that make to comprise a series of key elements not only comprise those key elements, but also comprise other key elements of clearly not listing, or be also included as the intrinsic key element of this process, method, commodity or equipment.The in the situation that of more restrictions not, the key element being limited by statement " comprising ... ", and be not precluded within process, method, commodity or the equipment that comprises described key element and also have other identical element.
A kind of capacity acquiring method and the system that above the application are provided, be described in detail, applied specific case herein the application's principle and embodiment are set forth, the explanation of above embodiment is just for helping to understand the application's method and core concept thereof; Meanwhile, for one of ordinary skill in the art, the thought according to the application, all will change in specific embodiments and applications, and in sum, this description should not be construed as the restriction to the application.

Claims (12)

1. the capacity acquiring method in distributed file system, is characterized in that, comprising:
According to the physical storage structure of disk, the request of calling disk metadata to the file system transmission of disk;
The disk metadata that reception is returned, described disk metadata comprises the total amount of storage block and the free time of storage block amount in disk;
By the calculating to the total amount of storage block in described disk and the free time of storage block amount, obtain the capacity information of disk;
The capacity information of disk is returned to df order;
Memory node calls df order to obtain the capacity information of each disk, and the capacity information of all disks is added and as the capacity information of this memory node;
Memory node periodically calls df order to obtain the capacity information of this memory node, and described capacity information is periodically transferred to host node.
2. method according to claim 1, is characterized in that, described capacity information comprises space total amount, Spatial Residual amount and space hold amount.
3. method according to claim 2, is characterized in that, the capacity of each storage block is fixed, described by the calculating to the total amount of storage block in described disk and the free time of storage block amount, obtains the capacity information of disk, comprising:
The total amount of storage block in the capacity of described storage block and described disk is multiplied each other, and result of calculation is as the space total amount of disk;
The free time amount of storage block in the capacity of described storage block and described disk is multiplied each other, and result of calculation is as the Spatial Residual amount of disk.
4. method according to claim 3, is characterized in that, also comprises:
The space total amount of disk is deducted to the Spatial Residual amount of disk, the space hold amount of result of calculation disk.
5. method according to claim 1, is characterized in that, class DF calls df order for realizing, and described memory node, by carrying out df order to obtain the space hold amount of each disk, comprising:
Memory node, by calling the getDfsUsed method of FSDataset class, finally calls the getDfsUsed method of FSVolume class;
The getDfsUsed method of described FSVolume class is realized and is called df order by class DF, to obtain the capacity information of disk.
6. method according to claim 1, is characterized in that, also comprises:
It is benchmark that memory node be take the capacity information of each disk, in corresponding disk, distributes and save data.
7. a procurement of reserve capacity system, is characterized in that, comprising:
Send request module, for according to the physical storage structure of disk, to the file system of disk, send the request of calling disk metadata;
Receive meta data block, for receiving the disk metadata of returning, described disk metadata comprises the total amount of storage block and the free time of storage block amount in disk;
Calculated capacity module, for the calculating of the free time amount by the total amount of described disk storage block and storage block, obtains the capacity information of disk;
Capacity returns to module, for the capacity information of disk is returned to df order;
Obtain capacity module, for memory node, call df order to obtain the capacity information of each disk, and the capacity information of all disks is added and as the capacity information of this memory node;
Transport module, periodically calls df order to obtain the capacity information of this memory node for memory node, and described capacity information is periodically transferred to host node.
8. system according to claim 7, is characterized in that, described capacity information comprises space total amount, Spatial Residual amount and space hold amount.
9. system according to claim 8, is characterized in that, the capacity of each storage block is fixed, and described calculated capacity module, comprising:
Computer memory total amount submodule, for the total amount of the capacity of described storage block and described disk storage block is multiplied each other, result of calculation is as the space total amount of disk;
Computer memory surplus submodule, for the free time amount of the capacity of described storage block and described disk storage block is multiplied each other, result of calculation is as the Spatial Residual amount of disk.
10. system according to claim 9, is characterized in that, described calculated capacity module, also comprises:
Computer memory occupancy submodule, for deducting the space total amount of disk the Spatial Residual amount of disk, the space hold amount of result of calculation disk.
11. systems according to claim 7, is characterized in that, class DF calls df order for realizing, described in obtain capacity module and comprise:
Call method submodule, by calling the getDfsUsed method of FSDataset class, finally calls the getDfsUsed method of FSVolume class for memory node;
Call instruction submodule, calls df order for the getDfsUsed method of described FSVolume class by class DF realization, to obtain the capacity information of disk.
12. systems according to claim 7, is characterized in that, also comprise:
Distribute data module, for memory node, take the capacity information of each disk is benchmark, in corresponding disk, distributes and save data.
CN201110444072.1A 2011-12-27 2011-12-27 Capacity acquiring method and system Active CN102542040B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201110444072.1A CN102542040B (en) 2011-12-27 2011-12-27 Capacity acquiring method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201110444072.1A CN102542040B (en) 2011-12-27 2011-12-27 Capacity acquiring method and system

Related Child Applications (2)

Application Number Title Priority Date Filing Date
CN201310659397.0A Division CN103678563A (en) 2011-12-27 2011-12-27 Capacity obtaining method and system
CN201310659390.9A Division CN103678562A (en) 2011-12-27 2011-12-27 Capacity obtaining method and file data allocation method

Publications (2)

Publication Number Publication Date
CN102542040A CN102542040A (en) 2012-07-04
CN102542040B true CN102542040B (en) 2014-01-29

Family

ID=46348920

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201110444072.1A Active CN102542040B (en) 2011-12-27 2011-12-27 Capacity acquiring method and system

Country Status (1)

Country Link
CN (1) CN102542040B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104881245A (en) * 2014-12-30 2015-09-02 北京飞杰信息技术有限公司 Configuration method and device of storage system
CN108197010A (en) * 2017-12-29 2018-06-22 郑州云海信息技术有限公司 Capacity statistical method, system, device and the storage medium of distributed file system
CN110134738B (en) * 2019-05-21 2021-09-10 中国联合网络通信集团有限公司 Distributed storage system resource estimation method and device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
NTFS文件系统簇映射剖析;刁树民等;《佳木斯大学学报(自然科学版)》;20070315(第02期);全文 *
利用VB 4.0获取系统信息的程序设计;武雅丽等;《现代电子技术》;19980225(第02期);第8-9页 *

Also Published As

Publication number Publication date
CN102542040A (en) 2012-07-04

Similar Documents

Publication Publication Date Title
US9747127B1 (en) Worldwide distributed job and tasks computational model
CN103106152B (en) Based on the data dispatching method of level storage medium
KR101959153B1 (en) System for efficient processing of transaction requests related to an account in a database
CN110008257A (en) Data processing method, device, system, computer equipment and storage medium
CN105074702A (en) Database system providing single-tenant and multi-tenant environments
CN100452046C (en) Storage method and system for mass file
CN109240946A (en) The multi-level buffer method and terminal device of data
CN103399894A (en) Distributed transaction processing method on basis of shared storage pool
CN106775446A (en) Based on the distributed file system small documents access method that solid state hard disc accelerates
CN102937964B (en) Intelligent data service method based on distributed system
US11080207B2 (en) Caching framework for big-data engines in the cloud
CN105683928B (en) For the method for data cache policies, server and memory devices
US20120224482A1 (en) Credit feedback system for parallel data flow control
CN111737168A (en) Cache system, cache processing method, device, equipment and medium
CN104270412A (en) Three-level caching method based on Hadoop distributed file system
CN103886109A (en) Method and device for realizing row lock of database
CN105354250A (en) Data storage method and device for cloud storage
CN109767274B (en) Method and system for carrying out associated storage on massive invoice data
CN102542040B (en) Capacity acquiring method and system
CN103678562A (en) Capacity obtaining method and file data allocation method
CN101783814A (en) Metadata storing method for mass storage system
CN104391947B (en) Magnanimity GIS data real-time processing method and system
CN109299059A (en) File storage, search method, device, storage medium and server
CN107644020B (en) Data storage and calling method and device
CN103678563A (en) Capacity obtaining method and system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant