CN110825703B - Method for realizing elastic expansion and contraction of file system based on timing task - Google Patents
Method for realizing elastic expansion and contraction of file system based on timing task Download PDFInfo
- Publication number
- CN110825703B CN110825703B CN201911057998.8A CN201911057998A CN110825703B CN 110825703 B CN110825703 B CN 110825703B CN 201911057998 A CN201911057998 A CN 201911057998A CN 110825703 B CN110825703 B CN 110825703B
- Authority
- CN
- China
- Prior art keywords
- file system
- capacity
- expansion
- user
- screening
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1727—Details of free space management performed by the file system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1737—Details of further file system functions for reducing power consumption or coping with limited storage space, e.g. in mobile devices
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention provides a method for realizing elastic expansion and contraction of a file system based on a timed task, which belongs to the technical field of information, and is characterized in that firstly, some related information of the file system in a VM (virtual machine) is acquired through an acquisition method, the information is compared with data of the file system stored in an openstack, the file system meeting the requirement is screened out (screening is carried out by necessarily having a mount point for each file system), and finally, shareId (file system Id) is grouped and screened according to the grouping whether the screened file system needs capacity expansion, capacity reduction and a charging mode, so that the file system which really needs capacity expansion or capacity reduction can be obtained.
Description
Technical Field
The invention relates to the technical field of information, in particular to a method for realizing elastic expansion and contraction of a file system based on a timing task.
Background
Cloud storage is a mode of online storage (english: cloud storage), i.e., data is stored on multiple virtual servers, usually hosted by third parties, rather than on dedicated servers. Hosting companies operate large data centers, and people who need data storage hosting meet the data storage requirements by buying or leasing storage space. The data center operator prepares the storage virtualized resources at the back end according to the needs of the customer, and provides the resources in a storage resource pool (storage pool), so that the customer can use the storage resource pool to store the files or objects by himself. In practice, these resources may be distributed over numerous server hosts.
The concept of cloud storage is similar to that of cloud computing, and refers to a system which integrates a large number of storage devices of different types in a network through application software to cooperatively work through functions such as cluster application, a grid technology or a distributed file system, and provides data storage and service access functions to the outside, so that the safety of data is ensured, and the storage space is saved. Briefly, cloud storage is an emerging solution for putting storage resources on the cloud for human access. The user can conveniently access data at any time and any place through connecting to the cloud through any internet-connected device.
With the gradual maturity of cloud storage, how to efficiently utilize cloud space to reduce cost is also a main problem gradually considered by various manufacturers, when the cloud space is just abundant, a charging mode mainly covers the year and the month, which causes a lot of tenants to purchase a large space, but the space utilization rate is not high, users do not know the space needed by themselves, the space is not enough when the users buy less space, and waste is caused, and after the cloud space is reached, if the users want to keep data continuously, but the data is not much, the tenants also have to pay the same price, so another mode, payment as needed, gradually appears, and elastic expansion and contraction also becomes a main support for realizing the mode.
File storage (file system) is also a type of cloud space, unlike block storage and object storage: the process of file storage requires support of a virtual machine, and some differences exist in implementation.
Disclosure of Invention
In order to solve the technical problems, the invention provides a method for realizing elastic expansion and contraction of a file system based on a timing task, the elastic expansion and contraction of the file system can be carried out through the timing task, a user selects a proper initial file system size to purchase, when the space utilization of the file system is very small, the timing task can correspondingly contract and expand the file system space of the user by acquiring information and carrying out corresponding processing, and similarly, when the space used by the user is about to be exhausted, the timing task can also correspondingly expand the file system space of the user, so that the utilization rate of the cloud space can reach a higher level.
The technical scheme of the invention is as follows:
a method for achieving elastic expansion and contraction of a file system based on a timing task is mainly used for achieving comparison and screening layer by layer, firstly, some related information of the file system existing in a VM is obtained through an acquisition method, the information is compared with data of the file system stored in an openstack, the file system meeting requirements is screened out (screening is achieved through the fact that each file system must have a mount point), finally, shareId is grouped, screening is conducted according to grouping to the fact whether the file system screened out in the past needs capacity expansion, capacity reduction and a charging mode, and finally the file system which really needs capacity expansion or capacity reduction can be achieved.
Further, in the above-mentioned case,
there must be one mount point to filter through each file system.
And further, acquiring the usage amount, the total capacity and the mounting point of the file system on the server during the timing task.
In a still further aspect of the present invention,
acquiring data of a file system through an openstack interface, matching mounting points with the mounting points, finding out a file system meeting requirements, judging whether the capacity expansion and the capacity reduction are needed or not according to the usage and the total capacity of the file system meeting the requirements, and storing the file system id (shareId), the user id (user id), the total capacity (size) and the usage (usedSzie) of the file system meeting the requirements in a list.
In a still further aspect of the present invention,
classifying the obtained list according to the userId, storing the list in a new list, carrying out url assembly on all shareIDs of each user, calling an inquiry interface of the bss, and judging whether the obtained file system payment type is the on-demand payment or not through traversal.
Further, in the above-mentioned case,
the interface supports querying all file system details at once.
In a still further aspect of the present invention,
and carrying out specific capacity expansion and capacity reduction judgment on the file system meeting the requirements, transmitting the parameters shareId, userId, token and the changed capacity size to a specific capacity expansion and capacity storage method, and comparing the introduced changed capacity size with the quota of the user.
If the volume size after the incoming change exceeds the quota, the capacity expansion will fail;
if the incoming changed volume size is smaller than the initial quota, the capacity reduction will fail.
The invention has the advantages that
By the method and the system, manpower waste for managing the capacity of the file system by the user can be reduced, a lot of funds can be effectively saved for the user, and simultaneously, an operator can also use all cloud space of the operator more efficiently, so that waste of resources is avoided.
Drawings
FIG. 1 is a schematic of the work flow of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention, and based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts belong to the scope of the present invention.
The invention realizes the elastic expansion and contraction of the file system based on the timing task, adds the acquisition task for acquiring the information of the file system into the timing task, correspondingly screens and judges the result of acquiring the file system by the timing task according to the result of the acquisition task, and finally correspondingly expands or contracts the selected file system which meets the requirement.
The method comprises the following specific steps:
step 1: a timing task is started.
Step 2: the space of the current disk and the use condition of the space are collected through a command of 'df-h', and the collected result is stored.
And 3, step 3: acquiring detailed information of all file systems through an openstack interface, matching mounting points in the acquired information with mounting points in the step 2 through two times of traversal (namely, one mounting point only has one file system), performing corresponding processing on data in the step 2, then performing capacity expansion and capacity reduction standard judgment on the file systems matched with the mounting points, and if the file systems meet the requirements, performing list packaging on some attributes of the file systems meeting the requirements.
And 4, step 4: and (4) screening the file systems obtained in the step (3), packaging the file systems in groups according to the user id, and after the grouping is finished, packaging all the file system ids of each user in url.
And 5: and screening the payment modes of the conforming file systems, selecting the file systems according to the requirements, judging the capacity expansion and the capacity reduction, packaging the capacity expansion or capacity reduction results, and calling a capacity expansion and capacity reduction interface to perform the capacity expansion and the capacity reduction.
The above description is only a preferred embodiment of the present invention, and is only used to illustrate the technical solutions of the present invention, and not to limit the protection scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (4)
1. A method for realizing elastic expansion and contraction of a file system based on a timing task is characterized in that,
after the comparison and screening layer by layer is completed, firstly, acquiring relevant information of a file system in a VM (virtual machine) by an acquisition method, comparing the information with data of the file system stored in an openstack, screening out the file system which meets the requirement, and finally, screening whether the file system which is screened out before needs capacity expansion, capacity reduction and a charging mode according to the grouping by grouping the Id-shared lines of the file system so as to finally obtain the file system which really needs capacity expansion or capacity reduction;
screening by having to have one mount point per file system;
acquiring the use amount, the total capacity and the mounting point of a file system on a server when a timing task is carried out;
acquiring data of a file system through an openstack interface, matching mounting points with the mounting points, finding out a file system meeting requirements, judging whether the file system meeting the requirements needs capacity expansion and capacity reduction according to the use amount and the total capacity acquired by the mounting points, and storing the file system I d-shareID, the user id-user Id, the total capacity size and the use amount usedSzie of the file system meeting the requirements in a list;
classifying the obtained list according to the userId, storing the list in a new list, carrying out url assembly on all shareIDs of each user, calling an inquiry interface of the bss, and judging whether the obtained file system payment type is the on-demand payment or not through traversal.
2. The method of claim 1,
the interface supports querying all file system details at once.
3. The method of claim 1,
and carrying out specific capacity expansion and capacity reduction judgment on the file system meeting the requirements, transmitting the parameters shareId, userId, token and the changed capacity size to a specific capacity expansion and capacity storage method, and comparing the introduced changed capacity size with the quota of the user.
4. The method of claim 3,
if the volume size after the incoming change exceeds the quota, the capacity expansion will fail;
if the incoming changed volume size is smaller than the initial quota, the capacity reduction will fail.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911057998.8A CN110825703B (en) | 2019-11-01 | 2019-11-01 | Method for realizing elastic expansion and contraction of file system based on timing task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911057998.8A CN110825703B (en) | 2019-11-01 | 2019-11-01 | Method for realizing elastic expansion and contraction of file system based on timing task |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110825703A CN110825703A (en) | 2020-02-21 |
| CN110825703B true CN110825703B (en) | 2023-04-11 |
Family
ID=69552187
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911057998.8A Active CN110825703B (en) | 2019-11-01 | 2019-11-01 | Method for realizing elastic expansion and contraction of file system based on timing task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110825703B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101516046A (en) * | 2009-03-18 | 2009-08-26 | 深圳市讯达康通讯设备有限公司 | Transmission method of Ethernet passive optical network |
| WO2011121746A1 (en) * | 2010-03-31 | 2011-10-06 | 株式会社日立ソリューションズ | File server apparatus, method of controlling storage system, and program |
| CN102681899A (en) * | 2011-03-14 | 2012-09-19 | 金剑 | Virtual computing resource dynamic management system of cloud computing service platform |
| CN103533081A (en) * | 2013-10-25 | 2014-01-22 | 从兴技术有限公司 | Cloud-computing-based billing system and implementation method thereof |
| CN105786985A (en) * | 2016-02-15 | 2016-07-20 | 浪潮(北京)电子信息产业有限公司 | Automatic method and device for expanding file systems automatically |
| CN105955662A (en) * | 2016-04-22 | 2016-09-21 | 浪潮(北京)电子信息产业有限公司 | Method and system for expansion of K-DB data table space |
| CN106383671A (en) * | 2016-09-29 | 2017-02-08 | 乐视控股（北京）有限公司 | Block device storage cluster capability expansion system and method |
| CN106815298A (en) * | 2016-12-09 | 2017-06-09 | 中电科华云信息技术有限公司 | Distributed sharing file system based on block storage |
| CN110147915A (en) * | 2018-02-11 | 2019-08-20 | 陕西爱尚物联科技有限公司 | A kind of method and its system of resource distribution |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105468989B (en) * | 2014-09-04 | 2018-10-12 | 中国石油化工股份有限公司 | Cloud storage quota management method based on linux kernel monitoring |
| CN105740165A (en) * | 2014-12-10 | 2016-07-06 | 国际商业机器公司 | Method and apparatus for managing file system of unified storage system |
| CN104539708B (en) * | 2014-12-29 | 2018-01-23 | 杭州华为数字技术有限公司 | A kind of capacity reduction method, device and the system of cloud platform resource |
| CN107800730B (en) * | 2016-08-30 | 2021-01-29 | 阿里巴巴集团控股有限公司 | Capacity expansion method and device for virtual disk |
| CN109144663B (en) * | 2018-07-13 | 2020-11-10 | 新华三云计算技术有限公司 | Host computer optimization method and device |
-
2019
- 2019-11-01 CN CN201911057998.8A patent/CN110825703B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101516046A (en) * | 2009-03-18 | 2009-08-26 | 深圳市讯达康通讯设备有限公司 | Transmission method of Ethernet passive optical network |
| WO2011121746A1 (en) * | 2010-03-31 | 2011-10-06 | 株式会社日立ソリューションズ | File server apparatus, method of controlling storage system, and program |
| CN102681899A (en) * | 2011-03-14 | 2012-09-19 | 金剑 | Virtual computing resource dynamic management system of cloud computing service platform |
| CN103533081A (en) * | 2013-10-25 | 2014-01-22 | 从兴技术有限公司 | Cloud-computing-based billing system and implementation method thereof |
| CN105786985A (en) * | 2016-02-15 | 2016-07-20 | 浪潮(北京)电子信息产业有限公司 | Automatic method and device for expanding file systems automatically |
| CN105955662A (en) * | 2016-04-22 | 2016-09-21 | 浪潮(北京)电子信息产业有限公司 | Method and system for expansion of K-DB data table space |
| CN106383671A (en) * | 2016-09-29 | 2017-02-08 | 乐视控股（北京）有限公司 | Block device storage cluster capability expansion system and method |
| CN106815298A (en) * | 2016-12-09 | 2017-06-09 | 中电科华云信息技术有限公司 | Distributed sharing file system based on block storage |
| CN110147915A (en) * | 2018-02-11 | 2019-08-20 | 陕西爱尚物联科技有限公司 | A kind of method and its system of resource distribution |
Non-Patent Citations (1)
| Title |
|---|
| 数字图书馆应用云存储长期保存资源模式与策略研究;王春叶;《农业图书情报》;20190711;第75-82页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110825703A (en) | 2020-02-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9742652B2 (en) | Proactive identification of hotspots in a cloud computing environment | |
| CN110276182B (en) | API distributed current limiting realization method | |
| CN105103506B (en) | For the method and system for the non-homogeneous bandwidth request allocation bandwidth in system for cloud computing | |
| CN111880917A (en) | Edge mixed cloud pipe platform | |
| US20140201362A1 (en) | Real-time data analysis for resource provisioning among systems in a networked computing environment | |
| US9148426B2 (en) | Securely identifying host systems | |
| US20130268940A1 (en) | Automating workload virtualization | |
| CN107181808A (en) | A kind of privately owned cloud system and operation method | |
| WO2020192649A1 (en) | Data center management system | |
| US20130262250A1 (en) | Allocation of Application Licenses within Cloud or Infrastructure | |
| US20120323821A1 (en) | Methods for billing for data storage in a tiered data storage system | |
| CN107888660B (en) | Cloud service resource allocation method, medium, device and computing equipment | |
| US20120331521A1 (en) | System and method for application centric cloud management | |
| US10152449B1 (en) | User-defined capacity reservation pools for network-accessible resources | |
| US20170289354A1 (en) | System and Method for Allocation And Management Of Shared Virtual Numbers | |
| US8548881B1 (en) | Credit optimization to minimize latency | |
| Petrovska et al. | Features of the distribution of computing resources in cloud systems | |
| CN109582439A (en) | DCN dispositions method, device, equipment and computer readable storage medium | |
| Islam et al. | A comparative study of major service providers for cloud computing | |
| CN110825703B (en) | Method for realizing elastic expansion and contraction of file system based on timing task | |
| CN112422325A (en) | Management method and system based on multi-cloud mixed platform | |
| CN111314234A (en) | Flow distribution method and device, storage medium and electronic equipment | |
| US20220030079A1 (en) | Methods and systems for recording user operations on a cloud management platform | |
| CN111708677B (en) | Cloud hard disk usage amount acquisition method in cloud computing environment | |
| US9680701B2 (en) | Differentiating image files in a networked computing environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 250100 S06 tower, 1036, Chao Lu Road, hi tech Zone, Ji'nan, Shandong. Applicant after: Inspur cloud Information Technology Co.,Ltd. Address before: 250100 S06 tower, 1036, Chao Lu Road, hi tech Zone, Ji'nan, Shandong. Applicant before: Tidal Cloud Information Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |