CN109960461A - A kind of mirror image file management system - Google Patents
A kind of mirror image file management system Download PDFInfo
- Publication number
- CN109960461A CN109960461A CN201711426456.4A CN201711426456A CN109960461A CN 109960461 A CN109960461 A CN 109960461A CN 201711426456 A CN201711426456 A CN 201711426456A CN 109960461 A CN109960461 A CN 109960461A
- Authority
- CN
- China
- Prior art keywords
- download
- mirror image
- image file
- control centre
- file management
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/065—Replication mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0653—Monitoring storage devices or systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a kind of mirror image file management systems, are made of control centre, Agent module and HDFS module；Control centre is mainly responsible for image file management, download schedule management, service logic control；Agent module is mainly responsible for the download protocol based on p2p and realizes that file download function, information are collected, management by district etc.；HDFS module is responsible for storing the full dose of image file；Control centre's control agents module and HDFS module.The present invention realizes the quick-downloading of image file by fragment and download schedule to image file.Elastic copy control based on temperature simultaneously realizes that the utilization rate of storage resource maximizes；The resilient expansion of cold and hot mirror image and cold and hot mirror image treatment process can accomplish without using lock and efficiently.
Description
Technical field
The present invention relates to a kind of electronic document management system more particularly to a kind of mirror image file management systems.
Background technique
User creates fictitious host computer and needs using mirror image in the usage scenario of public cloud, and furthermore user can also be by oneself
Fictitious host computer production be mirrored into and carry out preservation use.Current operating system supports the 60G system disk on basis, this external system
Support expansible customized system disk.But mirror image make after easily capacity have the size of tens G, so mirror image is literary
The management and scheduling of part are the key that realize space utilisation and creation speed.
Summary of the invention
In view of the above-mentioned problems, the present invention proposes a kind of mirror image file management system, by control centre (ICS), Agent module
It is formed with HDFS module；Control centre is mainly responsible for image file management, download schedule management, service logic control.
Agent module is mainly responsible for the download protocol based on p2p (peer-to-peer, P-2-P technology) and realizes under file
Function is carried, information is collected, management by district etc.；HDFS module is responsible for storing the full dose of image file；Control centre's control agents
Module and HDFS module.
Control centre receives the treatment process after the image download amount reported are as follows:
Set download average value standard；When the average value of the download of mirror image is greater than download average value standard max
When, increase a mirror back-up, when the average value of the download of mirror image is less than download average value standard min, reduces one
Mirror back-up.
Increase a hash data structure elastic_img, each of one each part of mirror image of data structure records is standby
The information such as the real-time download of part and mirror image are effective.
After treatment process includes the data that control centre receives the image download amount reported, the validity of mirror image is judged, more
The download of new part, and resilient expansion is determined whether to, if mirror image is invalid, the download information reported is ignored by control centre.
Treatment process includes when control centre receives deletion mirroring command, and control centre is first by mirror image from elastic_
It is set to invalid etc. receive in img and deletes the successful message of mirror image, then mirror image is deleted to simultaneously deletion of physically from elastic_img
Machine.
Treatment process includes the client value not updated in hostinfo, is returned and is downloaded according to download when creating virtual machine
The download of each backup is extracted when address from elastic_img.
Technical solution of the present invention realize the utility model has the advantages that
By the fragment and download schedule to image file, the quick-downloading of image file is realized.Simultaneously based on temperature
Elastic copy control realizes that the utilization rate of storage resource maximizes；The resilient expansion of cold and hot mirror image and cold and hot mirror image treatment process
Can accomplish without using lock and efficiently.
Detailed description of the invention
Fig. 1 is a kind of schematic diagram of mirror image file management system of the present invention.
Specific embodiment
In the following with reference to the drawings and specific embodiments, the present invention is further explained, and the embodiment of the present invention is only to illustrate this hair
The protection scope that bright technical solution is not intended to limit the present invention.
Such as Fig. 1, the present invention proposes a kind of mirror image file management system, by control centre, Agent module and HDFS module group
At；Control centre is mainly responsible for image file management, download schedule management, service logic control.
Agent module is mainly responsible for the download protocol based on p2p and realizes that file download function, information are collected, management by district
Deng；HDFS module is responsible for storing the full dose of image file；Control centre's control agents module and HDFS module.
Control centre receives the treatment process after the image download amount reported are as follows:
Set download average value standard；When the average value of the download of mirror image is greater than download average value standard max
When, increase a mirror back-up, when the average value of the download of mirror image is less than download average value standard min, reduces one
Mirror back-up.
Increase a hash data structure elastic_img, each of one each part of mirror image of data structure records is standby
The information such as the real-time download of part and mirror image are effective.
After treatment process includes the data that control centre receives the image download amount reported, the validity of mirror image is judged, more
The download of new part, and resilient expansion is determined whether to, if mirror image is invalid, the download information reported is ignored by control centre.
Treatment process includes when control centre receives deletion mirroring command, and control centre is first by mirror image from elastic_
It is invalid to be set in img, waits to receive and deletes the successful message of mirror image, then mirror image is deleted to simultaneously deletion of physically from elastic_img
Machine.
Treatment process includes the client value not updated in hostinfo, is returned and is downloaded according to download when creating virtual machine
The download of each backup is extracted when address from elastic_img.
Claims (6)
1. a kind of mirror image file management system, it is characterised in that be made of control centre, Agent module and HDFS module；
The control centre is responsible for image file management, download schedule management, service logic control；
The Agent module is responsible for the download protocol based on p2p and realizes that file download function, information are collected, management by district；
The HDFS module is responsible for storing the full dose of image file；
The control centre controls the Agent module and the HDFS module.
2. mirror image file management system according to claim 1, it is characterised in that the control centre receives the mirror reported
As the treatment process after download are as follows:
Set download average value standard；When the average value of the download of mirror image is greater than the download average value standard max
When, increase a mirror back-up；When the average value of the download of mirror image is less than the download average value standard min, reduce
One mirror back-up.
3. mirror image file management system according to claim 2, it is characterised in that the treatment process includes increasing by one
Hash data structure elastic_img, the real-time download of each backup of one each part of mirror image of the data structure records
And mirror image effective information.
4. mirror image file management system according to claim 2, it is characterised in that the treatment process includes the control
After center receives the data of the image download amount reported, the validity of the mirror image is judged, update the download of part, and judge
Whether resilient expansion is wanted, if the mirror image is invalid, the download information reported is ignored by the control centre.
5. mirror image file management system according to claim 2, it is characterised in that the treatment process includes when in control
The heart receives deletion mirroring command, and the mirror image is first set in vain from the elastic_img, waits to receive and delete by control centre
Except the successful message of mirror image, then the mirror image is deleted to simultaneously deletion of physically machine from the elastic_img.
6. mirror image file management system according to claim 2, it is characterised in that the treatment process includes not updating
Client value in hostinfo, when returning to download address according to download when creating virtual machine from the elastic_img
Extract the download of each backup.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711426456.4A CN109960461A (en) | 2017-12-25 | 2017-12-25 | A kind of mirror image file management system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711426456.4A CN109960461A (en) | 2017-12-25 | 2017-12-25 | A kind of mirror image file management system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109960461A true CN109960461A (en) | 2019-07-02 |
Family
ID=67021522
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711426456.4A Pending CN109960461A (en) | 2017-12-25 | 2017-12-25 | A kind of mirror image file management system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109960461A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090143145A1 (en) * | 2007-12-03 | 2009-06-04 | Microsoft Corporation | Read Redirection of Physical Media |
| JP2009217859A (en) * | 2008-03-07 | 2009-09-24 | Lenovo Singapore Pte Ltd | Reproducing method without interruption of medium file |
| CN102487401A (en) * | 2010-12-06 | 2012-06-06 | 腾讯科技（深圳）有限公司 | File download method and apparatus thereof |
| CN104424118A (en) * | 2013-08-23 | 2015-03-18 | 深圳市腾讯计算机系统有限公司 | Hotspot file self-adaption copy method and system |
| CN105975308A (en) * | 2016-05-04 | 2016-09-28 | 烽火通信科技股份有限公司 | Remote upgrading system for low-memory overheads in home gateways and remote upgrading method |
| CN107317828A (en) * | 2016-04-26 | 2017-11-03 | 腾讯科技（深圳）有限公司 | Document down loading method and device |
| CN107426258A (en) * | 2016-05-23 | 2017-12-01 | 华为技术有限公司 | A kind of upload of image file and method for down loading and device |
-
2017
- 2017-12-25 CN CN201711426456.4A patent/CN109960461A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090143145A1 (en) * | 2007-12-03 | 2009-06-04 | Microsoft Corporation | Read Redirection of Physical Media |
| JP2009217859A (en) * | 2008-03-07 | 2009-09-24 | Lenovo Singapore Pte Ltd | Reproducing method without interruption of medium file |
| CN102487401A (en) * | 2010-12-06 | 2012-06-06 | 腾讯科技（深圳）有限公司 | File download method and apparatus thereof |
| CN104424118A (en) * | 2013-08-23 | 2015-03-18 | 深圳市腾讯计算机系统有限公司 | Hotspot file self-adaption copy method and system |
| CN107317828A (en) * | 2016-04-26 | 2017-11-03 | 腾讯科技（深圳）有限公司 | Document down loading method and device |
| CN105975308A (en) * | 2016-05-04 | 2016-09-28 | 烽火通信科技股份有限公司 | Remote upgrading system for low-memory overheads in home gateways and remote upgrading method |
| CN107426258A (en) * | 2016-05-23 | 2017-12-01 | 华为技术有限公司 | A kind of upload of image file and method for down loading and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104715001B (en) | The method and system of write operation is performed for the shared resource in the cluster to data handling system | |
| CN102594849B (en) | Data backup and recovery method and device, virtual machine snapshot deleting and rollback method and device | |
| Rao et al. | Performance issues of heterogeneous hadoop clusters in cloud computing | |
| CN102955845B (en) | Data access method, device and distributed data base system | |
| CN102299904B (en) | System and method for realizing service data backup | |
| US20140081911A1 (en) | Optimizing automatic deletion of backup files | |
| CN103970585A (en) | Method and device for creating virtual machine | |
| CN102185928A (en) | Method for creating virtual machine in cloud computing system and cloud computing system | |
| CN103544045A (en) | HDFS-based virtual machine image storage system and construction method thereof | |
| CN106775446A (en) | Based on the distributed file system small documents access method that solid state hard disc accelerates | |
| CN102508886B (en) | Extensive makeup language (XML)-based method for synchronously updating increment of spatial data | |
| CN104156396B (en) | The method and system of database performance are improved under big data environment | |
| US20120278429A1 (en) | Cluster system, synchronization controlling method, server, and synchronization controlling program | |
| CN105069151A (en) | HBase secondary index construction apparatus and method | |
| CN105302922B (en) | A kind of distributed file system snapshot implementing method | |
| CN103902410A (en) | Data backup acceleration method for cloud storage system | |
| CN102143228A (en) | Cloud storage system, cloud client and method for realizing storage area network service | |
| CN103763368A (en) | Cross-data-center data synchronism method | |
| CN110083306A (en) | A kind of distributed objects storage system and storage method | |
| JP2004227169A (en) | Distributed database system | |
| George et al. | Hadoop MapReduce for mobile clouds | |
| CN105407044A (en) | Method for implementing cloud storage gateway system based on network file system (NFS) | |
| CN103546574B (en) | A kind of airborne built-in network file access control method based on static configuration table | |
| CN108259569A (en) | It is a kind of based on IPSAN share storage without acting on behalf of continuous data protection method | |
| CN102820998A (en) | Dual-fault-tolerant service system applicable to office applications and data storage method of dual-fault-tolerant service system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190702 |
|
| WD01 | Invention patent application deemed withdrawn after publication |