CN103377069A - Novel method for achieving virtual calculation deployment based on writing of log files into plug-in network attached storage (NAS) - Google Patents
Novel method for achieving virtual calculation deployment based on writing of log files into plug-in network attached storage (NAS) Download PDFInfo
- Publication number
- CN103377069A CN103377069A CN2012101264135A CN201210126413A CN103377069A CN 103377069 A CN103377069 A CN 103377069A CN 2012101264135 A CN2012101264135 A CN 2012101264135A CN 201210126413 A CN201210126413 A CN 201210126413A CN 103377069 A CN103377069 A CN 103377069A
- Authority
- CN
- China
- Prior art keywords
- nas
- client computer
- space
- log files
- virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention provides a novel method for achieving virtual calculation deployment based on writing of log files into a plug-in network attached storage (NAS). During virtual deployment, apart from distribution of normal disk space for installing an operation system and application programs, a client is required to load an NAS sharing catalogue uniformly to store the log files, application software can write generated log files into the NAS sharing catalogue directly and conveniently, large-capacity storage resource of the NAS is used as shared storage space of a plurality of applications, the client only needs a small amount of disk space for installing the operation system and necessary application software, occupied disk spaces can not be increased basically after the system is installed and configured, and the problem that the log data occupies virtual disk entity file space is solved. Meanwhile, the log files are stored in the NAS sharing catalogue, so that analyzing and processing of the log files do not occupy the performance of the client itself.
Description
Technical field
The present invention relates to a kind of Log-based file and write the new method that plug-in NAS realizes that virtual calculating is disposed, belong to areas of information technology, particularly virtual calculating field.
Background technology
Virtual calculating is the infotech of present main flow, by high performance generic server (such as based on Intel Xeon to strong or AMD Opteron multi-core CPU) the deploy virtualization software, can separate unit server deploy several to tens computing environment, mutually isolation between each computing environment, for the user, each computing environment is an independent computing machine, can be configured, restarts, change and meet and do not affect other computing environment.Like this so that the infosystem of enterprises is disposed required host number greatly reduces.Shown in Figure 1 is the general framework of virtual calculating.
At present enterprises affix one's name to virtual main flow be use VMWare company (
Www.vmware.com) Related product.In the said firm's product, dispose virtualized server and be called main frame (being also referred to as host in the relevant documentation), be called client computer (relevant documentation is also referred to as guest) for the single computing environment of carrying out concrete calculation task.
Under virtual mode, each new client computer also needs to dispose independent CPU, internal memory, hard disk, network interface card resource.Based on the difference of calculation task that each client computer is moved, need to be that this client computer is reserved certain hard drive space like this, so that installing operating system, application program, journal file etc.Each client computer will generate a group object file at main frame, comprise the configuration of these CPU, internal memory, hard disk, network interface card, the hard disk of client computer also will be presented as a virtual disk entity file (generally being named as vmdk in the VMWare system), and its big or small and client computer is distributed hard drive space in the same size.
After deployment is virtual, because every main frame can produce a large amount of daily records and (comprise all kinds of daily records of system itself in operational process, the daily record that produces with application software), so that after virtual the disk of each client computer be easy to be taken by journal file; And in the service operation process, these journal files are read, when analyzing, also need to access the virtual disk entity file of each client computer on main frame, so also can cause the disk read-write pressure of main frame.
At present when virtual deployment, general direct-connected storage (the Direct Attached Storage that adopts, be abbreviated as DAS), storage area network equipment (Storage Area Network, be abbreviated as SAN), network additive storage device (Network Attached Storage is abbreviated as NAS) deposits the used virtual disk entity file of client computer.After having moved tens client computer at virtualized main frame, will strengthen for free space, the disk read-write pressure of virtual disk entity file, especially constantly move and produce the application software of a large amount of daily record datas.
Under this deployment way, during the journal file of keeper, operating personnel's access application, will frequently login client computer, the management maintenance inefficiency.
Therefore the present invention is directed to this problem, proposed a kind of Log-based file and write the new method that plug-in NAS realizes that virtual calculating is disposed, when disposing, can effectively reduce disk space with the application software that this class is produced a large amount of daily record datas and take and improve runnability.
Summary of the invention
The invention provides and a kind ofly come the dispositions method of storing daily record file based on NAS network share directory, realize that each application software when carrying out virtual deployment, effectively addresses the above problem.
Gordian technique of the present invention is, when disposing virtualized client computer, except distributing normal disk space to be used for installing operating system, outside the application program, require the share directory of the unified NAS of loading of client computer network additive storage device to be used for the catalogue that journal file is deposited, so that application software directly writes to the journal file that generates in the NAS share directory, client computer only needs a small amount of disk space to be used for installing operating system like this, necessary application software, shared disk space is installed in system, after finishing, configuration basically can not increase, large capacity storage resource by the NAS network storage equipment is as the shared storage space of a plurality of application, so that solve daily record data to the space hold problem of virtual disk entity file; Journal file leaves on the NAS share directory simultaneously, to the analysis of journal file, the performance that processing does not take client computer self.
The invention has the beneficial effects as follows, avoided when virtual deployment that too much to cause virtual disk entity file corresponding to client computer to allocate the space in advance excessive or consume too fast problem owing to the journal file of application software produces; Simultaneously the journal file unification is left on the external NAS, can to the reading, analyze, process and to sign in on the client computer of journal file, reduce the resource occupation to client computer, improve journal file management maintenance efficient.
Description of drawings
Fig. 1 is the general Organization Chart of virtual calculating.
Fig. 2 is the traditional virtual deployment synoptic diagram of virtual disk entity file when leaving the SAN disk array in.
Fig. 3 is that Log-based file provided by the invention writes the Organization Chart that plug-in NAS realizes that virtual calculating is disposed.
Embodiment
Below in conjunction with accompanying drawing and for example, describe the specific embodiment of the present invention in detail.
When actual deployment, generally adopt DAS, SAN or NAS to deposit the virtual disk entity file of client computer.Accompanying drawing 2 is to leave the SAN disk array in as example take the virtual disk entity file.
In this case, need to divide in advance amount of capacity for the virtual disk entity file of each client computer.If it is excessive, too fast that the application software of installing on this client computer produces journal file, the capacity of dividing so in advance is easy to be fully used, and can cause the Insufficient disk space of client computer like this and affects the operation of this application software; If the journal file generation of this application software is less, the capacity of dividing so in advance is then excessive again, causes the waste of disk space.
Even the application system that client computer is disposed can realize that journal file is pressed the time section, resettlement is shared storage to NAS according to plan, because this operates general operation every day for several times, too short so that the log analysis of slice spacings, troubleshooting need to read a plurality of files simultaneously, and slice spacings is then long so that individual log file is excessive.
Simultaneously after certain application software is disposed, there are tens, during up to a hundred client computer, if need to read journal file analysis, troubleshooting, need so to login simultaneously a plurality of client computer.This external CPU own, disk resource that reads, also needs to take client computer when analyzing journal file, if journal file is excessive, also can affect the performance of client computer, surpassing 100MB such as size can affect the disk IO read-write performance, if journal file surpasses the 1GB size then needs just can open for a long time.
Therefore the present invention is directed to both of these case, a kind of dispositions method that comes the storing daily record file based on NAS network share directory has been proposed, so that on the virtual deployment mode basis of routine, minimizing is to the space hold of the virtual disk entity file of client computer, to the access of client computer, realize more easily the processing to journal file in the minimizing operational process.
Implementation method of the present invention is, to each client computer when disposing, except distributing normal disk space to be used for installing operating system, outside the application program, require the share directory of the unified NAS of loading of client computer network additive storage device to be used for the catalogue that journal file is deposited, by application software the journal file that generates is directly write in the NAS share directory, large capacity storage resource by the NAS network storage equipment is as the shared storage space of a plurality of application, client computer only needs a small amount of disk space to be used for installing operating system, necessary application software, shared disk space is installed in system, after finishing, configuration basically can not increase, so that solve daily record data to the space hold problem of virtual disk entity file.Journal file leaves on the NAS share directory simultaneously, to the analysis of journal file, the performance that processing does not take client computer self.
A kind of prioritization scheme synoptic diagram that shown in Figure 3 is for traditional virtual deployment shown in Figure 2, the daily record of application program will directly write in the NAS network storage equipment like this.For client computer and application program, the Log Directory size that it recognizes is exactly the total volume of the NAS network storage equipment, can not write full problem by the occurrence log file.
For example certain virtual based on being applied in of disposing of Linux after, the virtual disk entity file space that is assumed to its distribution is 32GB, so generally carries out disk allocation by following rule:
Installing operating system under/root directory is wherein installed application software under/app catalogue, deposit all kinds of journal files that application software produces under/logs catalogue.
In this example, the directory size of distributing for application log is 16GB, because the daily record data generation is too much, excessive, it is excessive to cause this catalogue to consume, if untimely the cleaning is easy to so cause the system disk insufficient space, and causes the abnormal running of this application.
Therefore use the Log-based file to write the new method that plug-in NAS realizes that virtual calculating is disposed, journal file can be directly write on the external unit, can effectively avoid journal file taking virtual disk entity file space.
For a top example, the distribution of its disk space can be optimized as follows:
So just so that the application software of each client computer can be placed on journal file on the plug-in NAS, realize the centralized management of journal file, the disk space that reduces single client computer takies.
Under this deployment way, during the journal file of keeper, operating personnel's access application, only need the directly unified NAS network storage of access, the situation that can effectively avoid frequently logining client computer and affect application, simultaneously also reduce the trouble that the user need to login many client computer, be conducive to improve deployment, maintenance efficiency.
Claims (1)
1. a Log-based file writes the new method that plug-in NAS realizes that virtual calculating is disposed, it is characterized in that: when disposing virtualized client computer, except distributing normal disk space to be used for installing operating system, outside the application program, require the share directory of the unified NAS of loading of client computer network additive storage device to be used for the catalogue that journal file is deposited, so that application software directly writes to the journal file that generates in the NAS share directory, large capacity storage resource by the NAS network storage equipment is as the shared storage space of a plurality of application, so that solve daily record data to the space hold problem of virtual disk entity file; Journal file leaves on the NAS share directory simultaneously, to the analysis of journal file, the performance that processing does not take client computer self.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012101264135A CN103377069A (en) | 2012-04-26 | 2012-04-26 | Novel method for achieving virtual calculation deployment based on writing of log files into plug-in network attached storage (NAS) |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012101264135A CN103377069A (en) | 2012-04-26 | 2012-04-26 | Novel method for achieving virtual calculation deployment based on writing of log files into plug-in network attached storage (NAS) |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103377069A true CN103377069A (en) | 2013-10-30 |
Family
ID=49462230
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012101264135A Pending CN103377069A (en) | 2012-04-26 | 2012-04-26 | Novel method for achieving virtual calculation deployment based on writing of log files into plug-in network attached storage (NAS) |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103377069A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103856354A (en) * | 2014-03-07 | 2014-06-11 | 浪潮电子信息产业股份有限公司 | Method for achieving unified management of logs of cluster storage system |
| CN105095641A (en) * | 2015-05-29 | 2015-11-25 | 努比亚技术有限公司 | Space capacity calculation method and apparatus |
| CN105183868A (en) * | 2015-09-15 | 2015-12-23 | 浪潮(北京)电子信息产业有限公司 | Storage method and device for logs of file system |
| CN105224256A (en) * | 2015-10-13 | 2016-01-06 | 浪潮(北京)电子信息产业有限公司 | A kind of storage system |
| CN109460390A (en) * | 2018-11-14 | 2019-03-12 | 郑州云海信息技术有限公司 | A kind of log storing method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040030822A1 (en) * | 2002-08-09 | 2004-02-12 | Vijayan Rajan | Storage virtualization by layering virtual disk objects on a file system |
| US20050027785A1 (en) * | 2003-07-28 | 2005-02-03 | Erol Bozak | Maintainable grid managers |
| US20070156842A1 (en) * | 2005-12-29 | 2007-07-05 | Vermeulen Allan H | Distributed storage system with web services client interface |
| CN101808139A (en) * | 2010-03-31 | 2010-08-18 | 重庆索伦互联网信息服务有限公司 | Data storage system in cloud environment |
| CN101873336A (en) * | 2009-04-27 | 2010-10-27 | 上海融越信息技术有限公司 | NAS (Network Attached Storage) virtual file system based on storage device |
| CN102158546A (en) * | 2011-02-28 | 2011-08-17 | 中国科学院计算技术研究所 | Cluster file system and file service method thereof |
-
2012
- 2012-04-26 CN CN2012101264135A patent/CN103377069A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040030822A1 (en) * | 2002-08-09 | 2004-02-12 | Vijayan Rajan | Storage virtualization by layering virtual disk objects on a file system |
| US20050027785A1 (en) * | 2003-07-28 | 2005-02-03 | Erol Bozak | Maintainable grid managers |
| US20070156842A1 (en) * | 2005-12-29 | 2007-07-05 | Vermeulen Allan H | Distributed storage system with web services client interface |
| CN101873336A (en) * | 2009-04-27 | 2010-10-27 | 上海融越信息技术有限公司 | NAS (Network Attached Storage) virtual file system based on storage device |
| CN101808139A (en) * | 2010-03-31 | 2010-08-18 | 重庆索伦互联网信息服务有限公司 | Data storage system in cloud environment |
| CN102158546A (en) * | 2011-02-28 | 2011-08-17 | 中国科学院计算技术研究所 | Cluster file system and file service method thereof |
Non-Patent Citations (1)
| Title |
|---|
| 孙社文: "网络存储主流技术分析", 《北京工业职业技术学院学报》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103856354A (en) * | 2014-03-07 | 2014-06-11 | 浪潮电子信息产业股份有限公司 | Method for achieving unified management of logs of cluster storage system |
| CN105095641A (en) * | 2015-05-29 | 2015-11-25 | 努比亚技术有限公司 | Space capacity calculation method and apparatus |
| CN105095641B (en) * | 2015-05-29 | 2018-04-03 | 努比亚技术有限公司 | A kind of method and apparatus for calculating spatial content |
| CN105183868A (en) * | 2015-09-15 | 2015-12-23 | 浪潮(北京)电子信息产业有限公司 | Storage method and device for logs of file system |
| CN105224256A (en) * | 2015-10-13 | 2016-01-06 | 浪潮(北京)电子信息产业有限公司 | A kind of storage system |
| CN109460390A (en) * | 2018-11-14 | 2019-03-12 | 郑州云海信息技术有限公司 | A kind of log storing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9569457B2 (en) | Data processing method and apparatus for distributed systems | |
| Rao et al. | Performance issues of heterogeneous hadoop clusters in cloud computing | |
| Moon et al. | Introducing ssds to the hadoop mapreduce framework | |
| Prabhakar et al. | Provisioning a multi-tiered data staging area for extreme-scale machines | |
| Zhang et al. | iBridge: Improving unaligned parallel file access with solid-state drives | |
| CN104272386A (en) | Reducing power consumption by migration of data within tiered storage system | |
| CN103377069A (en) | Novel method for achieving virtual calculation deployment based on writing of log files into plug-in network attached storage (NAS) | |
| CN103176845A (en) | Method, system and device for virtual machine arrangement | |
| US20140215127A1 (en) | Apparatus, system, and method for adaptive intent logging | |
| Freitas et al. | GPFS scans 10 billion files in 43 minutes | |
| Costan et al. | TomusBlobs: scalable data‐intensive processing on Azure clouds | |
| Bayati et al. | Exploring benefits of nvme ssds for bigdata processing in enterprise data centers | |
| Kibe et al. | Proposal for improving throughput in supersaturated cloud | |
| Zhu et al. | Swapx: An nvm-based hierarchical swapping framework | |
| Ruan et al. | Improving Shuffle I/O performance for big data processing using hybrid storage | |
| Lee et al. | Introspection of virtual machine memory resource in the virtualized systems | |
| Deshmukh et al. | Virtual Memory Optimization Techniques in Cloud Computing | |
| Kang et al. | A hybrid storage access framework for high-performance virtual machines | |
| Nakashima et al. | File Placing Control for Improving the I/O Performance of Hadoop in Virtualized Environment | |
| Tong et al. | Experiences in Managing the Performance and Reliability of a {Large-Scale} Genomics Cloud Platform | |
| Wang et al. | Evaluating non-in-place update techniques for flash-based transaction processing systems | |
| CN103901962A (en) | Large-capacity storage system equipment | |
| Kon et al. | Highly consolidated servers with container-based virtualization | |
| Zhang et al. | An efficient cloud computing-based architecture for freight system application in china railway | |
| Lee et al. | Optimizing back-and-forth live migration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20131030 |
|
| WD01 | Invention patent application deemed withdrawn after publication |