CN103605630A - Virtual server system and data reading-writing method thereof - Google Patents
Virtual server system and data reading-writing method thereof Download PDFInfo
- Publication number
- CN103605630A CN103605630A CN201310628083.4A CN201310628083A CN103605630A CN 103605630 A CN103605630 A CN 103605630A CN 201310628083 A CN201310628083 A CN 201310628083A CN 103605630 A CN103605630 A CN 103605630A
- Authority
- CN
- China
- Prior art keywords
- virtual
- glusterfs
- write
- data
- virtual machine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a virtual server system and a data reading-writing method thereof. The virtual server system comprises a virtualization server and a Glusterfs server terminal, the virtualization server comprises a Glusterfs client terminal module, a virtual log generating module is additionally arranged on the Glusterfs client terminal module, and the virtual log generating module is configured with a corresponding virtual log cache region. According to the virtual server system, the virtual log generating module receives random reading/writing requests of a virtual machine, the virtual log generating module records the writing requests into the virtual log cache region in a sequential mode, and meanwhile a backstage supporter periodically synchronizes the data of the virtual log cache region to the Glusterfs server terminal. According to the virtual server system and the data reading-writing method of the virtual server system, the reading and writing performance of discs of the virtual machine can be greatly improved on the basis that original hardware equipment is not modified, delay is reduced, and therefore the user experience of desktop virtualization can be effectively improved.
Description
Technical field
The present invention relates to a kind of computer data memory technology, relate in particular to a kind of virtual server system and the data read-write method thereof that can apply KVM virtual machine.
Background technology
Along with IT technology is fast-developing, Intel Virtualization Technology reaches its maturity in recent years.Intel Virtualization Technology is that physical resource is abstracted into logical resource, hardware devices such as CPU, internal memory, disk, input and output becomes " resource pool " that can dynamic management, improve significantly hardware resource utilization, allow IT have more adaptive faculty to the variation of business.Due to the virtual reliability that can significantly reduce the overall cost of data center, the management of simplifying server, the rapid deployment of realizing server and raising data center, therefore increasing tissue starts to dispose virtual platform.Data center virtual mainly comprises the virtual of server and memory device, and the virtual of server is at present widely applied, but the virtualized progress of memory device is very slow.Owing to data storage method not being optimized, so server virtualization performance also can be affected in the expansion of data center.For example, moving a plurality of virtual machine instance on a station server, even if every virtual machine is all carrying out sequential write, but is to write at random from the aspect of server.So server disk equipment is all random read-write in virtualized environment, disk unit read-write fragmentation, randomization.For conventional hard such as SATA or SCSI, because its random read-write Performance Ratio order readwrite performance is far short of what is expected, therefore can affect the data storage efficiency of whole system.
At present, in prior art, also have and adopt Glusterfs distributed file system to integrate the disk resource on virtualized server, the technical scheme of disperseing the disk input and output of each virtualized server, but can not effectively to solve in virtualized environment be all the problem of random input and output to this scheme.
Summary of the invention
For these reasons, the object of this invention is to provide a kind of virtual server system and data read-write method thereof that can improve disk read-write performance.
Compared with prior art, useful technique effect of the present invention is:
1, the present invention takes full advantage of the modular nature of Glusterfs, in Glusterfs client modules, increase by a virtual log generation module, the read-write requests of sink virtual machine, if write request, in virtual log generation module inside, just this write request has been cached to local disk, avoided write request to continue by storage networking, the links such as random writing Glusterfs server module.
2, the present invention utilizes periodically reading out data from local disk of background thread refresh module (Flusher) in virtual log generation module, and data are synchronized to Glusterfs server end by storage networking, complete the actual storage of write request data, thereby can on the basis that does not change original hardware device, significantly improve virtual machine disk read-write performance, reduce and postpone.
3, the present invention is that virtual log generation module configures a local disk as virtual log buffer area, preferably uses SSD hard disk here, can further improve disk read-write performance.
Other features and advantages of the present invention will be set forth in the following description, and, partly from instructions, become apparent, or understand by implementing the present invention.Object of the present invention and other advantages can be realized and be obtained by specifically noted structure in instructions, claims and accompanying drawing.
Accompanying drawing explanation
Accompanying drawing is used to provide a further understanding of the present invention, and forms a part for instructions, for explaining the present invention, is not construed as limiting the invention together with embodiments of the present invention.In the accompanying drawings:
Fig. 1 is the composition structural representation of an embodiment of virtual server system provided by the invention;
Fig. 2 is the schematic diagram of the virtual log generation module encapsulation write request of virtual server system provided by the invention;
Fig. 3 is the processing flow chart that the virtual log generation module of virtual server system provided by the invention is accepted read-write requests.
Embodiment
In order to make object of the present invention, technical scheme and beneficial effect clearer, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein, only in order to explain the present invention, is not intended to limit the present invention.
In addition, in the step shown in the process flow diagram of accompanying drawing, can in the computer system such as one group of computer executable instructions, carry out, and, although there is shown logical order in flow process, but in some cases, can carry out shown or described step with the order being different from herein.
Before introducing the present invention, first explain two concepts of the prior art of using in the present invention.Respectively:
1) KVM virtual machine (kernel-based Virtual Machine), is a server virtualization module of increasing income, and is integrated in each main release version of Linux after Linux2.6.20.KVM virtual machine has become one of virtual machine monitor of academia and industry member main flow at present.
2) Glusterfs, is a distributed file system, and it has powerful ability extending transversely, can number of support PB memory capacity and the thousands of clients of processing by expansion.Glusterfs flocks together the storage resources of physical distribution by TCP/IP or InfiniBand RDMA network, by single global name space, carrys out management data.Glusterfs designs based on stackable user's space, can provide support for various data payload.
As shown in Figure 1, be the configuration diagram of virtual server system 100 provided by the invention, it comprises some virtualized servers 120 and the Glusterfs server end 130 connecting by storage networking 110:
On each virtualized server 120, operation has for example server virtualization module 121 and the Glusterfs client modules 122 of QEMU-KVM;
On each Glusterfs server end 130, stored virtual machine image file (not shown).
In the prior art, the server virtualization module 121 of virtualized server 120 can send to Glusterfs client modules 122 read-write requests of virtual machine, Glusterfs client modules 122 is searched the Glusterfs server end 130 at virtual machine image file place subsequently, and forward read-write requests to it, when Glusterfs server end 130 has completed after corresponding read-write requests operation, to Glusterfs client modules 122, return to corresponding operating result immediately, Glusterfs client modules 122 returns to read-write requests operating result to server virtualization module 121 again.Certainly, if this read-write requests is write request, real data will store Glusterfs server end 130 into so.
In order to improve significantly virtual machine magnetic disc i/o performance on the basis not changing original system hardware environment, improve data read-write efficiency, the present inventor proposes said system framework to do further improvement, in the Glusterfs of each virtualized server 120 client modules 122, set up a virtual log generation module 123, and be these virtual log generation module 123 configuration one corresponding storage spaces, also be virtual log buffer area 124, with the random read-write of virtual machine with the logged of order read-write in virtual log buffer area 124, background thread refresh module (Flusher) 125 in while virtual log generation module 123 is accesses virtual log buffer district 124 periodically, data are wherein synchronized to the Glusterfs server end 130 of bottom by storage networking 110.
In the specific implementation, above-mentioned virtual log buffer area 124 can adopt local disk, as SSD disk, and the record that storing virtual daily record generation module 123 is sent in a sequential manner.When having a new write request, virtual log generation module 123 is encapsulated as a recording and sending to virtual log buffer area 124 by this write request, is added to the afterbody of virtual log buffer area 124 storage data.Wherein, every record can adopt structure as shown in Figure 2 to encapsulate:
Magic number: the origin identification of record, for mark, this is a record;
Seq number: for the serial number of this record of mark;
Vdi id: for the id of virtual machine image file corresponding to this read-write requests of mark;
Offset: the side-play amount for this write request of mark with respect to vdi id respective file;
Size: for the size of these write request data of mark, it has also determined the size of real data data;
Flag: for this write request of mark corresponding mark and state;
Data: the real data that this write request is corresponding, this partial content length is variable, by size, is determined;
Magic number: for this end of record (EOR) of mark, to guarantee the integrality of this record, break down in case real data is write a half, cause real data imperfect;
Seq number: for this end of record (EOR) of mark.
Wherein, magic number and the seq number of every record head and the tail are identical parameters.
Below in conjunction with Fig. 3, explain the treatment scheme of above-mentioned virtual log generation module 123 when accepting read-write requests:
S301, while receiving a read-write requests;
S302, judge whether this read-write requests is write request:
If write request, execution step S303;
If non-write request, execution step S307;
S303, the head of constructing a record and afterbody, comprise magic number, seg number, vdi id, offset, size, a series of information such as flag;
S304, calculate this and be recorded in memory location side-play amount in virtual log buffer area;
S305, this record is write in virtual log buffer area;
S306, confirm that this record has write virtual log buffer area, returns to step S301;
S307, judge whether this read-write requests is read request:
If read request is carried out S308;
If non-read request, carries out S309.
S308, execution read request, return to step S301:
S309, carry out other requests.
In above-described embodiment, while performing step S308 receiving read request, virtual log generation module 123 can scan virtual log buffer area 124 to search desired data, if found, related data is returned to Glusterfs client modules 122, otherwise send read request to Glusterfs server end 130.
In addition, background thread refresh module (Flusher) 125 in virtual log generation module 123 is accesses virtual log buffer district 124 periodically, read data wherein, by storage networking, data are write to Glusterfs server end 130, thereby complete the actual storage of write request data.
Those skilled in the art should be understood that, above-mentioned each module of the present invention or each step can utilize general computer hardware equipment to realize, they both can concentrate on single calculation element, also can be distributed on the network being formed by a plurality of calculation elements, alternatively, they can be realized with the executable program code of calculation element, thereby, they can be stored in memory storage and be carried out by calculation element, or they are made into respectively to each integrated circuit modules, or a plurality of modules in them or step are made into single integrated circuit module to be realized.So the present invention is not restricted to any specific hardware and software combination.
Although the disclosed embodiment of the present invention as above, the embodiment that described content just adopts for the ease of understanding the present invention, not in order to limit the present invention.Technician in any the technical field of the invention; do not departing under the prerequisite of the disclosed spirit and scope of the present invention; can do any modification and variation what implement in form and in details; but scope of patent protection of the present invention, still must be as the criterion with the scope that appending claims was defined.
Claims (8)
1. a virtual server system, is characterized in that, comprising:
Virtualized server, it includes Glusterfs client modules, and described Glusterfs client modules includes virtual log generation module, and described virtual log generation module disposes virtual log buffer area;
Glusterfs server end, it connects described virtualized server by storage networking;
Wherein, described virtual log generation module sink virtual machine write request, virtual machine write request is packaged into record, sequentially write described virtual log buffer area, and described virtual log generation module is by virtual log buffer area described in background thread regular visit, and wherein data are synchronized to described Glusterfs server end by described storage networking.
2. virtual server system as claimed in claim 1, is characterized in that:
Described virtual log generation module is sink virtual machine read request also, according to virtual machine read request, access described virtual log buffer area to search related data, if find, related data is returned to described Glusterfs client modules, otherwise forwarded virtual machine read request to Glusterfs server end.
3. virtual server system as claimed in claim 2, is characterized in that:
Described record comprises that virtual machine image filename corresponding to record start sign, record serial number, write request, write request are with respect to side-play amount, write request size of data, write request mark and the state of corresponding virtual machine image file, actual data and the afterbody identification information write.
4. virtual server system as claimed in claim 1 or 2, is characterized in that:
Described virtual log buffer area is local memory device.
5. virtual server system as claimed in claim 4, is characterized in that:
Described memory device comprises SSD hard disk.
6. a data read-write method for virtual server system, described virtual server system comprises:
Virtualized server, it includes Glusterfs client modules, and described Glusterfs client modules includes virtual log generation module, and described virtual log generation module disposes virtual log buffer area;
Glusterfs server end, it connects described virtualized server by storage networking;
Described data read-write method comprises:
Write step: virtual log generation module sink virtual machine write request, virtual machine write request is packaged into record, sequentially write virtual log buffer area, and virtual log generation module is by background thread regular visit virtual log buffer area, and wherein data are synchronized to Glusterfs server end by storage networking.
7. data read-write method as claimed in claim 6, is characterized in that, described data read-write method also comprises:
Read step: virtual log generation module sink virtual machine read request, according to virtual machine read request accesses virtual log buffer district to search related data, if find, related data is returned to Glusterfs client modules, otherwise forwarded virtual machine read request to Glusterfs server end.
8. data read-write method as claimed in claim 7, is characterized in that:
Described writing in step, by virtual machine image filename corresponding to record start sign, record serial number, write request, write request with respect to side-play amount, write request size of data, write request mark and the state of corresponding virtual machine image file, actually write data and end of record identification information is packaged into record.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310628083.4A CN103605630B (en) | 2013-11-28 | 2013-11-28 | Virtual server system and data reading-writing method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310628083.4A CN103605630B (en) | 2013-11-28 | 2013-11-28 | Virtual server system and data reading-writing method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103605630A true CN103605630A (en) | 2014-02-26 |
| CN103605630B CN103605630B (en) | 2017-05-10 |
Family
ID=50123859
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310628083.4A Active CN103605630B (en) | 2013-11-28 | 2013-11-28 | Virtual server system and data reading-writing method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103605630B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104915302A (en) * | 2014-03-10 | 2015-09-16 | 华为技术有限公司 | Data transmission processing method and data transmission unit |
| CN104951238A (en) * | 2014-03-24 | 2015-09-30 | 国际商业机器公司 | Method and device for managing data storage in distributed virtual environment |
| CN106657076A (en) * | 2016-12-26 | 2017-05-10 | 北京神州绿盟信息安全科技股份有限公司 | TCP service implementation method and device of network namespace |
| CN106873910A (en) * | 2017-01-25 | 2017-06-20 | 东南大学 | A kind of SSD caching methods based on openstack platforms |
| CN107766120A (en) * | 2016-08-23 | 2018-03-06 | 华为技术有限公司 | The recording method of object information and relevant device in a kind of virtual machine |
| US10129357B2 (en) | 2015-08-21 | 2018-11-13 | International Business Machines Corporation | Managing data storage in distributed virtual environment |
| CN110703987A (en) * | 2019-08-31 | 2020-01-17 | 苏州浪潮智能科技有限公司 | Method and system for improving IO performance of virtual disk |
| CN110888778A (en) * | 2019-11-14 | 2020-03-17 | 西安雷风电子科技有限公司 | Cloud desktop-based log file monitoring system and method |
| CN116483284A (en) * | 2023-05-12 | 2023-07-25 | 深圳天融信创新科技有限公司 | Method, device, medium and electronic equipment for reading and writing virtual hard disk |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101179569A (en) * | 2007-02-15 | 2008-05-14 | 腾讯科技（深圳）有限公司 | Method, device and system for obtaining tourism log of electronic pet |
| CN101969386A (en) * | 2010-11-09 | 2011-02-09 | 道有道(北京)科技有限公司 | Log acquisition device and log acquisition method |
-
2013
- 2013-11-28 CN CN201310628083.4A patent/CN103605630B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101179569A (en) * | 2007-02-15 | 2008-05-14 | 腾讯科技（深圳）有限公司 | Method, device and system for obtaining tourism log of electronic pet |
| CN101969386A (en) * | 2010-11-09 | 2011-02-09 | 道有道(北京)科技有限公司 | Log acquisition device and log acquisition method |
Non-Patent Citations (2)
| Title |
|---|
| 周凡夫 等: "GlusterFS缓存机制研究", 《计算机应用与软件》, vol. 29, no. 7, 31 July 2012 (2012-07-31) * |
| 王光磊 等: "基于虚拟客户/服务器的清华嵌入式浏览器THEWB的设计与实现", 《计算机应用研究》, no. 4, 31 December 2002 (2002-12-31), pages 96 - 99 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104915302B (en) * | 2014-03-10 | 2018-07-03 | 华为技术有限公司 | Data transmission processing method and data link |
| CN104915302A (en) * | 2014-03-10 | 2015-09-16 | 华为技术有限公司 | Data transmission processing method and data transmission unit |
| CN104951238A (en) * | 2014-03-24 | 2015-09-30 | 国际商业机器公司 | Method and device for managing data storage in distributed virtual environment |
| US10129357B2 (en) | 2015-08-21 | 2018-11-13 | International Business Machines Corporation | Managing data storage in distributed virtual environment |
| CN107766120A (en) * | 2016-08-23 | 2018-03-06 | 华为技术有限公司 | The recording method of object information and relevant device in a kind of virtual machine |
| CN107766120B (en) * | 2016-08-23 | 2021-08-20 | 华为技术有限公司 | Method for recording object information in virtual machine and related equipment |
| CN106657076A (en) * | 2016-12-26 | 2017-05-10 | 北京神州绿盟信息安全科技股份有限公司 | TCP service implementation method and device of network namespace |
| CN106657076B (en) * | 2016-12-26 | 2020-04-07 | 北京神州绿盟信息安全科技股份有限公司 | TCP service implementation method and device for network name space |
| CN106873910B (en) * | 2017-01-25 | 2020-06-05 | 东南大学 | SSD (solid State disk) caching method based on openstack platform |
| CN106873910A (en) * | 2017-01-25 | 2017-06-20 | 东南大学 | A kind of SSD caching methods based on openstack platforms |
| CN110703987B (en) * | 2019-08-31 | 2022-07-22 | 苏州浪潮智能科技有限公司 | Method and system for improving IO performance of virtual disk |
| CN110703987A (en) * | 2019-08-31 | 2020-01-17 | 苏州浪潮智能科技有限公司 | Method and system for improving IO performance of virtual disk |
| CN110888778A (en) * | 2019-11-14 | 2020-03-17 | 西安雷风电子科技有限公司 | Cloud desktop-based log file monitoring system and method |
| CN110888778B (en) * | 2019-11-14 | 2023-03-28 | 西安雷风电子科技有限公司 | Cloud desktop-based log file monitoring system and method |
| CN116483284A (en) * | 2023-05-12 | 2023-07-25 | 深圳天融信创新科技有限公司 | Method, device, medium and electronic equipment for reading and writing virtual hard disk |
| CN116483284B (en) * | 2023-05-12 | 2024-05-28 | 深圳天融信创新科技有限公司 | Method, device, medium and electronic equipment for reading and writing virtual hard disk |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103605630B (en) | 2017-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103605630B (en) | Virtual server system and data reading-writing method thereof | |
| CN107967124B (en) | Distributed persistent memory storage system and method | |
| JP6106901B2 (en) | Data processing method and device in distributed file storage system | |
| US9158540B1 (en) | Method and apparatus for offloading compute resources to a flash co-processing appliance | |
| Stoyanov et al. | Efficient live migration of linux containers | |
| CN104239234B (en) | The management of a kind of local cache efficiently and reading/writing method | |
| CN107832423B (en) | File reading and writing method for distributed file system | |
| CN103516549B (en) | A kind of file system metadata log mechanism based on shared object storage | |
| CN104267912A (en) | NAS (Network Attached Storage) accelerating method and system | |
| TW201220197A (en) | for improving the safety and reliability of data storage in a virtual machine based on cloud calculation and distributed storage environment | |
| CN114201421B (en) | Data stream processing method, storage control node and readable storage medium | |
| CN104765575A (en) | Information storage processing method | |
| AU2013403132A1 (en) | Data storage method, data storage apparatus, and storage device | |
| US9690713B1 (en) | Systems and methods for effectively interacting with a flash memory | |
| CN104765574A (en) | Data cloud storage method | |
| CN104184812A (en) | Multi-point data transmission method based on private cloud | |
| US8869155B2 (en) | Increasing parallel program performance for irregular memory access problems with virtual data partitioning and hierarchical collectives | |
| CN106951182A (en) | A kind of block device caching method and device | |
| CN101377788B (en) | Method and system of caching management in cluster file system | |
| CN104778100A (en) | Safe data backup method | |
| Chen et al. | Low‐overhead inline deduplication for persistent memory | |
| US10831369B2 (en) | System and method for synchronizing caches after reboot | |
| CN101382955B (en) | File reading method in cluster file system and system | |
| CN110046132B (en) | Metadata request processing method, device, equipment and readable storage medium | |
| CN105068760A (en) | Data storage method, data storage apparatus and storage device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |