CN114995960A - Virtual machine resource pool starting optimization method, system and medium - Google Patents

Virtual machine resource pool starting optimization method, system and medium Download PDF

Info

Publication number
CN114995960A
CN114995960A CN202210844913.6A CN202210844913A CN114995960A CN 114995960 A CN114995960 A CN 114995960A CN 202210844913 A CN202210844913 A CN 202210844913A CN 114995960 A CN114995960 A CN 114995960A
Authority
CN
China
Prior art keywords
virtual machine
common
mirror image
resource pool
memory
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210844913.6A
Other languages
Chinese (zh)
Inventor
所光
舒畅
曹徐辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Galaxy Qilin Software Changsha Co ltd
Original Assignee
Galaxy Qilin Software Changsha Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Galaxy Qilin Software Changsha Co ltd filed Critical Galaxy Qilin Software Changsha Co ltd
Priority to CN202210844913.6A priority Critical patent/CN114995960A/en
Publication of CN114995960A publication Critical patent/CN114995960A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5022—Mechanisms to release resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45575—Starting, stopping, suspending or resuming virtual machine instances
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45583—Memory management, e.g. access or allocation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention discloses a virtual machine resource pool starting optimization method, a system and a medium, wherein the method comprises the step of identifying a relevant virtual machine set after receiving a virtual machine resource pool starting commandS vm (ii) a Determining a set of virtual machinesS vm The common mirror set used; determining a common mirror image set to be prefetched based on the common mirror image set and a system idle memory; and executing the pre-fetching operation of the common image to be pre-fetched according to the common image set to be pre-fetched so as to improve the starting speed of the virtual machine resource pool. The virtual machine resource pool starting optimization method is a data prefetching technology based on data sharing degree weighting and oriented to a physical server with limited memory, and can prefetch data of a virtual hard disk for a plurality of virtual machines running on the same physical server, so that the starting and running speeds of the virtual machine resource pool of the physical server are improved, and memory resources are utilized more efficiently.

Description

Virtual machine resource pool starting optimization method, system and medium
Technical Field
The invention relates to a cloud computing and virtualization technology, in particular to a virtual machine resource pool starting optimization method, a virtual machine resource pool starting optimization system and a virtual machine resource pool starting optimization medium.
Background
The combination of the virtualization technology and the cloud computing technology brings a brand-new resource integration and use mode, the resource allocation and scheduling capacity provided by the virtualization technology can improve the utilization rate of cloud computing resources, improve the service quality of cloud computing services, provide value-added services with high availability, load balance and the like, and reduce the total ownership cost of tenants in the cloud computing center. However, when a plurality of virtual machines are deployed in a physical server, the resource allocation efficiency of the physical server becomes an important issue affecting the use efficiency of the server. The virtual machine technology is that one server is virtualized into a plurality of groups of independent virtual servers on a physical server, and a virtual operating environment is provided for a plurality of independent operating systems.
Openstack is a typical representative of open source cloud platforms, and is the most widely used proprietary cloud platform at present. It includes seven core components: computation (Nova), object store (Swift), authentication (Keystone), user interface (Horizon), chunk store (Cinder), network (Neutron), and mirroring service (liance). Each component is a collection of services, one service meaning one process running. When a plurality of virtual machines share one physical server, the Nova, the block storage shader and the mirror service company are used for comparatively influencing the starting performance of the virtual machine resource pool, especially when a large number of virtual machines are started at the same time in the same server. When a large number of virtual machines (e.g., 50-100) are started simultaneously on the same physical server, this process can last for a long time, and in the extreme, for several hours, due to competing use of computing, memory, and storage resources. Therefore, how to implement virtual machine resource pool start optimization becomes a key technical problem to be solved urgently.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the invention can pre-fetch data of a virtual hard disk for a plurality of virtual machines running on the same physical server, improves the starting and running speeds of the virtual machine resource pool of the physical server, and utilizes memory resources more efficiently.
In order to solve the technical problems, the invention adopts the technical scheme that:
a virtual machine resource pool starting optimization method comprises the following steps:
s1, identifying relevant virtual machine set after receiving the command of starting the virtual machine resource poolS vm ；
S2, determining a virtual machine setS vm The common mirror set used;
s3, determining a common mirror image set to be prefetched based on the common mirror image set and the system idle memory;
and S4, executing the pre-fetching operation of the common image to be pre-fetched according to the common image set to be pre-fetched so as to improve the starting speed of the virtual machine resource pool.
Optionally, the identifying the corresponding virtual machine set in step S1 includes: judging whether the current physical server has a startup or restart event, if so, adding all the virtual machines on the current physical server into the virtual machine setS vm (ii) a Otherwise, all the virtual machines are extracted from the command of receiving the starting of the virtual machine resource pool and added into the virtual machine setS vm 。
Optionally, in the public mirror image sets obtained in step S2, each public mirror image set includes two attributes, i.e., a size and a reuse degree, where the size is an actual file size of the public mirror image, and the reuse degree refers to a virtual machine setS vm The number of virtual machines using the common image.
Optionally, the virtual machine set is determined in step S2S vm The common mirror collection used includes:
s2.1, initializing the setS backing_file Is empty;
s2.2, aiming at virtual machine setS vm Is traversed for each current virtual machine obtained by the traversalvm i Magnetic disk ofdisk i Corresponding common mirror imagebacking_file i : if common mirror imagebacking_file i Is not in the setS backing_file Then add it to the collectionS backing_file Initializing two attributes of the size and the reusability of the public mirror image, wherein the initial value of the size is the actual file size of the public mirror image, and the initial value of the reusability is 1; otherwise, will assembleS backing_file Adding 1 to the reusability of the corresponding public mirror image already existing in the image;
s2.3, aggregatingS backing_file As a set of virtual machinesS vm The common set of mirrors used.
Optionally, step S3 includes:
s3.1, screening the reuse degree of each public mirror image in the public mirror image set, and forming the public mirror images with the reuse degree larger than a preset threshold ϑ into a hot spot public mirror image setS hot_files ；
S3.2, aiming at the hotspot public mirror image setS hot_files Solving a common mirror image set to be prefetched which satisfies the following constraintS prefetch ；
Figure 672660DEST_PATH_IMAGE001
，
In the above-mentioned formula, the compound has the following structure,xfor common mirror sets to be prefetchedS prefetch Is a common mirror image of (a) a,x.freqas a common mirror imagexThe degree of the reusability of the compound fertilizer,x.sizeas a common mirror imagexThe size of the file of (a) is,S hot_files is a common set of images for a hotspot,αis a parameter of the proportionality coefficient,m free and (4) idle memory for the system on the current physical server.
Optionally, the step S4 of executing the pre-fetching operation of the common image to be pre-fetched according to the common image set to be pre-fetched refers to: all common images for a common image set to be prefetchedxSeparately executing prefetch functions to mirror the common imagexThe data in (2) is prefetched into memory and locked into memory.
Optionally, the executing prefetch function mirrors the common imagexWhen the data is pre-fetched to the memory, the common mirror image is judgedxIf the format of (2) is a sparse file, executing a memory address mapping function to map the public mirror imagexThe effective data is mapped to an independent process in a read-only mode to be prefetched to the memory, and the address in which the effective data is stored is locked in the memory based on a memory address locking function; otherwise, directly executing the memory address mapping function to mirror the publicxMapping the data to an independent process in a read-only mode so as to prefetch the memory, and locking the address in which the effective data is stored in the memory based on a memory address locking function.
In addition, the invention also provides a virtual machine resource pool starting optimization system, which comprises:
a command receiving module for identifying relevant virtual machine set after receiving a command for starting the virtual machine resource poolS vm ；
A prefetch analysis module to determine a set of virtual machinesS vm The common mirror set used;
the system comprises a prefetching decision module, a prefetching module and a prefetching module, wherein the prefetching decision module is used for determining a common mirror image set to be prefetched based on a common mirror image set and a system idle memory;
and the pre-fetching execution module is used for executing the pre-fetching operation of the common mirror image to be pre-fetched according to the common mirror image set to be pre-fetched so as to improve the starting speed of the virtual machine resource pool.
In addition, the invention also provides a virtual machine resource pool starting optimization system which comprises a microprocessor and a memory which are connected with each other, wherein the microprocessor is programmed or configured to execute the virtual machine resource pool starting optimization method.
Furthermore, the present invention also provides a computer-readable storage medium having stored therein a computer program for being programmed or configured by a microprocessor to perform the virtual machine resource pool start-up optimization method.
Compared with the prior art, the invention mainly has the following advantages: the method for optimizing the starting of the virtual machine resource pool comprises the step of identifying a relevant virtual machine set after receiving a command for starting the virtual machine resource poolS vm (ii) a Determining a set of virtual machinesS vm The common mirror set used; determining a common mirror image set to be prefetched based on the common mirror image set and a system idle memory; root of herbaceous plantAnd executing the pre-fetching operation of the common image to be pre-fetched according to the common image set to be pre-fetched so as to improve the starting speed of the virtual machine resource pool. The invention discloses a virtual machine resource pool starting optimization method, which is a data prefetching technology based on data sharing degree weighting and oriented to a physical server with limited memory. Then, the public basic mirror image sets used by the virtual machine sets are obtained through inquiry, the size and the usability of each basic mirror image are obtained, and then the public basic mirror image sets needing to be prefetched can be identified according to the use condition of the current memory, the size and the usability of each basic mirror image and the use strategy set by the system. And finally, prefetching the common basic mirror image according to the common basic mirror image set needing to be prefetched, so that data of the virtual hard disk can be prefetched for a plurality of virtual machines running on the same physical server, the starting and running speeds of a virtual machine resource pool of the physical server are improved, and memory resources are utilized more efficiently.
Drawings
FIG. 1 is a schematic diagram of a basic flow of a method according to an embodiment of the present invention.
Fig. 2 is a flowchart of step S1 in the embodiment of the present invention.
Fig. 3 is a flowchart of step S2 in the embodiment of the present invention.
Fig. 4 is a flowchart of step S3 in the embodiment of the present invention.
Fig. 5 is a flowchart of step S4 in the embodiment of the present invention.
FIG. 6 is a prefetch process for sparse files according to an embodiment of the present invention.
FIG. 7 is a diagram illustrating data access after the method of the embodiment of the present invention is implemented.
Fig. 8 is a schematic diagram of the structure of each program module of the system according to the embodiment of the present invention.
Detailed Description
The subject matter described herein will be discussed with reference to example embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and thereby implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. Changes may be made in the function and arrangement of elements discussed without departing from the scope of the embodiments of the disclosure. Various examples may omit, substitute, or add various procedures or components as necessary. In addition, features described with respect to some examples may also be combined in other examples.
As used herein, the term "include" and its variants mean open-ended terms in the sense of "including, but not limited to. The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first," "second," and the like may refer to different or the same object. Other definitions, whether explicit or implicit, may be included below. Unless the context clearly dictates otherwise, the definition of a term is consistent throughout the specification.
Embodiments of the present invention will be described in further detail below with an Openstack cloud platform as an example.
As shown in fig. 1, the method for optimizing the starting of the resource pool of the virtual machine in this embodiment includes:
s1, identifying relevant virtual machine set after receiving the command of starting the virtual machine resource poolS vm ；
S2, determining a virtual machine setS vm The common mirror set used;
s3, determining a common mirror image set to be prefetched based on the common mirror image set and the system idle memory;
and S4, executing the pre-fetching operation of the common image to be pre-fetched according to the common image set to be pre-fetched so as to improve the starting speed of the virtual machine resource pool.
Step S1 includes two invoking occasions when the command for starting the virtual machine resource pool is received, first, when the physical service is powered off and then powered on or restarted, all the virtual machines on the server need to be restarted at this timeStarting up the system; second, when a user newly builds a virtual machine resource pool on a physical server and starts the resource pool. Specifically, as shown in fig. 2, the identifying the corresponding virtual machine set in step S1 in this embodiment includes: judging whether the current physical server has a startup or restart event, if so, adding all the virtual machines on the current physical server into the virtual machine setS vm (ii) a Otherwise, extracting all virtual machines from the command of receiving the starting of the virtual machine resource pool to join the virtual machine setS vm . If the starting or restarting event occurs, adding all the virtual machines on the current physical server into the virtual machine setS vm It can be expressed as:
S vm = set of all virtual machines on the server;
if the starting or restarting event does not occur, all the virtual machines are extracted from the command (command sent by Nova) for receiving the starting of the virtual machine resource pool and added into the virtual machine setS vm It can be expressed as:
S vm = { set of virtual machines started in Nova issue command }.
It should be noted that the occurrence of the boot or restart event may adopt a required determination manner as required, for example, whether the boot or restart event is determined by a system log, or may be implemented by a program or service that is automatically started by booting as required (the program may be automatically started after booting or restarting).
In this embodiment, in the common mirror image sets obtained in step S2, each common mirror image set includes a size (denoted assize) Degree of harmony (note asfreq) Two attributes, the size is the actual file size of the public mirror image, and the reusability refers to a virtual machine setS vm The number of virtual machines using the common image.
In the present embodiment, a virtual machine set is assumedS vm IncludednIf n is a positive integer, the virtual machines are collectedS vm The formalized representation is:
S vm ={vm 1 , vm 2 ,…vm n }，
in the above formula, the first and second carbon atoms are,vm 1 ～vm n are respectively a virtual machine setS vm In 1EnA virtual machine, then any ith virtual machine can representvm i （1≤i≤n). In this embodiment, assume a setS backing_file Is a common set of images that may be prefetched.
As shown in FIG. 3, the set of virtual machines is determined in step S2S vm The common mirror set used includes:
s2.1, initializing the setS backing_file Is empty;
s2.2, aiming at virtual machine setS vm Is traversed for each current virtual machine obtained by the traversalvm i Magnetic disk ofdisk i Corresponding common mirror imagebacking_file i : if common mirror imagebacking_file i Is not in the aggregateS backing_file Then add it to the collectionS backing_file And initializes its two attributes of size and multiplexing degree, and size: (size) The initial value of (a) is the actual file size of the common mirror image, the degree of multiplexing (b)freq) Is 1, can be expressed as:
backing_file i .freq=1，
backing_file i .size=backing_file_size，
wherein the content of the first and second substances,backing_file_sizethe actual file size of the public mirror image; otherwise, will assembleS backing_file The reusability of the corresponding common mirror image already existing in (1) can be expressed as:
backing_file i .freq++；
step S2.2 requires to go through 1EnThe virtual machines finally obtain a common image set which can be prefetched, namely: collectionS backing_file ；
S2.3, will collectClosing boxS backing_file As a set of virtual machinesS vm The common set of mirrors used.
As shown in fig. 4, step S3 includes:
s3.1, screening the reuse degree of each public mirror image in the public mirror image set, and forming the public mirror images with the reuse degree larger than a preset threshold value ϑ into a hot spot public mirror image setS hot_files It can be expressed as:
S hot_files ={x|x∈S backing_file and is made ofx.freq>ϑ}；
S3.2, aiming at the hotspot public mirror image setS hot_files Solving a common mirror image set to be prefetched satisfying the following constraintS prefetch ；
Figure 452398DEST_PATH_IMAGE001
，
In the above formula, the first and second carbon atoms are,xfor common mirror sets to be prefetchedS prefetch Is a common mirror image of (a) a,x.freqas a common mirror imagexThe degree of the reusability of the compound fertilizer,x.sizeas a common mirror imagexThe size of the file of (a) is,S hot_files is a common set of images for a hotspot,αis a proportionality coefficient parameter (constant, value can be set according to requirements),m free and (4) idle memory for the system on the current physical server.
System idle memory on current physical serverm free Can be obtained by a resource monitoring module in the system on the current physical server, because the idle memory of the system is obtainedm free Are well known in the art and therefore the details of their implementation are not set forth herein in detail. The resource monitoring module is used for monitoring the running state information of the physical server and the virtual machine, and the content comprises the following steps: and the memory resource occupation data of the physical server and the virtual disk I/O performance data of the virtual machine.
In this embodiment, it is assumed that the ratio of the memory occupied by prefetching to the free memory is a scaling factor parameterαThen, thenα×m free The maximum memory available for the prefetch decision module. Suppose thatS prefetch Is to be prefetched of a common mirrored set, anS prefetch ⊆S hot_files . Then, solving the constraint satisfaction problem can obtain the optimal prefetch setS prefetch . The optimization of the problem satisfied by the constraint is aimed at maximizing the reuse of data pre-fetched into memory (first line of the constraint), because the greater the reuse, the better the temporal locality of the data reuse and the greater the resulting performance gain. The constraint satisfies the solution of the problem-the optimal prefetch setS prefetch Indicating the base set of image files that need to be prefetched.
As shown in fig. 5, the step S4 of executing the pre-fetching operation of the common image to be pre-fetched according to the common image set to be pre-fetched refers to: all common images for a common image set to be prefetchedxSeparately executing prefetch functions to mirror the common imagexThe data to be prefetched into the memory and locked in the memory can be expressed as:prefetch_file(x) Whereinprefetch_fileIs a pre-fetch function.
Prefetch functionprefetch_fileHas the main function of mirroring a common imagexThe effective data in the virtual machine is prefetched into the memory and locked in the memory, and after all the virtual machines are started, the locked prefetched memory is released as appropriate according to the system memory occupation condition provided by the resource monitoring module. To increase the prefetch functionprefetch_fileEfficiency of, prefetch functionprefetch_fileDuring prefetching, the influence of the sparse file data format on prefetching needs to be considered. In this embodiment, execution of the prefetch function mirrors the common imagexWhen the data is pre-fetched to the memory, the common mirror image is judgedxIf the format of (a) is a sparse file, a memory address mapping function (specifically, mmap function in this embodiment) is executed to mirror the public imagexThe effective data is mapped to an independent process in a read-only mode to be prefetched into the memory, and the address in which the effective data is stored is locked in the memory based on a memory address locking function (specifically, an mlock function in the embodiment); otherwise, directly executing the memory address mapping function to mirror the publicxMapping the data to an independent process in a read-only mode to prefetch the data to the memory, and locking the address in which the effective data is stored in the memory based on a memory address locking function.
Fig. 6 shows a prefetching process oriented to sparse files, where a row at the bottom is a storage structure distribution of the sparse files in the disk, a row at the middle is a logical address space distribution of the sparse files, and a row at the top is a data distribution mapped to the memory by using a mmap method. The sparse file is a file including a plurality of independent and discontinuous data areas in a storage structure of a disk, for example, the sparse file shown in fig. 6 includes three independent and discontinuous data areas, namely a data area 1 to a data area 3, because the three independent and discontinuous data areas enable a data hole to be included between adjacent data areas in a logic address space of the sparse file, effective data stored in the file system of the sparse file is far smaller than the size of the file, and more data holes exist, a pre-fetching function is performedprefetch_fileWhen prefetching a sparse file, only the contents of the file with actual data need to be prefetched into the memory. In the embodiment, an mmap method is adopted, effective data of a sparse file is mapped to an independent process in a read-only mode, and an address for storing the effective data is locked in a memory by an mlock method. Therefore, when other processes or virtual machines in the system access the data of the data file, the data can be directly hit in the memory, and the access to the back-end storage is reduced.
Fig. 7 is a schematic diagram illustrating data access after the method of the present embodiment is implemented. The lowest layer is a data storage layer, and virtual disk files of three virtual machines are assumed to be stored in a file system stored in a disk or a network block, and are named as: mirror 1, mirror 2, mirror 3. Suppose that three image files all use the same common imagebacking_file. Then, for example, when virtual machine 1 accesses data of mirror 1, the data may be stored in mirror 1 or a common mirrorbacking_fileWhen the data valid value is in the common mirror imagebacking_fileIn time, the virtual machines are directly from the common imagebacking_fileTaking data, when the effective value of the data is in mirror 1 and common mirrorbacking_fileWhen both exist, the virtual machine fetches the data from image 1. When the mirror images 1, 2 and 3 have more public data, the storage space of the mirror image file can be greatly reduced by using the public mirror image file, for example, when the virtual machines 1, 2 and 3 have the same operating system, the three mirror images can use the public mirror image filebacking_file. In the implementation process, the pre-fetching control module is integrated in a Nova module of an Openstack cloud platform. Step one, Nova uses the method of the embodiment to map the public mirror image into a memory in an mmap mode; second, when three virtual machines need to access their respective images, valid data is stored in a common imagebacking_fileThen, the public mirror image is directly fetched from the memorybacking_fileWhen valid data is stored in the mirror image 1, the mirror image 2 and the mirror image 3, the virtual machine still acquires the data from the disk by taking the previous data access path. It should be noted that the operating system itself has a buffer mechanism, and after the virtual machine accesses the data of the disk, the data will also be stored in the buffer managed by the operating system, but the buffer itself also has a removal policy, and when the operating system determines that the buffer usage frequency is not high, the buffer will also be removed. Compared with the pre-fetching algorithm provided by the method of the embodiment, the buffer managed by the operating system needs to access a disk when the disk is accessed for the first time, the performance is low, and in addition, data can be eliminated in the subsequent access process, and the strategy is not intelligent enough.
As shown in fig. 8, this embodiment further provides a virtual machine resource pool starting optimization system, including:
a command receiving module for identifying relevant virtual machine set after receiving a command for starting the virtual machine resource poolS vm ；
A prefetch analysis module to determine a set of virtual machinesS vm The common mirror set used;
the system comprises a prefetching decision module, a prefetching module and a prefetching module, wherein the prefetching decision module is used for determining a common mirror image set to be prefetched based on a common mirror image set and a system idle memory;
and the pre-fetching execution module is used for executing the pre-fetching operation of the common mirror image to be pre-fetched according to the common mirror image set to be pre-fetched so as to improve the starting speed of the virtual machine resource pool.
In addition, the embodiment further includes a prefetch control module, configured to control execution of the virtual machine resource pool start optimization method, and control execution of data prefetching based on data sharing weight for the memory-limited physical server, that is, execution of the command receiving module, the prefetch analysis module, the prefetch decision module, and the prefetch execution module; in addition, the present embodiment further includes a resource monitoring module, configured to monitor operating state information of the physical server and the virtual machine, where the content includes: and the memory resource occupation data of the physical server and the virtual disk I/O performance data of the virtual machine. Other modules in the system, including the prefetch analysis module and the prefetch decision module, use the resource utilization status information provided by the resource monitoring module. When the virtual machine starting module works, the command receiving module receives a virtual machine resource pool starting command from Nova, identifies and filters the virtual machine starting command which does not need to be prefetched, and identifies a virtual machine set related to the virtual machine resource pool starting command which needs to be prefetched. The command receiving module has two calling occasions, and when the physical service is started up or restarted after being shut down, all virtual machines on the server need to be restarted at the moment; second, when a user newly builds a virtual machine resource pool on a physical server and starts the resource pool. And the pre-fetching analysis module inquires and obtains a public basic mirror image set used by the virtual machine sets according to the virtual machine sets which are obtained at the last stage and need to be started, and obtains the size and the reusability of each basic mirror image. And the pre-fetching decision module calculates a public basic mirror image set to be pre-fetched according to the use condition of the current memory, the size and the reusability of each basic mirror image and the use strategy set by the system. The prefetch execution module prefetches the common base image according to a common base image set that needs to be prefetched. The prefetch execution module needs to consider the influence of the sparse file on prefetch data in the operation process, because the sparse file has a data hole, and the data hole does not need to be prefetched into a memory. The execution of each program module needs the cooperation of the Nova module, the Cinder module and the Glance module of the Openstack cloud platform to allocate and manage resources, the allocation of computing and memory resources is mainly completed by the responsibility of the Nova module, and the allocation and management of storage resources need the cooperation of the Nova module, the Cinder module and the Glance module.
In summary, the core of the method of the present embodiment is a data prefetching technique based on weighting of data sharing degree for a physical server with limited memory. The technology receives a virtual machine resource pool start command from Nova, and identifies a virtual machine set to which the virtual machine resource pool start command needs to be prefetched. And then, inquiring to obtain the common basic mirror image sets used by the virtual machine sets, obtaining the size and the utilization degree of each basic mirror image, and identifying the common basic mirror image set needing to be prefetched according to the use condition of the current memory, the size and the utilization degree of each basic mirror image and the use strategy set by the system. And finally, prefetching the common basic mirror image according to the common basic mirror image set needing prefetching. The embodiment is based on an Openstack cloud computing platform, and provides a data prefetching technology based on data sharing degree weighting and oriented to a physical server with limited memory for a plurality of virtual machines to share accessed functional virtual disk data. Based on the technology, before a large number of virtual machines are started, the public data which are required to be accessed by the corresponding virtual machines are extracted and prefetched into the memory, the operating systems are prevented from eliminating the public data or replacing the public data into the backup cache in the starting process, and after all the operating systems of the virtual machines are started, the data which are prefetched into the memory are eliminated according to the service condition of the memory of the server. The method of the embodiment has the advantages that: the method and the system can pre-fetch data of the virtual hard disk for a plurality of virtual machines running on the same physical server, improve the starting and running speeds of the virtual machine resource pool of the physical server, more efficiently utilize memory resources, improve the starting and running speeds of the virtual machine resource pool of the physical server, and enhance user experience.
In addition, the present embodiment also provides a virtual machine resource pool start optimization system, which includes a microprocessor and a memory connected to each other, where the microprocessor is programmed or configured to execute the virtual machine resource pool start optimization method.
Furthermore, the present embodiment also provides a computer-readable storage medium, in which a computer program is stored, the computer program being programmed or configured by a microprocessor to perform the virtual machine resource pool start optimization method.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Not all steps and modules in the above flows and system structure diagrams are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed, and can be determined as required. The apparatus structures described in the above embodiments may be software structures or logic structures, that is, some modules may be implemented by the same software entity, or some modules may be implemented by a plurality of software entities, or some components in a plurality of independent devices may be implemented together.
The term "exemplary" used throughout this specification means "serving as an example, instance, or illustration," and does not mean "preferred" or "advantageous" over other embodiments. The detailed description includes specific details for the purpose of providing an understanding of the described technology. However, the techniques may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the concepts of the described embodiments.
Although the embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, the embodiments of the present disclosure are not limited to the specific details of the embodiments, and various simple modifications may be made to the technical solutions of the embodiments of the present disclosure within the technical spirit of the embodiments of the present disclosure, and all of them fall within the scope of the embodiments of the present disclosure.
The previous description of the specification is provided to enable any person skilled in the art to make or use the specification. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations without departing from the scope of the disclosure. Thus, the description is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A virtual machine resource pool starting optimization method is characterized by comprising the following steps:
s1, identifying relevant virtual machine set after receiving the command of starting the virtual machine resource poolS vm ；
S2, determining a virtual machine setS vm The common mirror set used;
s3, determining a common mirror image set to be prefetched based on the common mirror image set and the system idle memory;
and S4, executing the pre-fetching operation of the common image to be pre-fetched according to the common image set to be pre-fetched so as to improve the starting speed of the virtual machine resource pool.
2. The method for optimizing resource pool startup of a virtual machine according to claim 1, wherein the step S1 of identifying a corresponding virtual machine set includes: judging whether the current physical server has a startup or restart event, if so, adding all virtual machines on the current physical server into a virtual machine setS vm (ii) a Otherwise, extracting all virtual machines from the command of receiving the starting of the virtual machine resource pool to join the virtual machine setS vm 。
3. The virtual machine resource pool startup optimization method of claim 1,it is characterized in that in the public mirror image sets obtained in step S2, each public mirror image set includes two attributes of size and reusability, the size is the actual file size of the public mirror image, and the reusability refers to the virtual machine setS vm The number of virtual machines using the common image.
4. The virtual machine resource pool starting optimization method according to claim 3, wherein the virtual machine set is determined in step S2S vm The common mirror set used includes:
s2.1, initializing the setS backing_file Is empty;
s2.2, aiming at virtual machine setS vm Is traversed for each current virtual machine obtained by the traversalvm i Magnetic disk ofdisk i Corresponding common mirror imagebacking_file i : if common mirror imagebacking_file i Is not in the aggregateS backing_file Then add it to the collectionS backing_file Initializing two attributes of the size and the reusability of the public mirror image, wherein the initial value of the size is the actual file size of the public mirror image, and the initial value of the reusability is 1; otherwise, will assembleS backing_file Adding 1 to the reusability of the corresponding public mirror image already existing in the image;
s2.3, aggregatingS backing_file As a set of virtual machinesS vm The common mirror set used.
5. The virtual machine resource pool starting optimization method according to claim 3, wherein the step S3 includes:
s3.1, screening the reuse degree of each public mirror image in the public mirror image set, and forming the public mirror images with the reuse degree larger than a preset threshold ϑ into a hot spot public mirror image setS hot_files ；
S3.2, aiming at the hotspot public mirror image setS hot_files Solving a common mirror image set to be prefetched satisfying the following constraintS prefetch ；
Figure 700085DEST_PATH_IMAGE001
，
In the above formula, the first and second carbon atoms are,xfor common mirror sets to be prefetchedS prefetch Is a common mirror image of (a) a,x.freqas a common mirror imagexThe degree of the reuse of the composite material,x.sizeas a common mirror imagexThe size of the file of (a) is,S hot_files is a common set of images for a hotspot,αis a parameter of the proportionality coefficient,m free and (4) idle memory for the system on the current physical server.
6. The virtual machine resource pool starting optimization method according to claim 3, wherein the step S4 of executing the pre-fetching operation of the common image to be pre-fetched according to the common image set to be pre-fetched refers to: all common images for a common image set to be prefetchedxSeparately executing prefetch functions to mirror the common imagexThe data in (2) is prefetched into memory and locked into memory.
7. The virtual machine resource pool start optimization method of claim 6, wherein the executing pre-fetch function mirrors a common imagexWhen the data is pre-fetched to the memory, the common mirror image is judgedxIf the format of (1) is a sparse file, executing a memory address mapping function to mirror the public image if the format of (1) is the sparse filexThe effective data is mapped to an independent process in a read-only mode to be prefetched to the memory, and the address in which the effective data is stored is locked in the memory based on a memory address locking function; otherwise, directly executing the memory address mapping function to mirror the publicxMapping the data to an independent process in a read-only mode to prefetch the data to the memory, and locking the address in which the effective data is stored in the memory based on a memory address locking function.
8. A virtual machine resource pool startup optimization system is characterized by comprising:
command receiving module forIdentifying relevant virtual machine set after receiving command of starting virtual machine resource poolS vm ；
A prefetch analysis module to determine a set of virtual machinesS vm The common mirror set used;
the system comprises a prefetching decision module, a prefetching module and a prefetching module, wherein the prefetching decision module is used for determining a common mirror image set to be prefetched based on a common mirror image set and a system idle memory;
and the pre-fetching execution module is used for executing the pre-fetching operation of the common mirror image to be pre-fetched according to the common mirror image set to be pre-fetched so as to improve the starting speed of the virtual machine resource pool.
9. A virtual machine resource pool start-up optimization system comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to perform the virtual machine resource pool start-up optimization method of any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored, wherein the computer program is programmed or configured by a microprocessor to perform the virtual machine resource pool start-up optimization method according to any one of claims 1 to 7.
CN202210844913.6A 2022-07-19 2022-07-19 Virtual machine resource pool starting optimization method, system and medium Pending CN114995960A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210844913.6A CN114995960A (en) 2022-07-19 2022-07-19 Virtual machine resource pool starting optimization method, system and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210844913.6A CN114995960A (en) 2022-07-19 2022-07-19 Virtual machine resource pool starting optimization method, system and medium

Publications (1)

Publication Number Publication Date
CN114995960A true CN114995960A (en) 2022-09-02

Family

ID=83022700

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210844913.6A Pending CN114995960A (en) 2022-07-19 2022-07-19 Virtual machine resource pool starting optimization method, system and medium

Country Status (1)

Country Link
CN (1) CN114995960A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103067425A (en) * 2011-10-20 2013-04-24 中国移动通信集团公司 Creation method of virtual machine, management system of virtual machine and related equipment thereof
CN103544045A (en) * 2013-10-16 2014-01-29 南京大学镇江高新技术研究院 HDFS-based virtual machine image storage system and construction method thereof
CN104050015A (en) * 2014-06-27 2014-09-17 国家计算机网络与信息安全管理中心 Mirror image storage and distribution system for virtual machines
CN112486858A (en) * 2016-03-17 2021-03-12 华为技术有限公司 Data prefetching method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103067425A (en) * 2011-10-20 2013-04-24 中国移动通信集团公司 Creation method of virtual machine, management system of virtual machine and related equipment thereof
CN103544045A (en) * 2013-10-16 2014-01-29 南京大学镇江高新技术研究院 HDFS-based virtual machine image storage system and construction method thereof
CN104050015A (en) * 2014-06-27 2014-09-17 国家计算机网络与信息安全管理中心 Mirror image storage and distribution system for virtual machines
CN112486858A (en) * 2016-03-17 2021-03-12 华为技术有限公司 Data prefetching method and device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
刘圣卓: ""面向虚拟集群的镜像存储与传输优化"", 《中国博士学位论文全文数据库 信息科技辑》 *
张晨: ""基于容器访问模式的Docker缓存策略研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Similar Documents

Publication Publication Date Title
US9606822B2 (en) Lightweight on-demand virtual machines
CA2566681C (en) Dynamic memory management of unallocated memory in a legal partitioned data processing system
US9946641B2 (en) Memory management model and interface for new applications
US7529867B2 (en) Adaptive, scalable I/O request handling architecture in virtualized computer systems and networks
EP2721480B1 (en) Memory manager with enhanced application metadata
US8307369B2 (en) Power control method for virtual machine and virtual computer system
US8145872B2 (en) Autonomic self-tuning of database management system in dynamic logical partitioning environment
US20080104589A1 (en) Adaptive, Scalable I/O Request Handling Architecture in Virtualized Computer Systems and Networks
US9785470B2 (en) Memory management model and interface for unmodified applications
US20110167239A1 (en) Methods and apparatuses for usage based allocation block size tuning
US7805582B2 (en) Method of managing memory in multiprocessor system on chip
AU2011213795A1 (en) Efficient cache reuse through application determined scheduling
US20080104590A1 (en) Adaptive, Scalable I/O Request Handling Architecture in Virtualized Computer Systems and Networks
CN114995960A (en) Virtual machine resource pool starting optimization method, system and medium
CN114625544B (en) Memory balloon management method and device based on dynamic allocation of virtual machine service load

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20220902

RJ01 Rejection of invention patent application after publication