JP3861087B2 - Virtual machine management apparatus and program - Google Patents

Description

  The present invention relates to a virtual machine management apparatus that optimally arranges a plurality of virtual machines on a plurality of servers.

  2. Description of the Related Art Conventionally, a virtual machine (VM) technology is known in which a server machine resource is divided into a plurality of pieces using software and independent, and a plurality of virtual machines are simultaneously operated on one server. In order to operate such a virtual machine system more efficiently, a technique for optimally arranging virtual machines on a server has been devised.

For example, in a system that operates a plurality of virtual machines on one server, there is also a virtual computer system that dynamically allocates resources to virtual machines according to the load amount of each virtual machine (see Patent Document 1). .
JP 2002-202959 A (page 6, FIG. 1)

  As described above, the conventional system manages a plurality of virtual machines by operating them on a single server. When a virtual machine is operated using a plurality of servers, it is arranged on each server. Since the virtual machines are managed independently for each server, there is a load imbalance among the servers depending on the operating status of the virtual machines.

The present invention has been made in view of the above circumstances, and an object thereof is to provide a virtual machine management apparatus and the like that can efficiently operate a plurality of virtual machines on a plurality of servers.
Another object of the present invention is to provide a virtual machine management apparatus and the like that can perform optimal placement of virtual machines across a plurality of servers.

In order to achieve the above object, a virtual machine management device according to the first aspect of the present invention provides:
A virtual machine management apparatus connected via a network to a plurality of servers that operate a plurality of virtual machines,
In the first predetermined period, for each virtual machine, an operation information management database that stores actual measurement data indicating performance acquired every predetermined time;
Reading means for reading actual measurement data indicating the performance of each virtual machine every predetermined time from the operation information management database;
Using the actual measurement data indicating the performance of each read virtual machine for each predetermined time, the virtual machine at each time when each virtual machine is operated on any of a plurality of servers for each predetermined time . A combination calculation unit that calculates a performance value , and calculates a combination of the virtual machine and the server in which the performance value of the virtual machine for each determined predetermined time is maximized in a second predetermined period in which a plurality of the predetermined times are collected ; ,
Means for relocating each virtual machine by storing a file of each virtual machine in a storage area of the server corresponding to the virtual machine according to the combination of the virtual machine and the server calculated by the combination calculating means;
It is characterized by providing.

  According to such a configuration, a combination of a virtual machine and a virtual machine server that uses the resources of all virtual machine servers to maximize the performance of each virtual machine is obtained, and the placement of the virtual machines is based on this combination. Therefore, it is possible to optimally arrange virtual machines across a plurality of servers, and a plurality of virtual machines can be efficiently operated on a plurality of servers.

The reading means may further read data relating to the storage capacity of each server and data relating to the storage capacity of each virtual machine from the operation information management database,
The combination calculation means is the measurement data read by the reading means and indicating the performance of each virtual machine every predetermined time, the data relating to the storage capacity of each server, the data relating to the storage capacity of each virtual machine, May be used as a calculation condition to calculate a combination of the virtual machine and the server.

Means for receiving a virtual machine creation request;
Means for selecting a server capable of storing the virtual machine file indicated by the creation request in the storage area based on the usage amount of the storage area of each server and the file capacity of the virtual machine indicated by the received creation request; ,
Means for storing a virtual machine file indicated by the creation request in the storage area of the selected server;
May be provided.

A program according to the second aspect of the present invention is
To a computer connected to multiple servers running multiple virtual machines via a network,
A storage step of storing, in the operation information management database, actual measurement data indicating performance acquired at predetermined time intervals for each virtual machine in the first predetermined period;
A step of reading out actual measurement data indicating the performance of each virtual machine every predetermined time from the operation information management database;
At each time when each virtual machine is operated on one of a plurality of servers for each predetermined time , using the measured data indicating the performance of each virtual machine for each predetermined time read by the reading step. A performance value of each virtual machine is obtained, and a combination of a virtual machine and a server is calculated such that the performance value of the virtual machine at the obtained predetermined time becomes the maximum in a second predetermined period in which a plurality of the predetermined times are collected. Combination calculation step,
Relocating each virtual machine by storing the file of each virtual machine in the storage area of the server corresponding to the virtual machine according to the combination of the virtual machine and the server calculated by the combination calculating step;
Is executed.

  According to the present invention, a plurality of virtual machines can be efficiently operated on a plurality of servers.

Hereinafter, a virtual machine system according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows the configuration of a virtual machine system according to an embodiment of the present invention. This virtual machine system includes a client terminal 1, a virtual machine management server 2, a virtual machine server 3, and a file server 4. The virtual machine management server 2, the virtual machine server 3, and the file server 4 are connected by a network 10 such as the Internet. The client terminal 1 is connected to the virtual machine management server 2 via a network such as a LAN.

  The client terminal 1 is a terminal operated by an administrator, and includes a computer including a control unit, a storage unit, a display unit, an input unit, a communication control unit, and the like. The control unit of the client terminal 1 is connected to the virtual machine management server 2 by reading and executing an operation program stored in the storage unit, and transmits various requests, and receives and displays data according to the transmitted requests. And so on.

  For example, as shown in FIG. 2, the virtual machine management server 2 includes a computer including a storage unit 21, a control unit 22, a communication control unit 23, and the like.

  The storage unit 21 is composed of, for example, a hard disk device. The storage unit 21 stores an operation program to be executed by the control unit 22 and various data necessary for processing. The storage unit 21 includes a VM server management DB 211, a file server management DB 212, a VM management DB 213, a VM image file management DB 214, an operation status management DB 215, and the like.

  The VM server management DB 211 stores information related to each virtual machine server 3 (virtual machine server information). For example, as shown in FIG. 3, the virtual machine server information includes a virtual machine server name, a host OS name, a virtual machine software name, an IP address, a CPU cycle number, a memory capacity, a hard disk capacity, a hard disk IO bandwidth, and a network IO bandwidth. , Including benchmark results, virtual machine IDs of virtual machines running on the virtual machine server, and the like.

  The file server management DB 212 stores information (file server information) regarding each file server 4. The file server information includes data such as an IP address, a server name, and a hard disk capacity as shown in FIG.

  The VM management DB 213 stores information (virtual machine information) related to virtual machines that run on each virtual machine server 3. For example, as shown in FIG. 5, the virtual machine information includes a virtual machine ID, a user name, an OS name, a CPU cycle, a memory capacity, a hard disk capacity, an IP address, a performance ID for specifying the performance data of the virtual machine, and the like. Contains data.

  The VM image file management DB 214 stores information (image file information) related to the image file of the virtual machine (the contents of the disk and the memory as a file). For example, as shown in FIG. 6, the image file information includes an image file name (model name), an OS name, a memory capacity, a hard disk capacity, each software name, an IP address of a file server that stores the image file, and the image file storage. Includes data such as the previous file path.

  The operating status management DB 215 stores actual performance data (performance data) acquired for each virtual machine at predetermined time intervals (for example, every hour) for a predetermined period (for example, for one month). In the present embodiment, for example, as shown in FIG. 7, performance data for one month acquired every hour (0 hour, 1 hour,..., 23:00) is stored for each virtual machine ID. And

  The control unit 22 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and controls the entire virtual machine management server 2. The control unit 22 reads out and executes an operation program or the like stored in advance in the storage unit 21, thereby causing a VM (Virtual Machine) server management unit 221, a VM optimum arrangement unit 222, a VM automatic creation unit 223, and the like. Realize logically.

  When receiving the virtual machine list request from the client terminal 1, the VM server management unit 221 obtains the IDs (virtual machine IDs) of all virtual machines arranged in each virtual machine server 3 from the VM server management DB 211 of the storage unit 21. read out. Then, the virtual machine information corresponding to each read virtual machine ID is read from the VM management DB 213, screen data for displaying all or a part of the read virtual machine information is generated, and transmitted to the requesting client terminal 1.

  The VM optimal arrangement unit 222 calculates a combination of the virtual machine server 3 and the virtual machine that maximizes the performance of each virtual machine based on the performance data of each virtual machine stored in the operation status management DB 215, Based on the calculated combination, optimal placement processing for rearranging each virtual machine is performed. Normally, when a user uses a virtual machine, it is assumed that the time zone to be used is different for each user. Therefore, in this optimum placement process, an optimum placement that focuses on the time zone in which each virtual machine operates is different. Is what you do.

Specifically, first, in order to obtain a combination of a virtual machine server and a virtual machine that maximizes the performance of each virtual machine, for example, all virtual machines M _i (i = 1) registered in the VM management DB 213 are used. ,..., N) are arranged on each virtual machine server S _j (j = 1, 2,..., J), the performance a _{i of} as many virtual machines M _i as possible is possible. The combination that becomes the maximum at as many times t (t = 1, 2,..., 23 (hours)) is calculated according to the calculation methods shown in the following (1) to (3). Note that the hard disk capacity of the virtual machine M _i is h _i (i = 1, 2,...), The memory capacity is r _i (i = 1, 2,...), And the hard disk capacity of the virtual machine server S _j is H _j (j = 1, 2,...), The memory capacity is R _j (j = 1, 2,...), And the performance is A _j (j = 1, 2,...). Data is stored in the storage unit 21. The performance A _j is, for example, a benchmark test execution result. _{Further, x ij} is for indicating the presence or absence of virtual machine placement _{M i} in the virtual machine server _{S j,} and _{x ij} = 1 in the case of arranging the virtual machine _{M i} to a virtual machine server _{S j,} do not place In this case, x _ij = 0.
Y _ij indicates ON / OFF of the power of the virtual machine M _i in the virtual machine server S _j . When the power of the virtual machine M _i is turned on, Y _ij = 1 and is not included. In this case, Y _ij = 0. Further, it is assumed that the performance a _i during operation of the virtual machine M _i after the rearrangement is _{equal to} or higher than the preset minimum performance a _{min of the} virtual machine.

(1) On the virtual machine server S _j (j = 1, 2,...), The number n _jt of virtual machines operating at each time (time) t is _used as the actual measurement data of the performance of the virtual machine at each time. Calculate based on Details of the processing for calculating the number of operating machines n _jt will be described later.
(2) (6) when t satisfies the condition virtual machine _{M i} of equation (t = 0,1, ···, 23 ) determined by the performance _{a it} (number 1) in.
(3) Under the conditions of (Equation 1), (Equation 3), (Equation 4), and (Equation 5), the combination that optimizes the value of (Equation 2) is combined and an optimization algorithm is used. Use to calculate. The combination optimization algorithm to be used is arbitrary. For example, a genetic algorithm or a simulated annealing method may be used.

（数２）

（数３）

（数４）

（数５）

（数６）

(Equation 1)

(Equation 2)

(Equation 3)

(Equation 4)

(Equation 5)

(Equation 6)

The VM optimum arrangement unit 222 uses the file server 4 as a temporary area based on the combinations calculated according to the calculation methods shown in the above (1) to (3), etc. The virtual machines are rearranged by copying them to the storage area of the virtual machine server 3 corresponding to.
Further, the VM optimum arrangement unit 222 receives performance data from each virtual machine server 3 and records it in the operation status management DB 215.

  The VM automatic creation unit 223 performs a process of automatically creating a virtual machine in response to a virtual machine creation request from the client terminal 1. Specifically, when a virtual machine creation request is received from the client terminal 1, the image file information registered in the VM image file management DB 214 is read out, and screen data for displaying all or part of the image file information is generated and the client is created. The input of the image file name by the user (administrator) is received by transmitting to the terminal 1 or the like. For the image file having the input image file name, data such as the IP address of the file server 4 of the image file storage destination and the file path of the image file storage destination is acquired from the image file information, and the selected image file The copy destination virtual machine server 3 is determined, the corresponding image file is read from the file server 4 and copied to the virtual machine server 3.

  The communication control unit 23 is for performing data communication with other computers via the network 10.

The virtual machine server 3 is a server for operating a virtual machine, and includes a computer including a control unit, a storage unit, a communication control unit, and the like.
The control unit includes a CPU, a ROM, a RAM, and the like, and logically implements a plurality of virtual machines by reading and executing various programs stored in advance in a storage unit such as a hard disk. An example of a schematic diagram of the logical configuration of the virtual machine server 3 is shown in FIG. As shown in the figure, software (virtual machine control program) for controlling a virtual machine is provided on an OS (host OS) for controlling a physical machine (CPU, memory, hard disk, etc.). The program divides the resources (processor, memory, hardware, etc.) of the virtual machine server 3 and realizes each virtual machine that uses the divided resources independently. The storage unit (allocated memory or hard disk) of each virtual machine stores various programs such as OS and applications, and various data such as virtual machine IDs. Each virtual machine executes a program (performance acquisition program) for acquiring performance data at predetermined time intervals (one hour interval in this embodiment), and the execution result (performance data) is a virtual machine ID and time data (execution time, etc.) ) And the like are transmitted to the virtual machine management server 2. The performance data may be data representing the performance of the virtual machine, and the content thereof is arbitrary. For example, the CPU operation rate may be used, and the time until the result is output after executing a predetermined calculation process may be used. Good.

  The file server 4 includes a computer that includes a control unit, a storage unit, a communication control unit, and the like. The storage unit of the file server 4 stores a virtual machine image file, and reads and transmits the designated image file in response to a request from the virtual machine management server 2.

Next, the operation of the virtual machine system according to this embodiment will be described.
First, an automatic creation process for automatically creating a virtual machine will be described with reference to a flowchart of FIG.
First, the control unit 22 of the virtual machine management server 2 displays image file information registered in the VM image file management DB 214 and specifies an image file in response to a virtual machine creation request from the client terminal 1. Data of a screen for accepting input of information (for example, an image file name) is supplied to the client terminal 1, and data such as an image file name input by a user (administrator) is received (step S1). Then, the image file information corresponding to the input image file name and the like is read from the VM image file management DB 214, and the IP address, file path, and the like of the file server 4 where the designated image file is stored are acquired (step S2). ).

  Next, the control unit 22 initializes the virtual machine server table (step S3). The virtual machine server table is a server determination table in which the IP address of the virtual machine server 3 and candidate flags corresponding to each IP address are set. In this initialization process, the virtual machine server table is registered in the VM server management DB 211. The IP addresses of all existing virtual machine servers 3 are set in the table, and “0” is set in the candidate flags corresponding to all the IP addresses.

  Next, the control unit 22 randomly selects an IP address from the server determination table, searches the VM server management DB 211 for the virtual machine information of the virtual machine server 3 of the selected IP address, and refers to the virtual machine server 3. Whether or not there are any remaining resources that can copy the image file designated by the user (that is, whether or not the free space data of the hard disk of the virtual machine server 3 is acquired and the free space is larger than the size of the image file) ) Is determined (step S4). If it is determined that there are still resources available for copying the image file in the selected virtual machine server 3 (step S4: YES), the image file is copied to the virtual machine server 3 (step S5). This process ends.

  If it is determined that there are no remaining resources available for copying the image file in the selected virtual machine server 3 (step S4: NO), “1” is set in the candidate flag corresponding to the virtual machine server 3. (Step S6), it is determined whether all the candidate flags in the server determination table are set to “1” (Step S7). If all the candidate flags are set to “1” (Step S7: YES), for example, “ An error message display screen such as “Add a virtual machine server” is transmitted to the client terminal 1 for display (step S8), and the process is terminated.

If all candidate flags in the server determination table are not set to “1” (step S7: NO), the process returns to step S4, the next virtual machine server 3 is determined at random, and the image file can be copied. The process of determining whether or not resources remain is repeated.
In the process of selecting the virtual machine server 3 in step S4, an IP address for which “1” is not set in the corresponding candidate flag is selected. Specifically, when the flag corresponding to the randomly selected IP address is “1”, the IP address may be discarded and the IP address may be selected at random again. An IP address in which “0” is set in the candidate flag to be extracted may be extracted first, and may be selected at random from the IP addresses.

Next, an optimal placement process for optimally placing a plurality of virtual machines on a plurality of virtual machine servers 3 will be described with reference to the flowchart of FIG.
First, the control unit 22 of the virtual machine management server 2 reads the performance data of each virtual machine from the operation status management DB 215 (step S11).
Next, the control unit 22 calculates the number n _jt of virtual machines operating at each time (time) t on the virtual machine server S _j (j = 1, 2,...) (Step S12). .

The details of the working machine calculation process for calculating the number n _jt of working machines will be described with reference to the flow of FIG.
The control unit 22 performs a predetermined initialization process (number of virtual machines running on the server S _j at time t n _jt = 0, time t = 0, virtual machine power supply Y _ij = 0, etc.) (steps) S21).
Then, one virtual machine ID in the virtual machine server information of the virtual machine server S _j registered in the VM server management DB 211 is extracted, and a performance ID corresponding to the extracted virtual machine ID is acquired from the VM management DB 213, The performance data specified by the performance ID is read from the operation status management DB 215 (step S22).

Then, based on the read performance data in step S22, d date of the virtual machine _{M i} (e.g., 1, 2, ..., 30) at t in (t = 0, 1, 2, ... , 23) is the performance data value P _idt , and the average value of the performance data for one month at each time (time) t is P _it , according to (Equation 7), the virtual machine M _i at time (time) t It calculates an average value _{P it} performance (step S23).

(Equation 7)

Then, the average value P _it calculated in step S23, it is determined whether or not a preset minimum performance value P _th or more (step S24), and if it is the lowest performance value P _th or more, the virtual Assuming that the machine M _i is operating, 1 is added to the number n _jt of operating machines, and Y _ij = 1 (step S25).

After the process of step S25, or when _it is determined in step S24 that the average value _{Pit is} smaller than the minimum performance value _Pth, all virtual machines registered in the virtual machine server information of the virtual machine server _Sj It is determined whether the process has been completed for the ID (step S26). If the process has not been completed, the process returns to step S22, the next unprocessed virtual machine ID is extracted, and the above process (the virtual machine M) is extracted. It is determined whether _i is operating, and if it is operating, a process of adding 1 to the number of operating machines n _jt is performed.
If it is determined in step S26 that the processing has been completed for all virtual machine IDs registered in the virtual machine server information, t = 23, that is, operation is performed for the entire time (0 to 23:00). It is determined whether or not the process of counting the number of machines n _jt has been completed (step S27). If it has not been completed (if t ≠ 23), 1 is added to t (step S28) and the process returns to step S23. The above processing is performed for the next time t.
If it is determined in step S27 that t = 23, this process ends.

The control unit 22 executes the above-described calculation process on each virtual machine server S _j (j = 1, 2,...), And operates on each virtual machine server S _j at each time (time) t. The number of virtual machines n _jt is calculated.

After calculating the number of virtual machines n _jt on each virtual machine server S _j , the control unit 22 performs t time (t = 0, 1, 1) of the virtual machine M _{i that} satisfies the above-described equation (6). ..., the performance a _it at 23), together with the obtained by previously shown (number 1), the combination of virtual machines and virtual machine server performance of each virtual machine is maximized, i.e., previously shown Under the conditions of (Equation 1), (Equation 3), (Equation 4), and (Equation 5), the combination that maximizes the value of (Equation 2) shown above is, for example, genetic A combination that maximizes the performance of each virtual machine is obtained by calculating using a combination optimization algorithm such as an algorithm or a simulated annealing method (step S13).

Then, the control unit 22 determines whether or not a combination that maximizes the performance of each virtual machine is obtained in step S13 (step S14). When the combination is obtained, the file server 4 is set as a temporary area. The virtual machine is rearranged by, for example, copying the image file of each virtual machine to the storage area of the virtual machine server 3 corresponding to the virtual machine (step S15), and in the VM server management DB 211 The virtual machine ID in each virtual machine server information is changed and registered, and this process ends.
If it is determined in step S14 that the combination solution has not been obtained, an error message display screen such as “Add a virtual machine server” is transmitted to the client terminal 1 and displayed. (Step S16), and this process is terminated.

  As described above, according to the present invention, a combination of a virtual machine and a virtual machine server 3 that uses the resources of all the virtual machine servers 3 to maximize the performance of each virtual machine is obtained. Since the virtual machines are arranged based on them, it is possible to optimally arrange virtual machines across a plurality of servers, and a plurality of virtual machines can be efficiently operated on a plurality of servers.

The present invention is not limited to the above embodiment, and various modifications and applications are possible.
For example, each of the above formulas is an example, and various formulas for obtaining a combination of a virtual machine and a virtual machine server that maximize the performance of each virtual machine can be applied. Further, in the above description, when calculating the combination of the virtual machine and the virtual machine server, the conditional expression related to the storage capacity of the virtual machine and the virtual machine server is used, but other performance (for example, hard disk IO bandwidth, network Conditional expressions relating to IO bandwidth etc. may be used.

The system of the present invention can be realized using a normal computer system, not a dedicated system. For example, a program for executing the above operation is stored in a computer-readable recording medium (FD, CD-ROM, DVD, etc.) and distributed, and the program is installed in the computer to execute the above processing. The virtual machine management server 2 or the like may be configured. Alternatively, it may be stored in a disk device of a server device on the network 10 such as the Internet and downloaded to a computer, for example.
In addition, when the OS realizes the above functions by sharing the OS or jointly of the OS and the application, only the part other than the OS may be stored and distributed in the medium, or may be downloaded to the computer. Good.

It is a figure which shows the structure of the virtual machine system of the 1st Embodiment of this invention. It is a figure which shows the structure of a virtual machine management server. It is a figure which shows an example of the data structure of virtual machine server information. It is a figure which shows an example of the data structure of file server information. It is a figure which shows an example of the data structure of virtual machine information. It is a figure which shows an example of the data structure of image file information. It is a figure which shows an example of the data structure of performance data. It is an example of the schematic diagram of the logical structure of a virtual machine server. It is a flowchart for demonstrating automatic creation processing. It is a flowchart for demonstrating optimal arrangement | positioning processing. It is a flowchart for demonstrating an operating machine calculation process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Client terminal 2 Virtual machine management server 3 Virtual machine server 4 File server 10 Network 21 Storage part 211 VM server management DB
212 File server management DB
213 VM management DB
214 VM image file management DB
215 Operation status management DB
DESCRIPTION OF SYMBOLS 22 Control part 221 VM server management part 222 VM optimal arrangement | positioning part 223 VM automatic creation part 23 Communication control part

Claims (4)

A virtual machine management apparatus connected via a network to a plurality of servers that operate a plurality of virtual machines,
In the first predetermined period, for each virtual machine, an operation information management database that stores actual measurement data indicating performance acquired every predetermined time;
Reading means for reading actual measurement data indicating the performance of each virtual machine every predetermined time from the operation information management database;
Using the actual measurement data indicating the performance of each read virtual machine for each predetermined time, the virtual machine at each time when each virtual machine is operated on any of a plurality of servers for each predetermined time . A combination calculation unit that calculates a performance value , and calculates a combination of the virtual machine and the server in which the performance value of the virtual machine for each determined predetermined time is maximized in a second predetermined period in which a plurality of the predetermined times are collected ; ,
Means for relocating each virtual machine by storing a file of each virtual machine in a storage area of the server corresponding to the virtual machine according to the combination of the virtual machine and the server calculated by the combination calculating means;
A virtual machine management apparatus comprising: The reading means further reads data relating to the storage capacity of each server and data relating to the storage capacity of each virtual machine from the operation information management database,
The combination calculation means is the measurement data read by the reading means and indicating the performance of each virtual machine every predetermined time, the data relating to the storage capacity of each server, the data relating to the storage capacity of each virtual machine, To calculate the combination of the virtual machine and the server using
The virtual machine management apparatus according to claim 1 . Means for receiving a virtual machine creation request;
Means for determining a server capable of storing the virtual machine file indicated by the creation request in the storage area based on the use amount of the storage area of each server and the file capacity of the virtual machine indicated by the received creation request When,
Means for storing a virtual machine file indicated by the creation request in the storage area of the determined server;
Virtual machine management system according to claim 1 or 2, characterized in that it comprises a. To a computer connected to multiple servers running multiple virtual machines via a network,
A storage step of storing, in the operation information management database, actual measurement data indicating performance acquired at predetermined time intervals for each virtual machine in the first predetermined period;
A step of reading out actual measurement data indicating the performance of each virtual machine every predetermined time from the operation information management database;
At each time when each virtual machine is operated on one of a plurality of servers for each predetermined time , using the measured data indicating the performance of each virtual machine for each predetermined time read by the reading step. A performance value of each virtual machine is obtained, and a combination of a virtual machine and a server is calculated such that the performance value of the virtual machine at the obtained predetermined time becomes the maximum in a second predetermined period in which a plurality of the predetermined times are collected. Combination calculation step,
Relocating each virtual machine by storing the file of each virtual machine in the storage area of the server corresponding to the virtual machine according to the combination of the virtual machine and the server calculated by the combination calculating step;
A program that executes

Images