WO2012137272A1 - リソース管理方法および管理サーバ - Google Patents
リソース管理方法および管理サーバ Download PDFInfo
- Publication number
- WO2012137272A1 WO2012137272A1 PCT/JP2011/058390 JP2011058390W WO2012137272A1 WO 2012137272 A1 WO2012137272 A1 WO 2012137272A1 JP 2011058390 W JP2011058390 W JP 2011058390W WO 2012137272 A1 WO2012137272 A1 WO 2012137272A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- server
- resource
- trial
- job
- information
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/70—Admission control; Resource allocation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
- G06F11/3428—Benchmarking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/815—Virtual
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5019—Workload prediction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/508—Monitor
Definitions
- the present invention relates to a resource management method and a management server that can effectively use resources on a network.
- Resource group arrangement management is greatly related to cost reduction required for resource group maintenance and improvement of server performance provided to users.
- a method based on operations in a virtual machine system is disclosed as resource group arrangement management (see, for example, Patent Document 1).
- the virtual computer system has a plurality of virtual devices that share the physical resources of the computer and execute an application, a virtualization unit that manages the plurality of virtual devices, and a management unit that controls the virtualization unit.
- resource supply information for example, resource allocation amount
- resource request information between the plurality of virtual devices and the management means Perform dynamic resource allocation of physical resources.
- the arrangement of resource groups that enable the required performance does not depend on the calculation method, does not prepare in advance, is not limited as much as possible to the required time for improvement, and also has a mechanism to perform at any time with available information I think it is necessary for managed resources.
- the present invention is an invention for solving the above-described problems, and an object thereof is to provide a resource management method and a management server that can effectively use resources on a network.
- the management server in a computer system in which a plurality of servers are connected to a network, the management server combines resources that can be secured in the network and uses a trial server (for example, a trial server).
- a trial server for example, a trial server
- a physical server 100b and a trial virtual server 130b are formed, and the trial server is applied as a server of the network when the trial server satisfies a predetermined condition.
- the management server employs resource information (for example, the resource information table 211) in which extraction probabilities are registered as results of application information when the trial server is applied to the resources that can be secured on the network, and the trial server.
- Job management information (for example, job management table 215) for managing jobs, which is a predetermined condition as to whether or not to perform, is stored in the storage unit. If the extraction probability is greater than or equal to the random probability generated by the random number generator for each resource in the resource information, the management server selects the trial resource, and combines the selected resources as a trial server. Forming the trial server and executing the job using the trial server thus formed. If the result of the execution is satisfactory, the trial server is applied as a server of the network.
- resources on the network can be used effectively.
- FIG. 1 is an overall configuration diagram showing a network system according to the present invention. It is detail drawing which shows the structure of a management server. It is a detailed view showing a configuration of a physical server to which a virtual server and a trial virtual server are applied.
- FIG. 4 is a detailed diagram illustrating a relationship between a physical server to which the virtual server and the trial virtual server illustrated in FIG. 3 are applied and a disk device.
- FIG. 4 is a detailed diagram illustrating configurations of a physical server and a trial physical server. It is explanatory drawing which shows the relationship of the resource rearrangement when the physical server for trial and the virtual server for trial are employ
- (a) is a figure which shows an example of a resource information table
- (b) is a figure which shows the example of the utilization rate graph of a resource information table.
- FIG. 1 is an overall configuration diagram showing a network system according to the present invention.
- a management server 200 one or more physical servers 100, an I / O switch expansion device 160, and a bus connection control device 150 are connected via a network switch 140.
- a storage device 180 having a disk volume 190 is connected to the physical server 100 via an I / O switch expansion device 160 and an I / O device 170.
- the physical server 100 includes a physical server having a virtualization mechanism unit 131 that provides the virtual server 130.
- the management server 200 is the center of control for improving resource allocation in order to effectively use a group of resources in the network system.
- a resource group is a general term for resources, and means a combination of components (for example, a processor, a memory, an HDD, an FC (Fibre Channel), a LAN, etc.) constituting a server.
- the management server 200 collects information on resources on the network from the physical server 100, combines unused resources of resources that are not allocated, and uses a trial server (trial physical server 100b, trial resource). Virtual server 130b) is formed and applied as a server on the network when the trial server satisfies a predetermined condition.
- the trial server is a general expression including the trial physical server 100b and the trial virtual server 130b.
- the management server 200 has a resource information table 211 (see FIG. 7) in which an extraction probability Ep (Extraction probability) is registered as a record of application information when the trial server is applied to resources that can be secured on the network.
- a job management table 215 (see FIG. 11) for managing jobs, which is a predetermined condition as to whether or not to adopt a trial server, is stored in the memory 201 (storage unit) (see FIG. 2).
- the management server 200 has an extraction probability Ep equal to or higher than the random number probability Rp (Random numbers probability) generated by the random number generator 221 (see FIG. 2) for each resource in the resource information table 211 (see FIG. 7). For example, a resource is selected as a resource, a trial server is formed by combining the selected resources, and a predetermined job is executed using the formed trial server. It can be applied as a server above.
- the bus connection control device 150 executes, for example, a performance expansion method by SMP (Symmetric Multi Processor) connection between physical servers 100 (for example, between server blades).
- SMP connection is a technique for expanding performance by combining a plurality of server blades into one to create a higher performance server.
- the bus connection control device 150 allocates the memory 101 and the processor 102 to the physical server 100 (physical server 100a, trial physical server 100b, virtual server physical server 100c).
- FIG. 2 is a detailed diagram showing the configuration of the management server.
- the management server 200 includes a memory 201 (storage unit), a processor 202 (processing unit), a user I / F 203, a network I / F 204, and a disk I / F 205.
- the memory 201 includes a resource arrangement improving unit 206 (see FIG. 16) and an information collecting unit 210 (see FIG. 15).
- the various tables include a resource information table 211 (see FIG. 7), and improved resource information.
- Table 212 (see FIG. 8), history table 213 (see FIG. 9), server relation table 214 (see FIG. 10), job management table 215 (see FIG. 10), job result management table 216 (see FIG. 12), improvement It consists of a recording table 217 (see FIG. 13) and an improvement designation / job relationship table 218 (see FIG. 14).
- the resource allocation improvement unit 206 includes a random number generation unit 221, a resource allocation / trial improvement unit 207 (see FIG. 17), and an improvement allocation application unit 209 (see FIG. 19).
- the resource allocation / trial improvement unit 207 includes a server It has the trial improvement part 208 (refer FIG. 18) including the group production
- the user I / F 203 is an I / F for inputting a user request to the management server 200 when each table in the memory 201 is updated based on the user definition.
- a network I / F 204 is an I / F that connects each server to the network.
- the disk I / F 205 is an I / F that connects each server to the disk volume 190 (see FIG. 1).
- the resource allocation improvement unit 206 can improve resource allocation by a trial method or a calculation method.
- the resource extraction probability Ep is set by a trial method such as new / improvement / maintenance / addition / reduction / fault identification, and then the resource securing / trial improvement unit 207 is called.
- the resource securing / trial improving unit 207 investigates the usage rate of each resource from a group of servers connected to the network as much as possible, secures trial resources, and calls the trial improving unit 208.
- the trial improvement unit 208 forms a trial server with the resources that have been secured and executes the job, and also executes it on the current server, and compares the job result between the current server and the trial server to confirm improvement of the server group. To do.
- the success / failure of the improvement record is updated with the result, and the improvement placement application unit 209 applies the generated trial server or trial virtual server if the improvement is improved.
- the random number generation unit 221 is a random number generation unit, and the generated random probability Rp (random number value) is used to compare with the extraction probability Ep of the resource information table 211 (see FIG. 7) when selecting a resource. It is done. For the random value, for example, a minimum value and a maximum value are given, and a random number is generated between them.
- Each resource in the resource information table 211 (see FIG. 7) stores an extraction probability Ep, and is set so that the extraction probability Ep of any resource does not become 0%. If the extraction probability Ep of each resource is equal to or greater than the random value generated by the random number generation unit 221, the resource is adopted as a trial resource. It is also a feature of the present invention that the random number generation unit 221 generates a random number for each resource every time.
- the extraction probability Ep of the first resource is 37.5
- a random number value 12 is obtained when an integer random number is generated with a minimum value 0 and a maximum value 99. Then, 12 ⁇ 37.5, and the target resource is adopted. If the extraction probability Ep of the next second resource is 12.5, a random number is generated every time instead of comparing with the previously generated random value 12.5. If the random number value for the second resource is 14, 14> 12.5, and the second resource is not adopted.
- the resource extraction probability Ep is a trial with an initial value of 50%. Each time a trial is performed, the extraction probability Ep in the resource placement improvement unit 206 is changed (see FIG. 16, steps S1206 to S1209). It is updated and becomes a value between 0% ⁇ extraction probability Ep ⁇ 100%.
- the method of this embodiment creates a server by randomly extracting unallocated resources.
- the server performance is evaluated by job. For this reason, there is a possibility that a combination of resources that has not been conceived in the past can be made.
- there is no room for contingency regarding resource selection but in this embodiment, there is a possibility that contingency may be included in resource selection.
- produced in the random number generation part 221 is used as one of the means to implement
- FIG. 3 is a detailed diagram showing the configuration of a physical server to which a virtual server and a trial virtual server are applied.
- FIG. 4 is a detailed view showing the relationship between a physical server to which the virtual server and the trial virtual server shown in FIG. 3 are applied and a disk device.
- the physical server 100c (100) includes a memory 101, a processor 102, a user I / F 103, a network I / F 104, and a disk I / F 105.
- the memory 101 includes a virtualization mechanism unit 131 having a virtual allocation information acquisition unit 313 (see FIG. 23).
- the virtualization mechanism unit 131 includes a plurality of virtual servers 130 (current virtual server 130a, on the network).
- a trial virtual server 130b for trying whether to apply as a server is included.
- the OS (Operating System) 301 of the memory 101 includes a comparison information acquisition unit 310 (see FIG. 20), a server relation information acquisition unit 311 (see FIG. 21), and a resource information acquisition unit 312 (see FIG. 23).
- a storage device 180 composed of a plurality of disk volumes 190 is connected to the physical server 100c.
- the disk volume 190 has a plurality of virtual disks 191 via a virtual I / O device 171 (see FIG. 4).
- the storage apparatus 180 may be a server built-in type or an external apparatus via a fiber channel or the like.
- the virtualization mechanism unit 131 sends the processor 102, the memory 101, the virtual disk 191, and the virtual I / O device 171 to the virtual server 130a and the trial virtual in response to an instruction from the resource allocation improvement unit 206 (see FIG. 2) of the management server 200. Assign to server 130b.
- the resource allocation improvement unit 206 (see FIG. 2) of the management server 200 changes the setting of the virtualization mechanism unit 131, whereby the allocated resource is changed.
- the trial virtual server 130b is prepared according to the extraction probability Ep of the resource information table 211 (see FIG. 7) for comparison with the current virtual server 130a by the trial improvement unit 208 (see FIG. 2) of the management server 200. It is a virtual server.
- FIG. 5 is a detailed diagram showing the configuration of the physical server and the trial physical server.
- FIG. 5 shows the configuration of the trial physical server 100b for trying whether to apply the current physical server 100a shown in FIG. 1 as a server on the network.
- the OS 301 of the memory 101 includes a comparison information acquisition unit 310 (see FIG. 20), a server relation information acquisition unit 311 (see FIG. 21), and a resource information acquisition unit 312 (see FIG. 23).
- FIG. 6 is an explanatory diagram showing the relationship of resource rearrangement when the trial physical server and the trial virtual server are adopted for improvement by trial. This will be described with reference to FIG.
- the information collection unit 210 (see FIG. 15) of the management server 200 acquires the unassigned resource group / unassigned resource group unused unit 108.
- the resource allocation improvement unit 206 creates the trial physical server 100 b on the unassigned resource group / unassigned resource group unused unit 108 and the trial virtual server 130 b on the unassigned virtual resource group 109.
- the performance comparison between the created trial physical server 100b and the current physical server 100a is performed, and similarly, the performance comparison between the created trial virtual server 130b and the current virtual server 130a is performed.
- the improvement allocation application unit 209 sends the trial physical server 100b to the allocated resource group use unit 106.
- the trial virtual server 130b and the resources allocated to them are moved to the allocated virtual resource group 107, and the current physical server 100a and virtual server 130a and the resources allocated to them are moved to the unallocated resource group / allocation.
- the resource allocation is improved by moving to the unused portion 108 of the used resource.
- the active physical server 100a and virtual server 130a shown in FIG. 6 are servers created by selecting resources that satisfy the conditions set by the administrator, whereas the trial physical server 100b and trial virtual server 130b are A server that created a server by selecting resources by random sampling.
- the trial physical server 100b has a configuration in which the CPU (Central Processing Unit) employs 3 GHz, but the performance is improved over the configuration in which the CPU of the current physical server 100a employs 3 GHz. It turns out to happen to happen.
- the desired performance may not be achieved with the assumed combination of resources, but in this embodiment, a combination of resources that the administrator does not expect can be realized.
- FIG. 7 is an explanatory diagram illustrating an example of a resource information table and a usage rate graph.
- FIG. 7A shows an example of the resource information table 211
- FIG. 7B shows an example of the usage rate graph 707 of the resource information table 211.
- the resource information table 211 illustrated in FIG. 7A is held by the management server 200 and is updated based on information collected by the information collection unit 210 and input information from the user I / F 203.
- the resource information table 211 includes a resource name 701, a type 702, a virtual relationship 703, a logical / unique value setting 704, a physical position 705, an allocation state 706, a usage rate graph 707, a failure presence / absence 708, trial information 709, and an extraction probability Ep (710). ).
- the type 702 stores “FC” representing a fiber channel device.
- the virtual relationship 703 stores information on the parent device that provides the virtual FC device “virtual FC2”.
- the logical / unique value setting 704 stores “WWN (World Wide Name) information” used for external storage allocation and the like.
- the physical location 705 stores “I / O expansion device 1” of the device information on which “FC1” is mounted.
- the allocation state 706 is “Srv1” currently allocated
- the usage rate graph 707 is “Graph1” in which the usage rate (band in the case of FC) is recorded in the past time
- the failure status 708 is user designation or failure “None” indicating that a failure has not been determined so far due to a specific trial
- “Trial” in the trial information 709 is the type of trial for which the resource was the last target (duplication with trials with the same improvement ID) In order to prevent the investigation, the number of times of the investigation target resource of the trial is recorded in parentheses.)
- FC1 becomes an assignable resource in each trial in the extraction probability Ep (710). Represents a probability used for allocation to a server created by the server group generation unit 220.
- the extraction probability Ep (710) is set to 50% as an initial value, and is updated by a process of changing the extraction probability Ep in the resource placement improvement unit 206 every time a trial is performed.
- the extraction probability Ep is a value between a value greater than 0% and 100%, and is a probability that the server group generation unit 220 determines whether to use resources.
- the usage rate graph shown in FIG. 7B is a graph showing the usage rate of the CPU, for example, and shows that the usage rate changes with time.
- the probability used for allocation to the server created by the server group generation unit 220 is specifically the extraction probability Ep (710) and random number generation for the resources in the resource information table 211. If the extraction probability Ep is equal to or greater than the probability generated by the random number generation unit 221 by comparing with the probability generated by the unit 221, it means that the corresponding resource is applied to the server to be created.
- FIG. 8 is an explanatory diagram showing an example of the improved resource information table.
- the improved resource information table 212 is held by the management server 200 and is updated based on information on the trial physical server 100b and the trial virtual server 130b adopted by the resource securing / trial improvement unit 207 (see FIG. 2).
- the improved resource information table 212 includes a resource name 801, a type 802, a virtual relationship 803, a logical / unique value setting 804, a physical location 805, an allocation state 806, and a failure presence / absence 808.
- the resource name 801, type 802, virtual relationship 803, logical / unique value setting 804, physical location 805, allocation state 806, and failure presence / absence 808 shown in FIG. 8 are the resource name 701, type 702, virtual relationship 703 shown in FIG. This corresponds to logical / unique value setting 704, physical location 705, allocation state 706, and failure presence / absence 708.
- FIG. 9 is an explanatory diagram showing an example of a history table.
- the history table 213 includes a resource information update date / time 901, a changed resource number 902, and a changed relation number 903, which are the date / time when the information collection unit 210 (see FIG. 2) updated the resource information.
- the number of changed resources 902 is the frequency with which the resource information table 211 is changed
- the number of changed relationships 903 is the frequency with which the server relationship table 214 (see FIG. 10) is changed.
- the resource allocation improvement unit 206 refers to the history table 213 in the improvement method collection processing in step S1201 of FIG. 16. If the frequency of change is high, the resource allocation improvement unit 206 improves the resource allocation by trial. If the frequency is low, the resource allocation improvement improves by calculation. It may be helpful to select an improvement method.
- FIG. 10 is an explanatory diagram showing an example of a server relationship table. This will be described with reference to FIGS. 3 and 5 as appropriate.
- the server relationship table 214 is information collected by the information collection unit 210, that is, information obtained from the server relationship information acquisition unit 311, the resource information acquisition unit 312, and the virtual allocation information acquisition unit 313, or the user I / F 203. It is created based on the information input from.
- the server relation table 214 includes a server name 1001 of each server, a resource sharing 1002 indicating a server sharing a resource, a mutual communication 1003 indicating a server performing mutual communication, and a startup indicating a server having a similar startup time. The time 1004 is included.
- FIG. 11 is an explanatory diagram showing an example of a job management table.
- the job management table 215 is a job content table used when the trial improvement unit 208 (see FIG. 2) compares servers in operation.
- the job management table 215 includes a job ID 1101, a job content 1102, a comparison value 1103, and a determination method 1104 used for comparison.
- job A is executed and adopted if it can be executed without any problem.
- job B is executed, the time is measured, and if the execution time is shorter than the comparison value of 5 seconds, it is adopted.
- job 3 if data can be saved and saved, it is adopted.
- job 4 if job C is executed and can be executed without any problem on all the related server groups, it is adopted.
- job 5 if a communication test such as PING is performed and communication is possible with all the related server groups, the job is adopted.
- an application program is executed and a specific application program can be operated without error.
- job 7 when a memory load job is executed and a result better than load result 1 is recorded, the job 7 is adopted.
- FIG. 12 is an explanatory diagram showing an example of a relationship table between improvement designations and jobs.
- the improvement specification / job relationship table 218 includes an improvement specification 1201 and a job ID 1202 that are specified when resource allocation is improved.
- the job ID 1202 shown in FIG. 12 corresponds to the job ID 1101 shown in FIG.
- the management server 200 sets the job ID 1202 corresponding to the improvement designation 1201 to the trial physical server 100b and the trial virtual server 130b. To execute.
- the management server 200 executes the job ID 1202 corresponding to the improvement designation 1201 for the current physical server 100a and the current virtual server 130a to be compared. That is, the trial improvement unit 208 of the management server 200 designates the comparison information acquisition unit 310 and acquires the execution result.
- job 1 is executed, and it is determined whether or not the job 1 can be executed without any problem with reference to the job management table 215 of FIG.
- job 2 is executed, and it is determined whether or not the execution time can be shortened compared to the comparison value with reference to the job management table 215 of FIG.
- FIG. 13 is an explanatory diagram showing an example of a job result management table.
- the job result management table 216 is updated with information collected by the trial improvement unit 208 of the management server 200, and holds a combination of resources, jobs, and job result histories for which the comparison information acquisition unit 310 has executed a job with each improvement ID. .
- the job result management table 216 includes an improvement ID 1301, a server group 1302, a resource 1 (1303), a resource 2 (1304), a resource 3 (1305), a resource 4 (1306), a job ID 1307, and a job result 1308.
- the job result management table 216 includes an improvement ID 1301, a server group 1302, a resource 1 (1303), a resource 2 (1304), a resource 3 (1305), a resource 4 (1306), a job ID 1307, and a job result 1308.
- the job result management table 216 includes an improvement ID 1301, a server group 1302, a resource 1 (1303), a resource 2 (1304), a resource 3 (1305), a
- the server of “Srv3” is composed of a combination of resources of CPU1, Mem1, HDD1, and LAN1, and job 3 and job 6 are executed. You can see that it is successful.
- the virtual server of “VSrv2” is configured by a combination of resources of VCPU2, Vmem2, VHDD2, and VFC2, and it can be understood that job 2 was executed but could not be executed due to time-out.
- FIG. 14 is an explanatory diagram showing an example of the improvement record table.
- the improvement record table 217 is a table for recording improvement when the trial improvement unit 208 performs improvement by trial.
- the improvement record table 217 includes an improvement ID 1401, an improvement designation 1402 that determines the type of trial, an adopted configuration 1403 that stores improvement targets for the entire server / resource group to be managed, an information collection policy 1404, a server group 1405 to be improved, It includes a server group relationship 1406, success / failure 1407, trial excess judgment 1408, such as an estimated time to stop improvement by trial and the number of repetitions, and the number of trial repetitions 1409.
- “Test1” that is an improvement ID 1401 is generated by the resource allocation improvement unit 206 in the update record update process in step S1202 of FIG.
- the improvement designation 1402 indicates “maintenance, Srv3”. Is stored.
- the trial improvement unit 208 performs a call of the comparison information acquisition unit 310 in step S ⁇ b> 1403 of FIG. 18 from the improvement specification / job relationship table 218 (see FIG. 12). A job to be executed by the information acquisition unit 310 is determined.
- “Minimum resource” is stored by user designation.
- the information of the adopted configuration 1403 is obtained when the trial improvement unit 208 in the server group created by the server group generation unit 220 finds a plurality of improved servers by comparison processing with the active server in step S1404 in FIG. Used for selection.
- the information collection policy 1404 is stored as designated by the user (for example, “periodic”).
- the information collection policy 1404 may be automatically set when the information collection unit 210 next performs information acquisition policy setting processing in step S1101 (see FIG. 15). In this case, it is preferable to use the policy that has been adopted at the time of improvement having “success” in the success / failure 1407 of the improvement record table 217 from the column of the information collection policy 1404 and adopt it.
- a related server is stored from the server relation table 214 (see FIG. 10), and when a resource is specified, the resource information table 211 is used.
- the server to which the resource is allocated is specified, and related servers are stored from the server relationship table 214.
- “Srv1, Srv2” whose relationship is described in the row “Srv3” of the server relationship table 214 is stored together with “Srv3”.
- the information of the server group 1405 is used throughout, for example, in order to speed up the convergence time until sizing of each part and acquisition of improvement results.
- the server of the server group 1405 is reduced when the migration target is deleted based on the information in the server relation table 214.
- the server group relation 1406 records the relation item of the server relation table 214 referred to when the server is stored in the server group 1405.
- “intercommunication” and “start-up time” that have values in the “Srv3” row of the server relation table 214 are stored.
- the success / failure 1407 stores information indicating whether the improvement with the improvement ID 1401 has succeeded or failed by the success / failure update process of the improvement record in step S1214 of FIG.
- the trial improvement unit 208 stores “timed out (required time 107h)” in the time limit / number of times processing of step S1406 in FIG.
- the excess determination 1408 stores a user designation, a default time, or the number of times. In the case of Test1, since it is set to 100h (100 hours) and the improved arrangement is not found even if it exceeds 100h, it is determined that the trial improvement unit 208 has timed out in the time limit / number of times processing in step S1406 in FIG. The success / failure 1407 is updated with “time out”.
- the management server 200 executes “maintenance” trial of “Srv3” “periodically”.
- the management server 200 forms trial physical servers “Srv1”, “Srv2”, and “Srv3” in the server group 1405 by combining the resource groups, and tries to save the data of the job 3.
- the server to be migrated is selected in step S1506 (see FIG. 19) of the improvement placement application unit 209. Reduce and implement migration. Details will be described later.
- the management server 200 executes “improvement” trial of “VSrv1” by “monitoring”.
- the management server 200 forms practical virtual servers “VSrv1” and “VSrv2” in the server group 1405 by combining the resource groups, and tries to measure the time by executing the job B of the job 2, but it cannot be improved. I understand that.
- resource aggregation is designated as the adopted configuration 1403 and a plurality of resource arrangements that can be improved are found, this means that a configuration in which many resources at the same physical location are allocated is adopted.
- the management server 200 forms a trial physical server “Srv5”, executes job C of job 4, and it can be seen that the management server 200 has succeeded.
- the management server 200 forms three trial physical servers and executes job A of job 1 and it can be seen that it is successful.
- FIG. 15 is a flowchart showing processing of the information collecting unit. This will be described with reference to FIGS. 2, 3, and 5 as appropriate.
- the information collection unit 210 collects resource information from the relationship information acquisition unit 311, the resource information acquisition unit 312, and the virtual allocation information acquisition unit 313 of the servers arranged in each server.
- a process of arranging the server relation information acquisition unit 311, the resource information acquisition unit 312, and the virtual allocation information acquisition unit 313 is performed.
- a predetermined step for example, step S1102 and step S1106) is performed as long as the required time permits, and when the specified or user-specified time has been reached, the processing is rounded up and proceeds to the next processing.
- the information collection unit 210 uses the information collection policy 1404 for acquiring information from the input value from the user I / F 203 or the information collection policy 1404 in which the result in the success / failure column of the improvement record table 217 (see FIG. 14) tends to be successful.
- a policy (for example, periodically or by monitoring) is set (step S1101). Based on the set policy, the server relation information acquisition unit 311, the resource information acquisition unit 312, and the virtual allocation information acquisition unit 313 collect resource information.
- the information collection unit 210 detects a server (step S1102).
- the resource information acquisition unit 312 and the server relationship information acquisition unit 311 detect servers that have not been applied or the information acquisition policy has not been updated.
- the virtual allocation information acquisition unit 313 detects an unapplied physical server 100c (see FIG. 3).
- the information collection unit 210 arranges each information acquisition unit for the server detected in step S1102 (step S1103). Specifically, the server related information acquisition unit 311, the resource information acquisition unit 312, and if necessary the virtual allocation information acquisition unit 313 are pushed to the server detected by the information collection unit 210, reflecting the setting in step S 1101. install.
- the push installation means that the management server 200 remotely operates a target server on the network and installs necessary software.
- the information collection unit 210 determines the policy set in step S1101 (step S1104). In the case of resource monitoring (step S1104, monitoring detection), the setting is periodically performed to step S1106 (step S1104, periodic detection). ), And proceeds to step S1105.
- step S1105 the information collection unit 210 acquires periodically.
- the information collection unit 210 receives information periodically transmitted from the resource information acquisition unit 312, the server relation information acquisition unit 311, and the virtual allocation information acquisition unit 313.
- step S1106 the information collection unit 210 waits for a response from the acquisition unit.
- the information collection unit 210 receives information sent from time to time from the resource information acquisition unit 312, the server relation information acquisition unit 311, and the virtual allocation information acquisition unit 313.
- the information collection unit 210 updates the resource information table 211 (see FIG. 7) and the server relation table 214 (see FIG. 10) based on the information acquired in steps S1105 and S1106 (step S1107).
- the information collection unit 210 updates the history table 213 (see FIG. 9) (step S1108).
- the information collection unit 210 records the date and time updated in step S1107 and the number of resources updated in the history table 213, and returns to step S1101.
- FIG. 16 is a flowchart showing processing of the resource arrangement improvement unit.
- FIG. 16 shows details of processing of the resource arrangement improvement unit 206.
- the resource arrangement improvement unit 206 collects improvement methods (step S1201). Specifically, the resource arrangement improvement unit 206 checks the resource information update date / time 901, the changed resource number 902, and the changed server change relationship number 903 in the history table 213 (see FIG. 9). Further, the presence / absence of a period in which the operation of all resources is stopped is checked from the usage rate graph of the resource information table 211 (see FIG. 7). It is confirmed whether or not the improvement method is designated by the user I / F 203.
- the resource allocation improvement unit 206 updates the improvement record (step S1202).
- the resource allocation improvement unit 206 creates a new improvement ID line in the improvement record table 217 (see FIG. 14), specifies the improvement specified in step S1201 (for example, calculation or trial), and adoption criteria (in the adoption configuration 1403). Stored).
- step S1201 if the number of changed resources 902 (see FIG. 9) is equal to or less than the specified value or there is an operation suspension period for all resources, and if there is no designation from the user, the calculation is stored in the improvement designation.
- the server name is also included.
- the resource is minimum (aiming for a configuration that uses the least amount of resources), resource aggregation (aiming for a configuration in which the physical location of the allocated resources of each server is close), resource distribution (allocated resources of each server) are stored in a wide distribution of physical positions of Note that “new” is not an improvement, but refers to a case where a new server is constructed according to a user's request.
- the resource allocation improvement unit 206 determines an improvement method (step S1203).
- the resource arrangement improvement unit 206 refers to the improvement record table 217 (see FIG. 14), and if it is an operation according to the improvement ID 1402 created in step S1202 (step S1203, operation), the process proceeds to step S1204. If it is a trial (step S1203, trial), the process proceeds to step S1205.
- step S1204 the resource arrangement improvement unit 206 performs optimization improvement processing by calculation according to a known example (International Publication Number WO2008-132924).
- the improvement record table 217 (see FIG. 14) / improved resource information table 212 (see FIG. 8) is updated with the calculation result after improvement and the resource arrangement information.
- the management server 200 treats information such as application and device performance and settings as utility function parameters, and repeats function adjustment and solution derivation according to the trend of improving resource allocation. To obtain an optimum parameter value (including optimum arrangement information). For a resource group whose parameters can be easily grasped, an improved resource allocation is derived, and when the resource allocation can be derived, the resource allocation is performed based on the information, and the improved resource information table 212 ( (See FIG. 8). Further, since the improvement record table 217 (see FIG. 14) is information used when selecting an improvement method (calculation or trial) at the next improvement, the management server 200 improves the success or failure of the improvement by the calculation. It records in the recording table 217 (refer FIG. 14).
- step S1205 the resource allocation improvement unit 206 determines trial / new / improvement / maintenance / addition / reduction / fault identification, and proceeds to the next step (steps S1206 to S1209).
- the next step is branch processing for adjusting the extraction probability Ep of the resource information table 211 (see FIG. 7).
- step S1206 in the case of new / improvement / server designation, processing (1) for changing the extraction probability Ep is executed. Based on the usage rate graph of the resource information table 211 (see FIG. 7), the extraction probability Ep in the table is changed as follows during the trial of the current improvement ID.
- the success record 1407 of the improvement record table 217 has success information, which is similar to the contents of the server group column / group server relation column of this improvement ID (the same number of servers and the same server relation). If there is an attempt with a small required time and required number of improvement IDs, the resource used by the server group is specified in the resource information table 211 (see FIG. 7), and the resource is extracted. It is advisable to increase the probability Ep and to make the convergence quicker. Then, the process proceeds to step S1210.
- the numerical value in parentheses in the column of trial information 709 in the resource information table 211 indicates the number of resources to be investigated in that trial in order to prevent duplicate investigations in trials with the same improvement ID. Is the number of times to record.
- step S1207 in the case of maintenance, processing (2) for changing the extraction probability Ep is executed.
- the resource arrangement improvement unit 206 changes the value of the column of the trial information 709 in the resource information table 211 to “maintenance” for the resource in the user-specified maintenance server identified from the resource information table 211 (see FIG. 7). Further, the extraction probability Ep of the resource that has become “maintenance” is lowered, and the extraction probability Ep of the other resources is raised. Then, the process proceeds to step S1212.
- step S1208 in the case of expansion / reduction (increase / decrease), processing (3) for changing the extraction probability Ep is executed.
- the resource arrangement improvement unit 206 sets the columns of the trial information 709 in the resource information table 211 (see FIG. 7) to “addition (#)” or “reduction (#)” for the resources specified by the user at random or in advance. Change to The numbers (#) in parentheses are used to determine whether all resources have been investigated.
- the resource that has been increased increases the extraction probability Ep, and decreases the extraction probability Ep of other resources.
- the resource that has been reduced decreases the extraction probability Ep, and increases the extraction probability Ep of other resources. Then, the process proceeds to step S1212.
- step S1209 when a failure is specified, a process (4) for changing the extraction probability Ep is executed.
- the resource allocation improvement unit 206 displays the failure information received from the prior user input, the management software, or the failure information 708 in the resource information table 211 (see FIG. 7) in the previous failure identification attempt.
- the trial information of the resource at the same physical location as the resource set to “present” is set to “failure (#)”, the resource extraction probability Ep is increased, and the extraction probability Ep is decreased for other resources.
- a resource is selected at random, the trial information of the resource at the same physical location including the resource is set to “failure (#)”, and the resource extraction probability Ep is increased.
- the process proceeds to step S1212.
- step S1210 the resource allocation improvement unit 206 determines whether the trial is new or improvement or server designation. If it is new (step S1211, new), the improvement record table 217 (see FIG. 14) is changed to the default value, user The input value is created (step S1211), and the process proceeds to step S1213. In the case of improvement / server designation, the process proceeds to step S1212.
- step S ⁇ b> 1212 the resource arrangement improvement unit 206 creates an improvement record table 217 based on the server relationship and the specified server information. If there is a specified server or resource, the resource allocation improvement unit 206 selects servers having a strong relationship between the server or resource and the server from the server relationship table 214 (see FIG. 10), and the improvement record table 217 Recorded in the column of the server group 1407. When selecting again in the process with the same improvement ID, the selection is made so as not to be a group that has been selected from the history of the improvement record table 217 (see FIG. 14).
- the resource allocation improvement unit 206 calls the resource securing / trial improvement unit 207 (see FIG. 17) (step S1213), and updates the success / failure 1407 column of the improvement record table 217 (see FIG. 14) (step S1214). When the time limit / number of times is exceeded, the success / failure 1407 column is updated to “failure” together with the required time / number of times information.
- the success / failure 1407 column is set to “impossible to add” or “cannot be reduced” together with the required time / number information.
- the number (#) is used to determine whether all resources have been investigated.
- the trial information 709 in the FC1 row shown in FIG. 7 is selected as the second survey target when the improvement designation is “addition”, and the trial information 709 in the LAN2 row is selected as the first survey target in the “failure” judgment. Represents the history of
- the resource allocation improvement unit 206 makes an adoption determination (step S1215). If the success / failure 1407 column of the improvement record table 217 (see FIG. 14) is blank (step S1215, retry), the process returns to step S1205. When the success / failure 1407 column of the improvement record table 217 is “failure”, “addition possible”, “reduction possible”, “extension impossible”, “reduction impossible”, “failure (resource name)”, “no failure” (step) (S1215, trial end), the process proceeds to step S1216. When the success / failure 1407 column of the improvement record table 217 is “improvement” (step S1215, adopted), the process proceeds to step S1217.
- step S1216 the resource arrangement improvement unit 206 notifies the user.
- Information corresponding to “failure”, “addition possible”, “reduction possible”, “extension impossible”, “reduction impossible”, “failure (resource name)”, “no failure” is presented to the user, and the resource information table 211 (See FIG. 7). For example, in the case of “failure”, the column of the failure 708 in the resource information table 211 is set to “present”, and in the case of “no failure”, “no” is set.
- step S1217 the resource arrangement improvement unit 206 calls the improvement arrangement application unit 209 (see FIG. 19), executes the improvement arrangement, and returns to step S1201.
- the management server 200 when the number of times that the job determination method is not satisfied exceeds a predetermined number, or when the trial time that does not satisfy the job determination method exceeds a predetermined time, the management server 200 The administrator can be notified that improvement cannot be made.
- FIG. 17 is a flowchart showing processing of the resource securing / trial improvement unit.
- FIG. 17 shows details of processing of the resource securing / trial improving unit 207. Note that the processing of a predetermined step (for example, step S1301) is performed as long as the required time permits, and when the time specified or specified by the user is reached, the processing is rounded up and proceeds to the next processing.
- a predetermined step for example, step S1301
- the resource reservation / trial improvement unit 207 calls the comparison information acquisition unit 310 of the active server group, measures performance with a predetermined job, and can use the corresponding resource from the resource information table 211 (see FIG. 7).
- the range is investigated (step S1301). Specifically, the resource securing / trial improving unit 207 specifies the performance of the resources allocated to the servers in the column of the server group 1405 in the improvement record table 217 (see FIG. 14), and specifies the job necessary for the test 1402 And it specifies based on the relationship table 218 (refer FIG. 12) of improvement designation
- the resource securing / trial improving unit 207 secures unallocated resources as trial resources based on the resource information table 211 (see FIG. 7) (step S1302).
- securing the trial resource an attempt is made to secure the time (for example, 100h (100 hours)) stored in the improvement ID table excess determination in the improvement record table 217 (see FIG. 14).
- the time for example, 100h (100 hours)
- an unused portion of a resource that does not change in the usage rate graph of the resource information table 211 (see FIG. 7) is secured as a trial resource.
- the server relationship information of the server group 1405 is similar (the same number of servers, the same server relationship, etc.), and success information is stored in the success / failure 1407 column. If there is an improvement ID, the resource that has not changed in the usage rate graph of the resource information table 211 (see FIG. 7) as a past performance for the required time in the success / failure 1407 column is indicated as an unused portion. Secure as a trial resource.
- the resource securing / trial improving unit 207 calls the trial improving unit 208 (see FIG. 18) (step S1303), and performs processing to determine whether or not the trial can be improved with the resources secured in step S1302.
- the resource securing / trial improvement unit 207 refers to the improved resource information table 212 (see FIG. 8), and improves the resource regarding whether the improved resource allocation is performed for the server group in the improvement record table 217 (see FIG. 14). Information investigation is performed (step S1304).
- the resource securing / trial improvement unit 207 determines whether or not the improvement has been made (step S1305).
- the resource securing / trial improvement unit 207 allocates resources in the improved resource information table 212 (see FIG. 8) to all servers in the improvement record table 217 (see FIG. 14) in the improved resource information survey in step S1304. If it is found (step S1305, improvement), the process ends. If it is not assigned (step S1305, no improvement), the process proceeds to step S1306.
- the resource securing / trial improvement unit 207 identifies, as an aggregation survey, resources that can be aggregated from the resource usage rate and the job result management table (see FIG. 13) (step S1306). That is, the resource allocation / trial improvement unit 207 finds a server having a combination of resources having the same or equivalent results for the job from the job ID 1307 and the job result 1308 of the job result management table 216 (see FIG. 13), and resource information With reference to the usage rate graph 707 of the table 211 (see FIG. 7), it is confirmed whether the sum of the graphs has never exceeded 100% in the past. If there is no excess in the past, the servers using the compared resources can be aggregated on either resource.
- the resource securing / trial improvement unit 207 determines whether or not aggregation is possible (step S1307). If there is a server that can be aggregated (step S1307, aggregation is possible), the process proceeds to step S1308, and no server that can be aggregated is found ( In step S1307, aggregation is not possible, and the processing is terminated as aggregation is not possible.
- step S1308 in the aggregation survey in step S1306, the resource securing / trial improvement unit 207 aggregates the servers that are determined to be aggregated. That is, it is preferable to coexist on the same resource and increase unallocated resources.
- FIG. 18 is a flowchart showing the process of the trial improvement unit.
- the trial improvement unit 208 generates (creates) a server group, places the comparison information acquisition unit 310 in the created server group, and compares the performance of the current server group and the created trial server group in a trial job. Then, a process for determining whether or not the server group has been improved is performed.
- the trial improvement unit 208 calls the server group generation unit 220 (see FIG. 24) (step S1401), and places the comparison information acquisition unit 310 in the created server group (step S1402). That is, the comparison information acquisition unit 310 is push-installed on each trial server group created by the server group generation unit 220.
- the trial improvement unit 208 calls the comparison information acquisition unit between the active server group and the trial server group (step S1403). Specifically, in the improvement record table 217 (see FIG. 14), from the improvement specification 1402 column and the server group 1405 column corresponding to the current improvement ID, the improvement specification / job relationship table 218 (see FIG. 12) is displayed. Based on this, the comparison information acquisition unit 310 executes the job on each trial server group, and stores the results collected after the execution of the job in the job result management table 216 (see FIG. 13).
- the trial improvement unit 208 includes information on the column of the determination method 1104 in the job management table 215 (see FIG. 11) for the job results of the current server group and each trial server group in the job result management table 216 (see FIG. 13). Are compared based on the information in the column of the adoption configuration 1403 of the improvement record table 217 (see FIG. 14) (whether the job execution time is reduced and the resource usage is reduced, etc.) (step S1404) and adopted. Decide whether or not to hire. In the case of adoption, the resource allocation information of the trial server is stored in the improved resource information table 212 (see FIG. 8). Further, at the time of employment, the comparison value can be updated according to the determination method of the job management table 215 (see FIG. 11).
- step S1405 When all the trial server groups are adopted (step S1405, no unimproved server) as a result of the comparison in step S1404, the trial improvement unit 208 terminates the process, and when there is no adoption (step S1405, not yet). The process proceeds to step S1406.
- step S1406 the trial improvement unit 208 ends the process when the time limit or the number of times is equal to or greater than the content of the excess determination 1408 column of the improvement record table 217 (see FIG. 14) (step S1406 or higher). If not exceeded (step S1406, not reached), the process returns to step S1401.
- FIG. 19 is a flowchart showing the processing of the improvement placement application unit.
- the improved arrangement application unit 209 checks whether the resource can be shifted in the future based on the trend of the resource usage rate in the past (step S1501).
- the improvement arrangement application unit 209 uses the resource of the transfer destination resource of the improvement resource information table 212 (see FIG. 8) and the same type of resource allocated to the server of the server group 1405 of the improvement record table 217 (see FIG. 14). Based on the resource usage graph 707 of the information table 211 (see FIG. 7), the areas are divided and the difference is compared. If there is a difference in the result of the job result 1308 column of the job result management table 216 (see FIG.
- the ratio based on the performance difference is applied to the area value. For example, in the case of a job for measuring the communication speed, a value obtained by multiplying the area of the usage rate by the speed ratio is set as a comparison numerical value.
- the result is reflected in the area value and compared.
- the LAN1 usage rate graph area is “50 (% of 24h)” and the LAN2 usage rate area is “30 (% of 24h)”.
- the performance of LAN2 is 5/3 or more times that of LAN1, it is possible to use the virtual LAN on LAN2 or other servers that are using LAN2 alternately. In this case, it is determined that the destination of LAN1 can be changed.
- the improved arrangement application unit 209 determines whether or not the shift can be made from the difference in area in step S1501 (step S1502), and the difference in area is smaller in the area of the usage graph than the current resource in all the transfer destination resources. In the case (step S1502, transition), the process proceeds to step S1503, and when any one is larger (step S1502, transition target change), the process proceeds to step S1504.
- step S1504 the improvement placement application unit 209 determines the server relationship (step S1504), and if there is no value in the column of the server group relationship 1406 in the improvement record table 217 (see FIG. 14) (step S1504, no relationship). ), The process proceeds to step S1505. If there is a value (step S1505, there is a relationship), the process proceeds to step S1506.
- step S1506 the improvement placement application unit 209 performs server group change processing.
- the server group shown in the column of the server group 1405 (see FIG. 14) is logically extracted from the server relationship table 214 shown in FIG. 10 based on the information described in the column of the server group relationship 1406. Determined by sum. For this reason, the number of servers in the contents of the column of the server group 1405 is reduced by reducing the number of items from the column of the server group relationship 1406.
- the inclusion relation can be found from the server relation table 214 of FIG. 10
- the contents of the column of the server group relation 1406 are changed to the relation of the server having the smaller number of servers, thereby Try reducing the number of servers in the content to a size that can be migrated.
- step S1503 the improvement placement application unit 209 proceeds with migration in accordance with the improvement resource information table 212 (see FIG. 8). After the migration, the resource allocation state 706 allocated in the resource information table 211 (see FIG. 7) is updated.
- step S1505 change of extraction probability Ep or update impossible information is updated. If it is allocated in the configuration of the server remaining in the improved resource information table 212 (see FIG. 8) and cannot be transferred due to the difference in area in step S1501, the success / failure 1407 of the improved record table 217 (see FIG. 14) is determined. Update the column to “Migration disabled” along with the required time / count information. Further, the extraction probability Ep of the resource information table 211 (see FIG. 7) is lowered for the corresponding resource. Alternatively, the user may be notified as recommended expansion resources that should be expanded.
- FIG. 20 is a flowchart showing the processing of the comparison information acquisition unit.
- the comparison information acquisition unit 310 acquires the job sent from the management server 100 (step S1601)
- the comparison information acquisition unit 310 executes the acquired job (step S1602) and is necessary for the determination method 1104 of the job management table 215 (see FIG. 11).
- the job result is transmitted to the trial improvement unit 208 of the management server 100 (step S1603), and the process ends.
- FIG. 21 is a flowchart showing the processing of the server related information acquisition unit.
- the server relation information acquisition unit 311 determines the policy set when the push installation is performed by the information collection unit 210 of the management server 200 (step S1701), and in the case of regular (step S1701, regular), the process proceeds to step S1702. In the case of monitoring (step S1701, monitoring), the process proceeds to step S1704.
- step S1702 the server relation information acquisition unit 311 acquires information on the contents of the items in the server relation table 214 (see FIG. 10) from the server device information of the OS and management software. Then, the server related information acquisition unit 311 transmits the acquired information to the information collection unit 210 of the management server 200 (step S1703), and the process returns to step S1701. Next, when step S1701 is executed, it is executed after a preset time has elapsed.
- step S1704 the server relation information acquisition unit 311 monitors whether there is a change in the items included in the information in the server relation table 214 (see FIG. 10), and determines whether there is a change (step S1705). If there is a change (step S1705, yes), the process proceeds to step S1702, and if there is no change (step S1705, no), the process returns to step S1701.
- FIG. 22 is a flowchart showing processing of the resource information acquisition unit.
- the resource information acquisition unit 312 determines the policy set when the push installation is performed by the information collection unit 210 of the management server 200 (step S1801). If the policy is regular (step S1801, regular), the process proceeds to step S1802. In the case of (step S1801, monitoring), the process proceeds to step S1804.
- step S1802 the resource information acquisition unit 312 acquires information on the contents of items in the resource information table 211 (see FIG. 7) from the device information of the server that the OS and management software have. Then, the resource information acquisition unit 312 transmits the acquired information to the information collection unit 210 of the management server 200 (step S1803), and returns to step S1801. Next, when executing step S1801, it is executed after elapse of a preset time.
- step S1804 the resource information acquisition unit 312 monitors resource information (for example, addition or reduction of resources) and determines whether there is a change (step S1805). If there is a change (step S1805, yes), the process proceeds to step S1802. If there is no change (step S1805, no), the process returns to step S1801.
- resource information for example, addition or reduction of resources
- FIG. 23 is a flowchart showing the processing of the virtual allocation information acquisition unit.
- the virtual allocation information acquisition unit 313 determines the policy set when the push installation is performed by the information collection unit 210 of the management server 200 (step S1901). If it is regular (step S1901, regular), the process proceeds to step S1902. In the case of monitoring (step S1901, monitoring), the process proceeds to step S1904.
- step S1902 the virtual allocation information acquisition unit 313 acquires information on the contents of items in the resource information table 211 (see FIG. 7) and the server relation table 214 (see FIG. 10) from the resource allocation information of the virtualization mechanism unit. To do. Then, the virtual allocation information acquisition unit 313 transmits the acquired information to the information collection unit 210 of the management server 200 (step S1903) and returns to step S1901. Next, when executing step S1901, it is executed after elapse of a preset time.
- step S1904 the virtual allocation information acquisition unit 313 monitors virtual allocation information to each virtual machine of the virtualization mechanism, and determines whether there is a change (step S1905). If there is a change (step S1905, yes), the process proceeds to step S1902, and if there is no change (step S1905, none), the process returns to step S1901.
- FIG. 24 is a flowchart showing the processing of the server group generation unit.
- the server group generation unit 220 for each trial resource secured in step S1302 (see FIG. 17), based on the extraction probability Ep (710) of the resource information table 211 (see FIG. 7), Random number generation unit 221 (see FIG. 2) generates random numbers for all resources in resource information table 211 or for each resource in a certain range, and as resources allocated to servers in server group generation unit 220 this time Whether to use or not to use is determined and extracted (step S2001).
- step S2001 is one of the features of this embodiment.
- the server group generation unit 220 allocates resource allocation information (resource information table 211 (see FIG. 7)) of the active server stored in the server group of the improvement record table 217 (see FIG. 14) corresponding to the current improvement ID. From the column of the state 706, the number of resources to be allocated to each server of the server group to be created is determined (step S2002). Note that the number of resources to be allocated can be increased or decreased by using the current number of allocated resources as a default value and adding fluctuations stochastically.
- the server group generation unit 220 generates server groups from the resources extracted in step S2001 by the number of servers in the improvement record table 217 (see FIG. 14) (step S2003). Specifically, the server group generation unit 220 refers to the column of the improvement designation 1402 corresponding to the current improvement ID of the improvement record table 217 (see FIG. 14) and the column of the server group relation 1406, and specifies the resource of the improvement designation. As a shared resource, the resource extracted in step S2001 is assigned by satisfying the resource specification required for improvement. In addition, the number of resources allocated to each server can be referred to the number determined in step S2002. If the number of resources extracted in step S2001 does not reach the requested number of resources, the process may be terminated and the process of the trial improvement unit 208 may be performed again.
- the management server 200 has a record of application information when a trial server formed by combining resources that can be secured from a network to which a plurality of servers constituting a computer system are connected is used as a server of the computer system.
- Job management for managing, for each job, a storage unit for resource information (for example, the resource information table 211) in which the extraction probability Ep is registered, and a job determination method that is a predetermined condition as to whether or not to adopt a trial server If the extraction probability is equal to or greater than the random number probability Rp generated by the random number generation unit 221 for each resource of the resource information and the resource management information storage unit storing information (for example, the job management table 215) Forming a trial server by combining the selected trial resources.
- the formation processing unit and the application processing unit express the processor 202 as a function-specific processing unit.
- the management server 200 uses the resource as a trial resource.
- a trial server for example, the trial physical server 100b and the trial virtual server 130b
- the trial server can be applied as a server on the network. Thereby, resources on the network can be used effectively.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
Abstract
Description
図1は、本発明に係るネットワークシステムを示す全体構成図である。図1に示すように、ネットワークシステムは、管理サーバ200と、1以上の物理サーバ100と、I/Oスイッチ拡張装置160、バス接続制御装置150とがネットワークスイッチ140を介して接続されている。物理サーバ100には、I/Oスイッチ拡張装置160、I/Oデバイス170を介して、ディスクボリューム190を有するストレージ装置180が接続されている。なお、物理サーバ100には、仮想サーバ130を提供する仮想化機構部131を有する物理サーバも含まれる。
図7は、リソース情報テーブルの一例および使用率グラフを示す説明図である。図7(a)はリソース情報テーブル211の一例を示し、図7(b)は、リソース情報テーブル211の使用率グラフ707の例を示す。図7(a)に示すリソース情報テーブル211は、管理サーバ200が保持しており、情報収集部210が収集した情報、ユーザI/F203からの入力情報を元に更新される。リソース情報テーブル211は、リソース名701、種類702、仮想関係703、論理・固有値設定704、物理位置705、割当状態706、使用率グラフ707、故障有無708、試行情報709、および抽出確率Ep(710)を含んで構成される。
図15は、情報収集部の処理を示すフローチャートである。適宜図2、図3、図5を参照して説明する。情報収集部210は、各サーバに配置したサーバの関係情報取得部311、リソース情報取得部312、および仮想割り当て情報取得部313からリソースの情報を収集する。管理サーバ200が管理していないサーバの場合には、サーバの関係情報取得部311、リソース情報取得部312、および仮想割り当て情報取得部313を配置する処理をする。なお、所定のステップの処理(例えば、ステップS1102、ステップS1106)は、所要時間の許す限り行うものとし、規定、またはユーザ指定の時間に達した場合は、処理を切り上げ次の処理に進む。
改善リソース情報テーブル212(図8参照)の全てのサーバに、リソースの割り当てが済んでいる場合、改善記録テーブル217(図14参照)の成否1407の列に所要時間・回数情報とともに「改善」と格納する。
リソース情報テーブル211(図7参照)の試行情報709の列の値が「保守」になっているリソース全てで、割当状態706の列に割り当てられているサーバが無くなっている場合、改善記録テーブル217(図14参照)の成否1407の列に所要時間・回数情報とともに「保守対応可」と格納する。すなわち、リソースがどのサーバにも割り当てられていない状態を意味する。
リソース情報テーブル211(図7参照)の試行情報709の列が「増設」になっているリソースの使用率が100%またはユーザ指定の値以上である場合、改善記録テーブル217(図14参照)の成否1407の列を所要時間・回数情報とともに「増設可」にする。同様に、リソース情報テーブル211(図7参照)の試行情報709の列が「削減」になっているリソースの使用率が0%またはユーザ指定の値以下である場合、改善記録テーブル217(図14参照)の成否1407の列を所要時間・回数情報とともに「削減可」にする。
リソース情報テーブル211(図7参照)の試行情報709の列の値が「故障」となっているリソースの割当状態706列に情報が格納されていない場合、ユーザに故障の可能性があるリソースとして、改善記録テーブル217(図14参照)の成否1407の列に所要時間・回数情報とともに「故障(リソース名)」を格納する。他方、リソースの割当状態706の列に情報が格納されており、試行情報709内の#が最後になっている場合、改善記録テーブル217(図14参照)の成否1407の列に所要時間・回数情報とともに「故障無し」と格納する。
130 仮想サーバ
131 仮想化機構部
140 ネットワークスイッチ
150 バス接続制御装置
160 I/Oスイッチ拡張装置
170 I/Oデバイス
180 ストレージ装置
190 ディスクボリューム
191 仮想ディスク
200 管理サーバ
201 メモリ(記憶部)
202 プロセッサ(処理部)
203 ユーザI/F
204 ネットワークI/F
205 ディスクI/F
206 リソース配置改善部
207 リソース確保・試行改善部
208 試行改善部
209 改善配置適用部
210 情報収集部
211 リソース情報テーブル
212 改善リソース情報テーブル
213 履歴テーブル
214 サーバの関係テーブル
215 ジョブ管理テーブル
216 ジョブ結果管理テーブル
217 改善記録テーブル
218 改善指定とジョブ関係テーブル
220 サーバ群生成部
310 比較情報取得部
311 サーバの関係情報取得部
312 リソース情報取得部
313 仮想割り当て情報取得部
Ep 抽出確率
Rp 乱数確率
Claims (20)
- ネットワークに複数のサーバが接続された計算機システムにおいて、管理サーバが、該ネットワークで確保できたリソースを組み合わせて試行用サーバを形成し、該試行用サーバが所定条件を満たす場合にサーバとして適用するリソース管理方法であって、
前記管理サーバは、前記ネットワークで確保できたリソースに対し、前記試行用サーバが適用された際の適用情報の実績である抽出確率が登録されたリソース情報と、前記試行用サーバを採用するか否かの前記所定条件であるジョブの判断方法をジョブごとに管理するジョブ管理情報とを記憶部に記憶しており、
前記管理サーバは、前記リソース情報の各リソースに対し、前記抽出確率が乱数生成部で発生させた乱数確率以上であれば、試行用のリソースとして選定し、
前記選定された試行用のリソースを組み合わせて前記試行用サーバを形成し、
前記形成された試行用サーバを用いて前記ジョブを実行し、
前記ジョブの実行結果が前記ジョブの判断方法を満たせば、前記試行用サーバを該ネットワークのサーバとして適用する
ことを特徴とするリソース管理方法。 - 前記管理サーバは、前記実行結果が前記ジョブの判断方法を満たさなければ、前記リソースの選定、前記試行用サーバの形成、前記ジョブの実行を繰り返し、前記繰り返したときのジョブの実行結果を改善記録情報として前記記憶部に記録する
ことを特徴とする請求の範囲第1項に記載のリソース管理方法。 - 前記管理サーバは、前記試行用サーバを適用するか否かを判定する際に、現用のサーバで前記ジョブを実行し、前記現用のサーバのジョブの実行結果と前記試行用サーバのジョブの実行結果とを比較し、前記試行用サーバの性能が前記現用のサーバの性能と比較して改善している場合、前記試行用サーバを該ネットワークのサーバとして適用する
ことを特徴とする請求の範囲第1項に記載のリソース管理方法。 - 前記管理サーバは、現用のサーバが指定され、かつ、保守要求がされた場合、前記現用のサーバと関係のあるサーバを特定し、前記特定されたサーバの数に対応する複数の試行用サーバを、前記選定されたリソースを組み合わせて形成し、前記複数の試行用サーバを用いて前記ジョブを実行し、前記ジョブの実行結果が前記ジョブの判断方法を満たせば、前記複数の試行用サーバを該ネットワークのサーバとして適用する
ことを特徴とする請求の範囲第1項に記載のリソース管理方法。 - 前記管理サーバは、ユーザが演算による方法を選択した場合、請求の範囲第1項に記載のリソース管理方法に代わり、該演算による方法でリソース配置の改善をする
ことを特徴とする請求の範囲第1項に記載のリソース管理方法。 - 前記管理サーバは、前記ジョブの実行を繰り返すごとに、前記抽出確率を変更する
ことを特徴とする請求の範囲第2項に記載のリソース管理方法。 - 前記管理サーバは、リソースとして選定される確率をあげる際、前記抽出確率をあげる
ことを特徴とする請求の範囲第2項に記載のリソース管理方法。 - 前記リソース情報には、リソースの使用率が含まれ、
前記リソースの使用率がユーザ指定の値以上である場合、リソースの増設を推奨する旨を前記改善記録情報に記録する
ことを特徴とする請求の範囲第2項に記載のリソース管理方法。 - 前記リソース情報には、リソースの使用率が含まれ、
前記リソースの使用率がユーザ指定の値以下である場合、リソースの削除を推奨する旨を前記改善記録情報に記録する
ことを特徴とする請求の範囲第2項に記載のリソース管理方法。 - 前記管理サーバは、前記ジョブの判断方法を満たさない回数が所定回数を超えた場合、または、前記ジョブの判断方法を満たさない試行時間が所定時間を超えた場合、リソース配置の改善ができない旨を管理者に通知する
ことを特徴とする請求の範囲第2項に記載のリソース管理方法。 - 計算機システムをなす複数のサーバが接続されるネットワークから確保できるリソースを組み合わせて形成する試行用サーバを前記計算機システムのサーバとして適用した際の適用情報の実績である抽出確率が登録されるリソース情報の記憶部と、
前記試行用サーバを採用するか否かの前記所定条件であるジョブの判断方法をジョブごとに管理するジョブ管理情報が記憶されるジョブ管理情報の記憶部と、
前記リソース情報の各リソースに対し、前記抽出確率が乱数生成部で発生させた乱数確率以上であれば、試行用のリソースとして選定し、前記選定された試行用のリソースを組み合わせて前記試行用サーバを形成する形成処理部と、
前記形成された試行用サーバを用いて前記ジョブを実行し、前記ジョブの実行結果が前記ジョブの判断方法を満たせば、前記試行用サーバを該ネットワークのサーバとして適用する適用処理部と、を有する
ことを特徴とする管理サーバ。 - 前記管理サーバは、前記実行結果が前記ジョブの判断方法を満たさなければ、前記リソースの選定、前記試行用サーバの形成、前記ジョブの実行を繰り返し、前記ジョブの実行結果を改善記録情報として前記記憶部に記録する
ことを特徴とする請求の範囲第11項に記載の管理サーバ。 - 前記管理サーバは、前記試行用サーバを適用するか否かを判定する際に、現用のサーバで前記ジョブを実行し、前記試行用サーバの実行結果が前記現用のサーバのジョブの実行結果より改善している場合、前記試行用サーバを該ネットワークのサーバとして適用する
ことを特徴とする請求の範囲第11項に記載の管理サーバ。 - 前記管理サーバは、現用のサーバが指定され、かつ、保守要求がされた場合、前記現用のサーバと関係のあるサーバを特定し、前記特定されたサーバの数に対応する複数の試行用サーバを、前記選定されたリソースを組み合わせて形成し、前記複数の試行用サーバを用いて前記ジョブを実行し、前記ジョブの実行結果が前記ジョブの判断方法を満たせば、前記複数の試行用サーバを該ネットワークのサーバとして適用する
ことを特徴とする請求の範囲第11項に記載の管理サーバ。 - 前記管理サーバは、ユーザが演算による方法を選択した場合、請求の範囲第11項に記載のリソース管理方法に代わり、該演算による方法でリソース配置の改善をする
ことを特徴とする請求の範囲第11項に記載の管理サーバ。 - 前記管理サーバは、前記ジョブの実行を繰り返すごとに、前記抽出確率を変更する
ことを特徴とする請求の範囲第12項に記載の管理サーバ。 - 前記管理サーバは、リソースとして選定される確率をあげる際、前記抽出確率をあげる
ことを特徴とする請求の範囲第12項に記載の管理サーバ。 - 前記リソース情報には、リソースの使用率が含まれ、
前記リソースの使用率がユーザ指定の値以上である場合、リソースの増設を推奨する旨を前記改善記録情報に記録する
ことを特徴とする請求の範囲第12項に記載の管理サーバ。 - 前記リソース情報には、リソースの使用率が含まれ、
前記リソースの使用率がユーザ指定の値以下である場合、リソースの削除を推奨する旨を前記改善記録情報に記録する
ことを特徴とする請求の範囲第12項に記載の管理サーバ。 - 前記管理サーバは、前記ジョブの判断方法を満たさない回数が所定回数を超えた場合、または、前記ジョブの判断方法を満たさない試行時間が所定時間を超えた場合、リソース配置の改善ができない旨を管理者に通知する
ことを特徴とする請求の範囲第12項に記載の管理サーバ。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2011/058390 WO2012137272A1 (ja) | 2011-04-01 | 2011-04-01 | リソース管理方法および管理サーバ |
US14/007,848 US9385964B2 (en) | 2011-04-01 | 2011-04-01 | Resource management method and management server |
JP2013508636A JP5525654B2 (ja) | 2011-04-01 | 2011-04-01 | リソース管理方法および管理サーバ |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2011/058390 WO2012137272A1 (ja) | 2011-04-01 | 2011-04-01 | リソース管理方法および管理サーバ |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2012137272A1 true WO2012137272A1 (ja) | 2012-10-11 |
Family
ID=46968713
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2011/058390 WO2012137272A1 (ja) | 2011-04-01 | 2011-04-01 | リソース管理方法および管理サーバ |
Country Status (3)
Country | Link |
---|---|
US (1) | US9385964B2 (ja) |
JP (1) | JP5525654B2 (ja) |
WO (1) | WO2012137272A1 (ja) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2016085524A (ja) * | 2014-10-23 | 2016-05-19 | 日本電気株式会社 | コンピュータシステム、情報処理装置、リソース割り当て方法及び情報処理装置のプログラム |
US9940157B2 (en) | 2015-06-10 | 2018-04-10 | Fujitsu Limited | Computer readable medium, method, and management device for determining whether a virtual machine can be constructed within a time period determined based on historical data |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9215142B1 (en) * | 2011-04-20 | 2015-12-15 | Dell Software Inc. | Community analysis of computing performance |
US9323579B2 (en) * | 2012-08-25 | 2016-04-26 | Vmware, Inc. | Resource allocation diagnosis on distributed computer systems |
US9298512B2 (en) * | 2012-08-25 | 2016-03-29 | Vmware, Inc. | Client placement in a computer network system using dynamic weight assignments on resource utilization metrics |
US10223235B2 (en) * | 2016-05-26 | 2019-03-05 | International Business Machines Corporation | Comprehensive testing of computer hardware configurations |
US10216599B2 (en) | 2016-05-26 | 2019-02-26 | International Business Machines Corporation | Comprehensive testing of computer hardware configurations |
US11487646B2 (en) * | 2019-03-01 | 2022-11-01 | Red Hat, Inc. | Dynamic test case timers |
US11741276B1 (en) * | 2022-12-16 | 2023-08-29 | Dk Crown Holdings Inc. | Systems and methods for modeling live events |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007526557A (ja) * | 2004-01-30 | 2007-09-13 | インターナショナル・ビジネス・マシーンズ・コーポレーション | コンピューティング・ユーティリティのためのコンピューティング環境のコンポーネント化された自動プロビジョニングおよび管理 |
JP2008527513A (ja) * | 2005-01-06 | 2008-07-24 | インターナショナル・ビジネス・マシーンズ・コーポレーション | グリッド環境にサブミットされたグリッド・ジョブによる使用の前のリソース機能の検査 |
JP2010205209A (ja) * | 2009-03-06 | 2010-09-16 | Hitachi Ltd | 管理計算機、計算機システム、物理リソース割り当て方法 |
JP2011018198A (ja) * | 2009-07-09 | 2011-01-27 | Hitachi Ltd | 管理装置及び管理方法 |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7487258B2 (en) | 2004-01-30 | 2009-02-03 | International Business Machines Corporation | Arbitration in a computing utility system |
US9104494B2 (en) | 2007-04-13 | 2015-08-11 | Nec Corporation | Virtual computer system and its optimization method |
-
2011
- 2011-04-01 WO PCT/JP2011/058390 patent/WO2012137272A1/ja active Application Filing
- 2011-04-01 JP JP2013508636A patent/JP5525654B2/ja active Active
- 2011-04-01 US US14/007,848 patent/US9385964B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007526557A (ja) * | 2004-01-30 | 2007-09-13 | インターナショナル・ビジネス・マシーンズ・コーポレーション | コンピューティング・ユーティリティのためのコンピューティング環境のコンポーネント化された自動プロビジョニングおよび管理 |
JP2008527513A (ja) * | 2005-01-06 | 2008-07-24 | インターナショナル・ビジネス・マシーンズ・コーポレーション | グリッド環境にサブミットされたグリッド・ジョブによる使用の前のリソース機能の検査 |
JP2010205209A (ja) * | 2009-03-06 | 2010-09-16 | Hitachi Ltd | 管理計算機、計算機システム、物理リソース割り当て方法 |
JP2011018198A (ja) * | 2009-07-09 | 2011-01-27 | Hitachi Ltd | 管理装置及び管理方法 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2016085524A (ja) * | 2014-10-23 | 2016-05-19 | 日本電気株式会社 | コンピュータシステム、情報処理装置、リソース割り当て方法及び情報処理装置のプログラム |
US9940157B2 (en) | 2015-06-10 | 2018-04-10 | Fujitsu Limited | Computer readable medium, method, and management device for determining whether a virtual machine can be constructed within a time period determined based on historical data |
Also Published As
Publication number | Publication date |
---|---|
JP5525654B2 (ja) | 2014-06-18 |
US9385964B2 (en) | 2016-07-05 |
JPWO2012137272A1 (ja) | 2014-07-28 |
US20140019624A1 (en) | 2014-01-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5525654B2 (ja) | リソース管理方法および管理サーバ | |
US11700303B1 (en) | Distributed data analysis for streaming data sources | |
EP2972746B1 (en) | Storage unit selection for virtualized storage units | |
US9716746B2 (en) | System and method using software defined continuity (SDC) and application defined continuity (ADC) for achieving business continuity and application continuity on massively scalable entities like entire datacenters, entire clouds etc. in a computing system environment | |
US7441024B2 (en) | Method and apparatus for applying policies | |
CN106302565B (zh) | 业务服务器的调度方法及系统 | |
CN101370030B (zh) | 基于内容复制的资源负载平衡方法 | |
CN100407153C (zh) | 需要时节点和服务器实例分配和解除分配 | |
JP4597488B2 (ja) | プログラム配置方法及びその実施システム並びにその処理プログラム | |
JP6434131B2 (ja) | 分散処理システム、タスク処理方法、記憶媒体 | |
US7437460B2 (en) | Service placement for enforcing performance and availability levels in a multi-node system | |
WO2012056596A1 (ja) | 計算機システム及び処理制御方法 | |
JP2004227359A (ja) | ポリシーに基づいたストレージシステムの運用管理方法 | |
US20090138594A1 (en) | Coordinating the monitoring, management, and prediction of unintended changes within a grid environment | |
JP2016511490A5 (ja) | ||
KR101959601B1 (ko) | 관리 시스템 및 관리 시스템을 제어하기 위한 방법 | |
JP2016511490A (ja) | 仮想データセンタリソース利用ポリシーの自動調整 | |
CN104717094A (zh) | 管理服务器和管理服务器的控制方法 | |
US20060200469A1 (en) | Global session identifiers in a multi-node system | |
KR101211207B1 (ko) | 캐시 클라우드 구조를 이용한 캐시 시스템 및 캐싱 서비스 제공 방법 | |
WO2014080492A1 (ja) | 計算機システム、クラスタ管理方法、及び管理計算機 | |
JP2008146627A (ja) | 複数のデータセンタにおけるストレージリソース管理のための方法および装置 | |
JP4441362B2 (ja) | ポート割当装置及びポート割当方法 | |
US20060092851A1 (en) | Method and apparatus for communicating predicted future network requirements of a data center to a number of adaptive network interfaces | |
JP7253007B2 (ja) | ストレージシステム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11862871 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2013508636 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14007848 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 11862871 Country of ref document: EP Kind code of ref document: A1 |