WO2018116460A1

WO2018116460A1 - Continuous integration system and resource control method

Info

Publication number: WO2018116460A1
Application number: PCT/JP2016/088459
Authority: WO
Inventors: 貴也井出; 哲守屋; 順史木下; 敬太嶋田
Original assignee: 株式会社日立製作所
Priority date: 2016-12-22
Filing date: 2016-12-22
Publication date: 2018-06-28
Also published as: JPWO2018116460A1; JP6692454B2

Abstract

This continuous integration system comprises a CI server for managing software tests, and a computation cluster for conducting a software test, wherein the CI server comprises: a CI service unit which receives software to be tested, causes the computation cluster to conduct a test using resources calculated by a resource estimation unit, and acquires the test results from the computation cluster; and the resource estimation unit, which estimates resources to be allocated to a test. The resource estimation unit comprises: a test ID calculation unit which assigns a unique test ID to each test to be conducted; a result acquisition unit which acquires information about consumed resources for each test ID; history information including and associating the test IDs of previously conducted tests with the acquired resource information; and an estimation unit which estimates, for each test ID, the resources required to conduct the test associated with the test ID, on the basis of the history information.

Description

Continuous integration system and resource control method

The present invention relates to a computer system and method for reducing computer resources consumed in a test for solving a problem of a program.

In software development, continuous integration (CI) that continuously repeats coding and software testing (hereinafter referred to as “test”) is performed. When CI is performed, a source code and a test code (hereinafter collectively referred to as a code group) generated by a user in a software configuration management (SCM) server are managed.

The software configuration management server (hereinafter referred to as SCM server) executes the test code on the continuous integration server (hereinafter referred to as CI server) in response to a request from the user or a code group update. The source code describes the processing of the software to be tested, and the test code includes a series of steps required for testing, such as processing to evaluate the source code (test processing) and starting of build and test processing. Processing is described.

In recent years, CI services that provide the functions of the CI server as a service have begun to appear. When the user updates the code group of the SCM server, the CI service automatically executes the user's test code on the CI server owned by the CI service provider and sends the execution result to the user by e-mail or chat. It is a service to notify.

In the above conventional technology, when a user's test code is executed in the CI service, each test is generated by a virtual machine or container (such as LXC (Linux Containers)) on the test execution base. Instance) and execute a test on the instance.

In the current CI service, a sufficient amount of computing resources (processors, memory, disk size, network bandwidth, etc., hereinafter referred to as resources) are uniformly allocated to instances for executing tests. This can suppress user test failures due to insufficient resources in the execution environment of the CI server, but there is a problem that resources are wasted because resources are uniformly allocated to tests with low load. It was.

By using the public cloud environment as the CI service execution platform, the infrastructure resources can be increased according to the load, so there is no risk of resource depletion. Makes it difficult to make a profit. In order to reduce the cost of resources, the CI service is required to efficiently use execution base resources for executing tests.

Patent Documents

1 and 2 are known as techniques for efficiently using CI execution base resources.

Patent Document 1 describes a test that reduces the amount of regression test by changing the test execution range so that only the test related to the update part of the source code is executed while ensuring the completeness of the test. A system is disclosed.

Patent Document 2 discloses a test to be executed from a set of tests based on quality attributes (SQA: Software Quality Attributes) given to software by a test and information on success or failure of past test results, while available resources are limited. A method for extracting a subset of is disclosed.

Japanese Patent Laid-Open No. 2015-069400 US Pat. No. 9,405,661

When the service provider provides the above-described CI as a service, it is difficult to use the resource that is the test execution base for efficiency in the prior art document. The reason is shown below.

Patent Documents

1 and 2 focus on the fact that most of the test is the same process as the previous test, and extract and extract some processes from the test based on the past test information and test results. By executing only processing, the amount of resources required for testing is reduced.

However, from the viewpoint of software quality assurance, if all tests are not executed, it cannot be guaranteed that there are no problems in the test process. For this reason, it is not desirable to extract and execute a test to be executed partially as in the conventional example. In particular, in the CI service, it is difficult in terms of service quality to extract and execute a test to be executed from a test generated by a user for the purpose of reducing resources by the service provider.

The present invention has been made in consideration of the above points, and by estimating the resources necessary for each test based on the historical information of past tests, the resource allocated to each test instance is changed, and the test is executed. The purpose is to reduce the amount of resources to be allocated.

The present invention includes a CI server having a processor and a memory, and managing one or more computers having a processor and a memory. The software test is performed in response to a request from the CI server. A continuous integration system including a calculation cluster to be executed, wherein the CI server receives software for executing the test, determines a resource to be allocated to the test, and notifies the calculation cluster of the resource. A CI service unit that executes the test and obtains an execution result from the calculation cluster; and a resource estimation unit that estimates a resource to be allocated to the test in response to a request from the CI service unit, and The resource estimation unit assigns a test ID that is unique for each test. ID information calculation unit, a result acquisition unit for acquiring information of resources consumed during execution of the test for each test ID, and the test ID executed in the past and the acquired resource information in association with each other And when the CI service unit requests resource estimation, the resource necessary for execution of the test is estimated for each test ID based on the history information, and the estimation result is used as resource allocation information. An estimation unit for notifying the CI service unit, wherein the CI service unit transmits a resource estimation request to be allocated to the test to the resource estimation unit, and the estimation result of the resource received from the resource estimation unit is transmitted to the test unit. Decide as a resource to be assigned to.

According to the present invention, by estimating the resources necessary for each test, it is possible to change the amount of computer resources allocated to the test, thereby reducing the resource consumption during the execution of the test.

It is a block diagram which shows Example 1 of this invention and shows the test which is a unit of resource estimation. 1 is a block diagram illustrating an example of a CI system including a resource estimation device according to a first embodiment of this invention. FIG. It is Example 1 of this invention and is a block diagram which shows an example of the functional element of a resource estimation apparatus. 1 is a block diagram illustrating an example of a hardware configuration of a resource estimation device according to a first embodiment of this invention. FIG. It is a figure which shows Example 1 of this invention and shows an example of the test ID management table contained in log | history information. It is a figure which shows Example 1 of this invention and shows an example of the log | history management table contained in log | history information. FIG. 7 is a sequence diagram illustrating an example of processing performed in the CI system according to the first embodiment of this invention. It is a figure which shows Example 1 of this invention and shows an example of the display screen which a resource estimation apparatus outputs. It is a figure which shows Example 2 of this invention and shows an example of the log | history management table stored in log | history information. FIG. 10 is a sequence diagram illustrating an example of processing performed in the CI system according to the second embodiment of this invention. It is Example 3 of this invention, and is a block diagram which shows an example of a structure of CI server.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating a test 150, which is a unit for estimating resource usage in the first embodiment. The test 150 in the first embodiment represents a series of test codes 151-1 to 151-3 that have been updated for the same purpose in order to test continuously developed software (source code). However, the notation “test” or “software test” without “test 150” indicates not a unit of information but a test as a software evaluation method.

Test codes 151-1 to 151-3 are a series of character strings representing a series of processes necessary for a process such as a process for evaluating software and a test such as build or test activation, and are described in one or more files. In the following description, in order to indicate the entire test code, the symbol “151” in which “−” and the subsequent symbols are omitted is used. The same applies to the reference numerals of other components.

Even if the test code 151-1 is sequentially changed to the test codes 151-2 to 151-3 according to software development, they are regarded as the same test 150. A test ID (identifier) 74 described later in FIG. 5 represents an identifier that uniquely defines the test 150.

In the first embodiment, a setting file (Jenkinsfile or the like) for automatically executing the test 150 in the calculation cluster 30 and a file defining an instance configuration (docker-compose.yml or the like) are also handled as the test code 151.

<Overall system configuration>
FIG. 2 is a block diagram illustrating an example of functional elements of a CI (Continuous Integration) system 1 including the resource estimation device 2. The role of each functional element of the CI system 1 and the relationship with the resource estimation device 2 are shown using this figure.

The CI system 1 is a computer system in which an SCM server 10, a CI server 20, a resource estimation device 2, and a calculation cluster 30 are connected via a communication network. The SCM server 10, the CI server 20, the resource estimation device 2, and the calculation cluster 30 are configured by hardware such as one or more computers or software such as one or more virtual machines and containers. To communicate with each other.

The SCM server 10 manages the code group 40 generated by the user terminal 50 in units of the repository 11 and the branch 12. When the user terminal 50 updates the code group 40 of the SCM server 10, the SCM server 10 transmits the updated code group 40 and code meta information 41 to the CI server 20.

The code group 40 includes a test code 151 and source code generated by the user terminal 50. The code meta information 41 is information accompanying the code group 40, and includes the test code 151 included in the code group 40, the repository 11 for specifying the source code, the ID of the branch 12, and the like.

The CI server 20 causes the calculation cluster 30 to execute a software test in response to a request from the user terminal 50 or an update of the repository 11 of the SCM server 10. The CI server 20 includes an SCM cooperation function 21, a resource estimation cooperation function 22, a calculation cluster cooperation function 23, and job information 24.

The SCM linkage function 21 has a function of receiving the code group 40 and the code meta information 41 from the SCM server 10 and generating job meta information 42 corresponding to the code meta information 41 with reference to the job information 24.

The resource estimation cooperation function 22 notifies the resource estimation device 2 of the code meta information 41 and the job meta information 42, and requests the estimation of the resource amount to be allocated to the test 150. Note that the test 150 is specified by the resource estimation device 2 based on the job identifier included in the job meta information 42 and the identifier of the test code 151 and the source code included in the code meta information 41.

When the resource estimation cooperation function 22 receives the resource allocation information 43 for the resource amount estimation request from the resource estimation device 2, the resource estimation cooperation function 22 determines the resource to be allocated to the instance 34 for executing the test 150, and logs information after the execution of the test 150 44 to the resource estimation device 2. The resource refers to a computer resource such as a processor allocation amount, a memory capacity, a disk size, and a network bandwidth of the host 33 in the calculation cluster 30.

The calculation cluster cooperation function 23 requests the calculation cluster 30 to generate the instance 34 (resource request), execute the test 150 on the instance 34 (execution request for the test 150), and delete the instance 34. The job information 24 manages job meta information 42 corresponding to the code meta information 41 received from the SCM server 10.

The job meta information 42 has a job ID corresponding to the code group 40. Depending on the function of the CI server 20, the job meta information 42 may include the ID of the repository 11 or the branch 12 of the code group 40 in the SCM server 10. In this case, the job meta information 42 can be used as an alternative to the code meta information 41. The job ID is, for example, a job name in Jenkins (registered trademark), a pipeline name or a group name in Concourse CI (registered trademark), and can be generated by the CI server 20.

The log information 44 includes the success or failure of the test 150 (execution result), the start time of the test 150 (execution time 85), the ID of the instance 34 generated in the host 33 of the calculation cluster 30, and the value of the resource amount allocated to the instance 34. Etc. As the ID of the instance 34, an ID generated by the calculation cluster management device 31 or the like can be used. The resource estimation device 2 can specify the correspondence between the ID of the instance 34 that executed the test 150 and the job ID corresponding to the code group 40 from the job meta information 42 and the log information 44.

Further, the log information 44 may include information on the amount of resources consumed by the instance 34 depending on the CI server 20. The resource allocation information 43 includes information for specifying the resource amount of the host 33 allocated to the instance 34 in the calculation cluster 30 in order to execute the test 150.

The calculation cluster 30 is an environment in which the test 150 is executed, and includes, for example, Kubernetes (registered trademark), OpenStack (registered trademark), and the like. The calculation cluster 30 includes a calculation cluster management device 31, a resource monitoring device 32, and one or more hosts 33-1 to 33-n.

The computing cluster management device 31 generates an instance 34 for executing the test 150 on the host 33 in response to a request from the CI server 20. The resource monitoring device 32 collects and holds the resource consumption of the instance 34. The resource consumption information held by the resource monitoring device 32 can be acquired by the resource estimation device 2 as the resource consumption result 45. The host 33 operates the instance 34. The host 33 is constructed by a physical machine or a virtual machine. The instance 34 is a virtual machine or container constructed on the host 33 in order to execute the test 150.

<Logical configuration and physical configuration>
FIG. 3 is a block diagram illustrating an example of functional elements of the resource estimation device 2. The resource estimation device 2 includes a test ID calculation unit 3, a result acquisition unit 4, a resource estimation unit 5, history information 6, and algorithm information 7.

The test ID calculation unit 3 calculates a test ID 74 that is unique for each test 150 with reference to the code meta information 41, the job meta information 42, and the test ID management table 70 shown in FIG. 5A. The result acquisition unit 4 and the resource estimation unit 5 is notified. This process will be described in detail in step P3 of FIG.

The result acquisition unit 4 uses the resource consumption result 45 acquired from the resource monitoring device 32, the log information 44 acquired from the CI server 20, and the test ID 74 acquired from the test ID calculation unit 3 to determine the instance 34 for each test ID 74. The consumption resource amount is calculated, and the value of the calculation result is stored in the history information 6. This process will be described in detail at step P10 in FIG.

The history information 6 holds information on the resource amount allocated to the instance 34 for each test ID 74. The history information 6 will be described in detail with reference to FIGS. 5A and 5B.

The resource estimation unit 5 acquires resource information 82 (detailed in FIG. 5) related to the test ID from the history information 6 based on the test ID 74 acquired from the test ID calculation unit 3. Then, the resource estimation unit 5 acquires the algorithm selected by the user terminal 50 from the algorithm information 7, and uses the algorithm, the resource amount of the instance 34 necessary for executing the test 150 specified by the test ID 74 Is calculated. This process will be described in detail at step P5 in FIG.

Algorithm information 7 holds one or more programs for executing a calculation process for estimating the resource amount. This program is called an algorithm in the first embodiment. The algorithm is a predetermined process for calculating the resource amount to be allocated to the instance 34 based on the resource information 82, and is, for example, regression analysis or an arithmetic average considering a standard deviation.

Each algorithm can provide an arbitrary margin set by the user of the user terminal 50 (for example, a resource amount 10% larger than the calculated value of the algorithm is used as the allocated resource amount) with respect to the calculated allocated resource amount. . In addition to the above, known or well-known techniques can be applied to the algorithm information 7 for estimating the resource amount.

FIG. 4 is a block diagram illustrating an example of a hardware configuration of the resource estimation device 2. The resource estimation device 2 includes an input / output device 62, a processor 63, an external storage device 64, a main storage device 65, and a communication control device 61. These are connected to each other via an internal bus 66.

The input / output device 62 is an input device such as a keyboard and a mouse for a user (or administrator) to perform various operation inputs, and an output device such as a liquid crystal display and a touch panel for displaying various information.

The processor 63 is hardware that performs arithmetic processing of the resource estimation device 2. The external storage device 64 is a storage device having a large storage capacity, such as a hard disk device or an SSD (Solid State Drive). The external storage device 64 holds the history information 6 and the algorithm information 7 shown in FIG. 2 and also holds execution files for various programs called by the main storage device 65.

The main storage device 65 is composed of a semiconductor memory, for example, and temporarily holds data called from various programs and tables. The main storage device 65 includes a program group of the test ID calculation unit 3, the result acquisition unit 4, and the resource estimation unit 5 shown in FIG. Further, the history information 6 and the algorithm information 7 may be held in the main storage device 65 instead of the external storage device 64 in order to speed up access to various tables. The external storage device 64 and the main storage device 65 can be accessed from the processor 63.

In the present embodiment, the test ID calculation unit 3, the result acquisition unit 4, and the resource estimation unit 5 will be described as programs. However, these may be configured by hardware, and software and hardware. It may be configured by wear.

The communication control device 61 is hardware having a function of controlling communication with the outside, and connects the resource estimation device 2 to the CI system 1 via the communication network 67.

The processor 63 operates as a functional unit that provides a predetermined function by processing according to the program of each functional unit. For example, the processor 63 functions as the test ID calculation unit 3 by performing processing according to the test ID calculation program. The same applies to other programs. Furthermore, the processor 63 also operates as a functional unit that provides each function of a plurality of processes executed by each program. A computer and a computer system are an apparatus and a system including these functional units.

Information such as programs and tables for realizing each function of the resource estimation device 2 is stored in an external storage device 64, a nonvolatile semiconductor memory, a hard disk drive, a storage device such as an SSD (Solid State Drive), an IC card, an SD card, It can be stored in a computer-readable non-transitory data storage medium such as a DVD.

<Table>
FIG. 5A is a diagram showing an example of the test ID management table 70 stored in the history information 6. FIG. 5B is a diagram showing an example of the history management table 80 stored in the history information 6. The test ID management table 70 is a table that holds information for calculating the test ID 74 from the code meta information 41 and the job meta information 42.

The test ID management table 70 includes a repository ID field 71, a branch ID field 72, a job ID field 73, and a test ID field 74 in one record. Each record corresponds to a test ID 74 that is unique for each test 150.

The repository ID 71 is a storage area for storing the identifier of the target repository 11. The repository ID 71 is identification information that uniquely identifies the repository 11 in which the code group 40 is managed in the SCM server 10, and is a value that is unique for each SCM server 10.

The branch ID 72 is a storage area for storing the identifier of the target branch 12. The branch ID 72 is identification information that uniquely identifies the branch 12 in the repository 11 in which the code group 40 is managed in the SCM server 10, and is a value that is unique for each repository 11.

The job ID 73 is a storage area for storing an identifier of a target job. The job ID 73 is identification information for uniquely identifying a job for executing a test in the CI server 20. Note that the correspondence relationship between the repository 11 and the job may include a plurality of jobs for one repository 11 or a plurality of repositories corresponding to one job. The job ID 73 is a unique value for each CI server 20.

Test ID 74 is a storage area for storing the identifier of the target test 150. The test ID 74 is a unique value for each test 150, and is calculated using a calculation method determined by the user terminal 50 based on the repository ID 71, the branch ID 72, and the job ID 73. The calculation method will be described in detail in step P4 of FIG. The test ID 74 is a unique value for each resource estimation device 2. *

The history management table 80 in FIG. 5B is a table that associates each test 150 with the resource information 82 assigned when the test 150 is executed.

The history management table 80 has a test ID 81 and resource information 82. Furthermore, the resource information 82 includes an allocation resource 83, a consumption resource 84, and an execution time 85 in one record. Each record represents resource information 82 when the test 150 specified by the test ID 81 is executed. In the CI system 1, since the same test 150 can be executed a plurality of times, a record with the same test ID 81 can appear a plurality of times.

Test ID 81 is a storage area for storing the identifier of the target test 150. The test ID 81 is an ID uniquely assigned to each test 150 and is the same as the test ID 74 registered in the test ID management table 70 of FIG. 5A.

The allocation resource 83 is a storage area for storing the target allocation resource 83. The allocated resource 83 is the amount of resources allocated by the CI server 20 to the instance 34 of the calculation cluster 30 during the test execution.

The consumed resource 84 is a storage area for storing the resource amount actually consumed in the test 150. The consumed resource 84 is the amount of resource actually consumed by the instance 34 of the test 150 when the test is executed. The allocation resource 83 and the consumption resource 84 can hold values for each resource type, such as the number of virtual CPU cores, the amount of memory, the amount of disk, and the network bandwidth as resource amounts.

The execution time 85 is a storage area that holds a target execution time 85. The execution time 85 is time information at the start of test execution.

In the first embodiment, various types of information are described as tables, but they may be configured by other than tables (for example, lists and associative arrays).

<Sequence diagram>
FIG. 6 is a sequence diagram showing an example of processing of the CI system 1 including the resource estimation device 2. Using this sequence diagram, after the user terminal 50 updates the code group 40 of the repository 11 of the SCM server 10, the CI server 20 causes the calculation cluster 30 to execute the test 150, and the execution result of the test 150 is displayed as the user terminal. Processing up to notification to 50 is shown.

The following shows the specific processing procedure. First, the user terminal 50 updates the code group 40 managed in the repository 11 of the SCM server 10 or the specific branch 12 in the repository 11 (P1). When the code group 40 is updated, the SCM server 10 transmits the code group 40 and the code meta information 41 to the CI server 20 (P2).

Next, the CI server 20 transmits a meta information group including the code meta information 41 and the job meta information 42 to the resource estimation device 2 (P3). The job ID included in the job meta information 42 is acquired by referring to the job information 24 in the CI server 20 based on the repository ID 71 and the branch ID 72 included in the code meta information 41.

Thereafter, the resource estimation device 2 calculates a test ID corresponding to the test 150 in the test ID calculation unit 3 (P4). For example, the resource estimation apparatus 2 refers to the history management table 80 in FIG. 5A, and the repository ID 71 included in the transmitted code meta information 41 matches the values of the job ID 73 included in the branch ID 72 and the job meta information 42. The test ID 74 of the record is acquired. On the other hand, if there is no corresponding record, the resource estimation device 2 may generate a test ID 74 with reference to the similar test 150 and register a new record.

Referring to a similar test 150, for example, the resource estimation apparatus 2 searches for records having the same ID or a set of IDs (for example, repository ID 71 and branch ID 72). Then, the resource estimation device 2 is performed by calculating its own test ID 74 with reference to the calculation rule of the test ID 74 (for example, when the test ID 74 of the retrieved record is a combination of the repository ID 71 and the branch ID 72) The own test ID 74 is also a set of the repository ID 71 and the branch ID 72).

As a result, when the branch 12 is newly generated under the repository 11 in the SCM server 10, even if there is no resource information of the target test ID 74 in the history information 6, the record having the same repository ID 71 and branch ID 72 is recorded. By substituting the resource information 82 of the history management table 80 from the test ID 74, it is possible to estimate the resource.

Further, when there is no similar test 150 search result, the resource estimation device 2 uses a predetermined ID or combination of IDs (for example, a combination of repository ID 71 and branch ID 72) as test ID 74, and sets repository ID 71 and branch ID 72 as The job ID 73 and the test ID 74 are generated and registered in the test ID management table 70 of the history information 6 as new records.

Thereafter, the resource estimation device 2 uses the resource estimation unit 5 to acquire the resource information 82 from all records including the test ID 74 from the history management table 80, and from one or more algorithms included in the algorithm information 7, the user terminal The algorithm designated by 50 is selected and the amount of resources allocated to the instance 34 of the test 150 is estimated (P5).

For example, when the user terminal 50 designates the arithmetic average as the algorithm, the resource estimation device 2 uses the instance 34 to calculate the arithmetic average of the resource amount values included in the consumed resources 84 of the history management table 80 among the acquired resource information. Estimated as the amount of resources allocated to.

When the test ID 81 in the history management table 80 does not include the test ID, the resource estimation device 2 calculates a similar test ID by a predetermined process, and uses the resource information 82 corresponding to the test ID 81. To estimate the amount of resources.

For example, when the test ID 74 is represented by a set of the repository ID 71 and the branch ID 72, the test ID 74 having the same repository ID 71 and the branch ID 72 of “master” is used as an alternative to the test ID. And so on. In the software development, when the branch 12 is newly generated, it is used to derive from the “master” branch. If there is no similar test ID 74, the resource estimation device 2 calculates a predetermined resource amount as an estimated value.

Thereafter, the resource estimation device 2 notifies the CI server 20 of the estimated resource amount as the resource allocation information 43 (P6).

Thereafter, the CI server 20 causes the calculation cluster 30 to execute the software test of the test 150 using the instance 34 to which the resource amount specified by the resource allocation information 43 is allocated (P7). Specifically, the CI server 20 requests the calculation cluster management device 31 to generate the instance 34 to which the resource amount specified by the resource allocation information 43 is allocated (P71).

The computing cluster management device 31 selects one host 33 and causes the host 33 to generate an instance 34 of the resource amount specified by the resource allocation information 43 (P72). Thereafter, the CI server 20 transmits the code group 40 received from the SCM server 10 to the generated instance 34 to execute a software test on the instance 34 (P73) and acquire the execution result of the software test (P74). ). When the software test ends, the CI server 20 requests the computing cluster management device 31 to delete the instance 34 corresponding to the software test (P75).

The computing cluster management device 31 deletes the specified instance 34 in the host 33 (P76). Note that the resource monitoring device 32 always collects information on resources consumed by the instance 34 on the calculation cluster 30 (P77).

After completion of the software test, the CI server 20 transmits the code meta information 41, job meta information 42, and log information 44 of the executed test 150 to the resource estimation device 2 (P8). The log information 44 includes the success or failure of the software test, the start time of the software test, the ID of the generated instance 34, the value of the resource amount allocated to the instance 34, and the like. Further, depending on the function of the CI server 20, the value of the resource amount consumed by the instance 34 may be included.

Thereafter, the resource estimation device 2 acquires the instance ID and the resource consumption result 45 (the amount of resources actually consumed in the software test of the test 150) from the resource monitoring device 32 (P9). The resource consumption result 45 records a preset value such as the maximum value or the average value of the resource amount used by the instance 34 when executing the software test for each resource type (eg, processor, memory, etc.). Have information. However, it is assumed that the resource amount value calculation method (maximum value or average value) is unified in the resource estimation device 2. This process may be omitted if the log information 44 includes the value of the resource amount consumed by the instance 34.

Thereafter, the resource estimation device 2 stores the result of the software test in the history information 6 (P10). Specifically, first, the resource estimation device 2 calculates a test ID from the code meta information 41 and the job meta information 42 in the test ID calculation unit 3 by the same process as in step P4. Next, the resource estimation device 2 acquires the allocation resource 83 and the execution time 85 from the log information 44 and acquires the consumption resource 84 from the resource consumption result 45 in the result acquisition unit 4. Then, the result acquisition unit 4 of the resource estimation device 2 adds the entry of the calculated test ID to the history management table 80, and stores the acquired allocated resource 83, execution time 85, and consumed resource 84.

However, if the log information 44 includes consumption resource information, the value of the consumption resource 84 may be acquired from the log information 44. If the value of the consumed resource 84 is a ratio notation such as a percentage, the resource value is converted into a resource value by a process such as setting the product of the allocated resource amount and the percentage value to the value of the consumed resource. For example, when 1 core is allocated to a virtual CPU and the CPU value of the resource consumption result is 30%, the virtual CPU consumed resource is converted to 0.3 core.

Thereafter, the resource estimation device 2 compares the value of the consumed resource 84 with a predetermined threshold value, determines whether the consumed resource 84 is close to the maximum value of the resource amount predetermined for the instance 34, and determines the maximum value. If it is close to, a notification that the load is excessive is output to the user terminal 50 (P11). Thereby, it is possible to notify the user terminal 50 that the load of the test 150 is large and the execution speed of the test 150 may be reduced. Note that, if the value of the consumed resource 84 exceeds a predetermined threshold, the resource estimation device 2 can determine that the load that the resource amount consumed by the host 33 is close to the predetermined maximum value is excessive.

Finally, the CI server 20 notifies the success or failure of the test 150 to the user terminal 50 and the SCM server 10 (P12).

Through the above processing, when the CI server 20 causes the calculation cluster 30 to execute the test 150, the resource amount of the host 33 allocated to the instance 34 that executes the test 150 using the resource estimation device 2 is set to the past test history. Estimate based on information. Focusing on the fact that most of the tests are the same processing as the previous test, and by estimating the amount of resources required for the current test 150 based on the history information of the resources allocated in the past, there is an excess of allocated resources. The amount of resources to be allocated can be reduced.

<Example of screen display>
FIG. 7 is a diagram illustrating an example of a display screen that the resource estimation device 2 presents to the user terminal 50 or the CI service manager. On this display screen 100, parameters such as an algorithm used for each test ID can be set, and a resource reduction effect can be displayed. The display screen 100 includes a test ID selection area 101, a setting display area 102, a resource reduction effect display area 103, and a history information display area 104. The display areas other than the test ID selection area 101 do not necessarily have to be displayed on the same screen, and may be displayed as independent screens.

In the test ID selection area 101, the user terminal 50 or the CI service administrator selects a test ID. In the setting display area 102, the resource reduction effect display area 103, and the history information display area 104, information related to the test ID selected in this area is displayed.

In the setting display area 102, the parameters set in the selected test ID are displayed, and the user terminal 50 or the CI service administrator can confirm or change the parameters from the pull-down menu in this area. The selectable parameters include a test ID setting method, an algorithm for estimating resources, the number of histories to be referenced (number of records in the history management table 80), a margin for correcting the resource estimation results, and the like.

The margin is a resource due to a difference between the estimated value and the actually consumed value by adding a predetermined margin set by the GUI, API, or setting file to the value estimated as the resource consumption allocated to the test 150. The shortage can be suppressed.

The test ID setting method is a parameter for determining what combination is set as the test ID 74 from the repository ID 71, the branch ID 72, and the job ID 73. When the test ID setting method is changed, the resource estimation device 2 updates the values of the test ID management table 70 and the test ID 74 in the history management table 80 in the history information 6. The reference history number is a value that sets how many pieces of history information are referred to in the past when the resource is estimated.

In the update of the test ID setting method, when a plurality of test IDs 74 are integrated into one, the test ID 74 and its history information are updated according to a predetermined procedure. For example, when a value that is a combination of the repository ID 71 and the branch ID 72 is originally used as the test ID 74 and only the repository ID 71 is changed to be used as the test ID 74 by this setting screen, the resource estimation device 2 performs the test ID management. In the table 70, the test ID 74 of all records having the repository ID is changed to the value of the repository ID 71, and all records having the test ID 74 including the repository ID 71 are changed from the history management table 80 to the repository ID 71. Processing such as integration as a new record as a test ID is performed.

The resource reduction effect display area 103 is an area for displaying the amount and ratio of resources reduced by the resource estimation device 2. The value displayed here is, for example, an average value of values obtained by subtracting the value of the resource amount actually assigned from the resource amount assigned by default.

The history information display area 104 is an area for displaying a time-series transition of the allocation amount and actual consumption amount of each resource. This area is represented by, for example, an area graph with the vertical axis representing the resource amount and the horizontal axis representing time. A graph is generated for each resource value. The graph shows, for example, a default allocation amount 105 indicating a resource amount allocated by default, a resource allocation graph 106 indicating a transition of the resource amount allocated to each test, and a transition of the resource amount actually consumed in each test. A resource consumption graph 107 is included.

The parameter setting method is not limited to the setting from the display screen. For example, an API (Application Programming Interface) or a setting file may be used.

Through the above processing, in the first embodiment, the resource estimation device 2 estimates the resource amount necessary for each test, thereby changing the allocated resource according to the test and reducing the resource consumption during the test execution. become.

As described above, the resource estimation device 2 generates the test ID 74 from the information (code meta information 41 and job meta information 42) specifying the test 150 received from the CI server 20, and stores the test ID 74 in the history management table 80 that matches the test ID 74. The resource amount (consumed resource 84) consumed during the past execution of the test 150 is acquired from the record.

Then, the resource estimation device 2 estimates the resource amount to be allocated to the current test 150 based on the past resource consumption (consumed resource 84), and notifies the CI server 20 of the estimated resource amount as the resource allocation information 43. .

The CI server 20 requests a resource from the calculation cluster 30 with the resource allocation information 43 and causes the host 33 to generate an instance 34. As a result, an instance 34 to which the resource amount estimated by the resource estimation device 2 based on the past resource consumption is assigned to the host 33 is generated.

Then, the CI server 20 transmits the test code 151 and the source code to the calculation cluster 30 and causes the instance 34 to execute the test 150. When the test 150 is completed, the CI server 20 receives the result of the test 150 from the host 33, and the resource estimation apparatus 2 receives the code meta information 41, the job meta information 42, and the log information 44 (including the allocation resource 83 and the execution time 85). ). The resource estimation device 2 receives the instance ID and the resource consumption result 45 (actual consumption resource 84) from the resource monitoring device 32.

Then, the resource estimation device 2 adds an entry of the test ID corresponding to the code meta information 41 and the job meta information 42 to the history management table 80, and stores the acquired allocation resource 83, execution time 85, and consumed resource 84. Thereby, for each test ID, the resource amount actually consumed in the past test 150 is accumulated as the consumed resource 84, and is used for estimation of the next resource amount. Thereby, the resource estimation device 2 can change the amount of computer resources allocated to the test 150 by estimating the resource amount necessary for each test 150 from the past results, Resource consumption can be reduced.

In the first embodiment, the resource estimation device 2 receives the resource consumption result 45 from the resource monitoring device 32. However, the resource estimation device 2 receives the resource consumption result from the resource monitoring device 32 via the CI server 20. 45 may be received.

As a second embodiment, processing of the resource estimation device 2 when estimating the consumed resource amount of each instance 34 for a test 150 that generates a plurality of instances 34 will be described. However, the description of the same processing or configuration as in the first embodiment is omitted.

When a plurality of instances 34 are generated in one test 150, the resource amount consumed for each instance 34 in the test 150 is estimated, and a history of the resource amount consumed for each instance 34 when the test 150 is executed is acquired. There is a need. In the second embodiment, the instance 34 and the resource amount are associated with each other based on the test code 151, the setting information included in the test code 151, and the log information 44 of the CI server 20.

For this reason, the resource estimation device 2 performs the code group in addition to the code meta information 41, the job meta information 42, the log information 44, and the resource consumption result 45 acquired in the first embodiment when estimating the resource or acquiring the resource consumption result 45. 40 is also acquired from the CI server 20.

<Table>
FIG. 8 is a diagram illustrating a table configuration example of the history management table 80A stored in the history information 6 in the second embodiment. Note that the configuration example of the test ID management table 70 is the same as that of the first embodiment, and is therefore omitted. The history management table 80A of the second embodiment is different from the history management table 80 of the first embodiment in that an instance name 211 field is added.

The history management table 80A is a table that associates the instance 34 generated by each test 150 with its resource information 82, and includes a test ID 81, an instance name 211, and resource information 82 in one record. The resource information 82 includes an allocation resource 83, a consumption resource 84, and an execution time 85. Each record holds resource information 82 when each instance 34 generated by the test 150 specified by the test ID 81 is executed. For this reason, the same test ID 81 and instance name 211 can appear in a plurality of records. Since the test ID 81 and the resource information 82 are the same as those in the first embodiment, the description is omitted.

The instance name 211 is a storage area for storing the name of the target instance 34. The instance name 211 is a name in the test code 151 of each instance 34 generated by the test 150, and is calculated based on the setting information included in the test code 151 and the test code 151. The calculation method will be described in detail in step P14 of FIG. The instance name 211 is unique for each test ID 81.

<Sequence diagram>
FIG. 9 is a sequence diagram illustrating an example of processing performed in the CI system 1 including the resource estimation device 2 according to the second embodiment. In this sequence diagram, from when the user terminal 50 updates the code group 40 of the repository 11 of the SCM server 10 until the calculation cluster 30 executes the test 150 and notifies the user terminal 50 of the execution result of the test 150. Indicates processing. In FIG. 9, the sequence represented by the dotted line is the same processing as that in FIG. 6 of the first embodiment, so that the description thereof will be omitted or only the related portions will be described briefly.

After the SCM server 10 transmits the code group 40 and the code meta information 41 to the CI server 20 (P2), the CI server 20 transmits the code meta information 41, the job meta information 42, and the code group 40 to the resource estimation device 2. (P200).

Thereafter, the resource estimation device 2 calculates a test ID corresponding to the test 150 in the test ID calculation unit 3 (P4).

After that, the resource estimation device 2 estimates the resource amount allocated to each instance 34 generated by the test ID in the resource estimation unit 5 (P201). Specifically, the resource estimation device 2 selects the instance name 211 from the test code 151 included in the code group 40 and the setting information accompanying the test code 151. Next, the resource estimation device 2 acquires the resource information 82 of the selected instance name 211 among the records related to the test ID from the history management table 80A for each instance name 211.

Then, the resource estimation device 2 estimates the resource amount to be allocated for each instance 34 using an algorithm selected by the user terminal 50 from one or more algorithms included in the algorithm information 7. If the test ID is not present in the history management table 80A, a similar test ID is referred to by a predetermined method as in the first embodiment, or a predetermined resource amount is calculated as an estimated value.

The case where the instance name 211 described in the code group 40 is not in the history management table 80A is the same as the case where there is no test ID described above, and the resource estimation device 2 detects a similar test ID 74 using a predetermined method. Then, it is determined whether or not the same instance name 211 exists in the detected test ID 74. If the same instance name 211 exists, the value is referred to. When the same instance name 211 does not exist, the resource estimation device 2 calculates a predetermined resource amount as the allocated resource amount of the instance name 211.

Thereafter, the resource estimation device 2 transmits the resource allocation amount for each instance name 211 to the CI server 20 as the resource allocation information 43 (P202). Thereafter, the CI server 20 allocates the resource amount specified by the resource allocation information 43 to each instance 34, and causes the software test of the test 150 to be executed (P7). The software test execution process is the same as that shown in FIG.

After completion of the software test, the CI server 20 transmits the code meta information 41, job meta information 42, and log information 44 of the executed test 150 to the resource estimation device 2 (P8). As shown in FIG. 2, the log information 44 includes the success or failure of the test 150, the start time of the test 150, the ID of the generated instance 34, the value of the resource amount allocated to each instance 34, the instance name 211 and the instance ID, Are included.

Depending on the function of the CI server 20, the value of the resource amount consumed by the instance 34 may be included.

Thereafter, the resource estimation device 2 acquires the resource consumption results 45 of the plurality of instance IDs included in the log information 44 from the resource monitoring device 32 (P203). The resource consumption result 45 is obtained by recording the maximum value or the average value of the resource amount used for each instance 34 for each resource (for example, CPU, memory, etc.). However, it is assumed that the resource estimation device 2 unifies whether the value of the resource amount is the maximum value or the average value. This process may be omitted if the log information 44 includes the value of the resource amount consumed by the instance 34.

After that, the resource estimation device 2 stores the result of the test 150 in the history information 6 (P204). Specifically, first, the resource estimation device 2 calculates a test ID from the code meta information 41 and the job meta information 42 in the test ID calculation unit 3 by the same process as in step P4.

Next, the result acquisition unit 4 of the resource estimation device 2 acquires the execution time 85 of the software test, the correspondence between the instance name 211 and the instance ID, and the allocated resource 83 for each instance ID from the log information 44, and resource consumption The consumption resource 84 for each instance ID is acquired from the result 45. Then, the resource estimation device 2 stores the allocation resource 83, consumption resource 84, and execution time 85 of each instance 34 together with the calculated test ID in the history management table 80 in association with the instance name. Here, when the log information 44 includes consumption resource information, the value of the consumption resource 84 may be acquired from the log information 44. Since the subsequent processing is the same as that of the first embodiment, the description thereof is omitted.

Through the above processing, the resource estimation device 2 can estimate the resource amount of each instance 34 for the test 150 that generates a plurality of instances 34 in the second embodiment.

FIG. 10 is a block diagram showing an example of the configuration of the CI server 20 and the calculation cluster 30 according to the third embodiment of the present invention. In the first embodiment, the CI server 20 and the resource estimation device 2 are configured as independent computers. However, the third embodiment is an example in which the CI server 200 and the resource estimation device are integrated as a CI server 200. Indicates.

In the first embodiment, the calculation cluster management device 31 and the resource monitoring device 32 are configured as independent computers in the calculation cluster 30. However, in the third embodiment, the calculation cluster management device 31 and the resource monitoring device 32 are integrated into one management computer 300. An example is shown.

The CI server 200 includes a CI service unit 20A and a resource estimation unit 2A. The resource estimation unit 2A functions in the same manner as the resource estimation device 2 of the first embodiment, and outputs the resource allocation information 43 to the resource estimation cooperation function 22. The CI service unit 20A functions in the same manner as the CI server 20 of the first embodiment, and manages software tests executed on the calculation cluster 30. As in the third embodiment, the CI server 20 and the resource estimation device 2 can be integrated into one computer.

In the calculation cluster 30, the management computer 300 includes a calculation cluster management unit 31A and a resource monitoring unit 32A. The calculation cluster management unit 31A functions in the same manner as the calculation cluster management device 31 of the first embodiment. The resource monitoring unit 32A functions in the same manner as the resource monitoring device 32 of the first embodiment.

As in the third embodiment, the calculation cluster management unit and the resource monitoring unit can be integrated into one management computer 300 in the calculation cluster 30 as well.

<Summary>
In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments are described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. In addition, any of the additions, deletions, or substitutions of other configurations can be applied to a part of the configuration of each embodiment, either alone or in combination.

In addition, each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. In addition, each of the above-described configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files for realizing each function can be stored in a recording device such as a memory, a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

Also, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

Claims

A CI server having a processor and memory for managing software testing;
A continuous integration system including one or more computers having a processor and a memory, and a computing cluster that executes a test of the software in response to a request from the CI server,
The CI server
A CI service unit that receives software for executing the test, determines a resource to be allocated to the test, notifies the calculation cluster of the resource, causes the test to be executed, and obtains an execution result from the calculation cluster;
A resource estimation unit that estimates a resource to be allocated to the test in response to a request from the CI service unit;
The resource estimation unit
A test ID calculation unit for assigning a unique test ID for each test;
A result acquisition unit for acquiring information of resources consumed during execution of the test for each test ID;
History information for storing the test ID executed in the past and the acquired resource information in association with each other;
When resource estimation is requested from the CI service unit, a resource necessary for execution of a test is estimated for each test ID based on the history information, and the estimation result is notified to the CI service unit as resource allocation information And an estimation unit that
The CI service unit
A continuous integration system, wherein a resource estimation request to be allocated to the test is transmitted to a resource estimation unit, and a resource estimation result received from the resource estimation unit is determined as a resource to be allocated to the test.
A continuous integration system according to claim 1, comprising:
The CI service unit
As the software for executing the test, code meta information including an identifier for specifying a test code and a source code is received, a job ID corresponding to the test is generated, and the code meta information and the job ID are included in the resource estimation request. And send it to the resource estimation unit,
The test ID calculator of the resource estimator is
A continuous integration system characterized by receiving the resource estimation request and calculating the test ID from the code meta information and the job ID.
A continuous integration system according to claim 2, comprising:
The code meta information includes a repository ID and a branch ID for storing the test code and source code,
The test ID calculation unit
A continuous integration system, wherein the test ID is calculated using at least one of the repository ID, branch ID, and job ID.
A continuous integration system according to claim 3,
The history information is
Including the repository ID, branch ID, job ID, and test ID,
The estimation unit includes
When the history information does not include resource information that matches the test ID, the resource information of the test ID that matches the repository ID and branch ID is substituted.
A continuous integration system according to claim 1, comprising:
The resource estimation unit
Algorithm information storing a plurality of algorithms for estimating the resource,
The CI service unit
The software for executing the test and the algorithm to be estimated are received, and the algorithm is included in the resource estimation request to be allocated to the test and transmitted to the resource estimation unit.
The resource estimation unit
The continuous integration system, wherein the algorithm is received and the estimation unit estimates the resource using the algorithm.
A continuous integration system according to claim 1, comprising:
The CI service unit
Accepting a margin of resources to be estimated as software for executing the test, and transmitting the resource estimation unit including the margin in an estimation request for resources to be allocated to the test,
The resource estimation unit
The continuous integration system, wherein the margin is received, and the estimation unit estimates the resource by adding the margin.
A continuous integration system according to claim 1, comprising:
The result acquisition unit
Select a resource amount consumed from the acquired resource information, and if the resource amount exceeds a predetermined threshold, output a notification indicating that the load of the test is excessive Integration system.
A continuous integration system according to claim 1, comprising:
The history information is
Including an instance name for executing the test, storing the test ID, one or more instance names, and resource information in association with each other;
The estimation unit includes
A continuous integration system characterized by estimating a resource necessary for execution of the test for each instance name in the test ID.
A continuous integration system according to claim 1, comprising:
The history information is
Including information on resources allocated when performing the test,
The resource estimation unit
A continuous integration system for generating, for each test ID, a screen that displays information on resources consumed and information on resources allocated during execution of the test in time series.
A CI server having a processor and a memory and managing a software test controls resources of the computer when causing a calculation cluster including one or more computers having a processor and a memory to execute the software test. A resource control method. A resource control method comprising:
A first step in which a CI service unit of the CI server receives software for executing the test;
A second step in which a CI service unit transmits a resource estimation request to be allocated to the test to the resource estimation unit of the CI server;
A third step in which the resource estimation unit receives a request for estimating a resource to be assigned to the test and assigns a unique test ID for each test;
Resources necessary for execution of a test for each test ID based on history information in which the resource estimation unit stores the test ID executed in the past and information on resources consumed during the execution of the test in association with each other And a fourth step of notifying the CI service unit of the estimation result as resource allocation information;
A fifth step in which the CI service unit receives the resource allocation information, notifies the resource allocation information to the calculation cluster, and executes the test;
A sixth step in which the CI service unit obtains an execution result of the test from the calculation cluster;
A seventh step in which the resource estimation unit acquires information on resources consumed during the execution of the test and stores the information in the history information;
A resource control method that includes a special information.
The resource control method according to claim 10, wherein
The first step includes
As software for executing the test, code meta information including an identifier for specifying a test code and a source code is received, and a job ID corresponding to the test is generated,
The second step includes
The code meta information and the job ID are included in the resource estimation request and transmitted to the resource estimation unit,
The third step includes
A resource control method, wherein the test ID is calculated from the code meta information and the job ID.
The resource control method according to claim 11, comprising:
The code meta information includes a repository ID and a branch ID for storing the test code and source code,
The third step includes
A resource control method, wherein the test ID is calculated using at least one of the repository ID, branch ID, and job ID.
The resource control method according to claim 12, comprising:
The history information is
Including the repository ID, branch ID, job ID, and test ID,
The fourth step includes
A resource control method characterized by substituting information of a resource of a test ID with a matching repository ID and a branch ID when there is no resource information matching the test ID in the history information.
The resource control method according to claim 10, wherein
The first step includes
Accepts software to perform the test and an estimation algorithm;
The second step includes
Including the algorithm in a request for estimating a resource to be allocated to the test, and transmitting it to the resource estimation unit;
The fourth step includes
Receiving the algorithm, selecting the received algorithm from algorithm information storing a plurality of algorithms set in advance to estimate the resource, and estimating the resource using the algorithm Control method.
The resource control method according to claim 10, wherein
The first step includes
Accept the software to perform the test and the margin of resources to estimate,
The second step includes
Including the margin in a request for estimating a resource to be allocated to the test, and transmitting it to the resource estimation unit;
The fourth step includes
A resource control method characterized by receiving the margin and estimating the resource by adding the margin.