JP6300750B2 - Verification program, verification apparatus, and verification method - Google Patents

Description

  The present invention relates to a verification program, a verification apparatus, and a verification method.

  In recent years, API (Application Programming Interface) has been actively used in programs. A user can effectively utilize functions provided by an OS (Operating System) or a library (Library) by using the API. Also, various test methods have been proposed in order to efficiently verify the behavior of API (or software that implements API) and perform debugging work.

  For example, a technique is known that can always create the same test condition by acquiring the system state at the time of test execution by API call and controlling related software in accordance with the acquired state (for example, Patent Document 1).

  In addition, a technology is known that can easily perform a test even if an unknown function occurs by building a test case by receiving information from a plug-in module (for example, a patent) Reference 2). In addition, when processing an external function that cannot use source code such as an API, a technique is known in which a model is provided and whether or not the external function is appropriately called is evaluated (for example, Patent Document 3). ).

International Publication No. 2009/150788 JP 2009-223700 A JP 2006-236336 A

  However, the above-described conventional technology cannot always perform exhaustive verification efficiently. Specifically, when the API to be verified has a predetermined relationship such as using the processing result of another API, the verification result may change depending on information issued as the processing result by another API. . In such a case, according to the above-described conventional technology, it is difficult to appropriately verify the API to be verified because information issued by other APIs as processing results cannot be properly grasped. In addition, it is possible to manually set combinations of information issued by other APIs as processing results, but such combinations are enormous and are not realistic.

  The present application has been made in view of the above, and an object thereof is to provide a verification program, a verification apparatus, and a verification method capable of efficiently performing comprehensive verification.

  The verification program according to the present application includes a creation procedure for creating a test case to be executed for a verification target API (Application Programming Interface) that is a verification target, the verification target API, and when the verification target API is executed. A setting procedure for setting a hierarchical relationship with another API that has a relationship of returning reference information that is information to be used as a return value, and the other layer in an upper layer based on the hierarchical relationship set by the setting procedure The reference information is acquired by calling an API, and a verification procedure for verifying a verification target API using the acquired reference information and the test case is executed by a computer.

  According to one aspect of the embodiment, there is an effect that exhaustive verification can be performed efficiently.

FIG. 1 is a diagram illustrating an example of a verification process according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of the verification apparatus according to the embodiment. FIG. 3 is a diagram illustrating the campaign API according to the embodiment. FIG. 4 is a diagram illustrating the advertisement group API according to the embodiment. FIG. 5 is a diagram (1) illustrating the advertising API according to the embodiment. FIG. 6 is a diagram (2) illustrating the advertising API according to the embodiment. FIG. 7 is a diagram (1) illustrating a creation process according to the embodiment. FIG. 8 is a diagram (2) illustrating the creation process according to the embodiment. FIG. 9 is a diagram (3) illustrating the creation process according to the embodiment. FIG. 10 is a diagram (1) illustrating the verification process according to the embodiment. FIG. 11 is a diagram (2) illustrating the verification process according to the embodiment. FIG. 12 is a diagram (3) illustrating the verification process according to the embodiment. FIG. 13 is a flowchart illustrating a creation processing procedure performed by the verification apparatus according to the embodiment. FIG. 14 is a flowchart illustrating a verification processing procedure performed by the verification apparatus according to the embodiment. FIG. 15 is a hardware configuration diagram illustrating an example of a computer that realizes the function of the verification apparatus.

  Hereinafter, modes for carrying out a verification program, a verification device, and a verification method according to the present application (hereinafter referred to as “embodiments”) will be described in detail with reference to the drawings. In addition, the verification program, verification apparatus, and verification method which concern on this application are not limited by this embodiment. In the following embodiments, the same portions are denoted by the same reference numerals, and redundant description is omitted.

[1. Example of verification process)
First, an example of the verification process according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a verification process according to the embodiment. In FIG. 1, the verification process 1 according to the embodiment will be described using the verification system 1 as an example. Specifically, FIG. 1 illustrates a flow of processing (corresponding to the verification program according to the present application) in which the verification apparatus 100 included in the verification system 1 verifies the operation of the API to be verified.

As illustrated in FIG. 1, the verification system 1 includes a user terminal 10, execution servers 20 ₁ , 20 ₂ , and 20 _3, and a verification device 100. The verification device 100 is communicably connected to the user terminal 10 and the execution servers 20 ₁ , 20 ₂ , and 20 ₃ via a communication network (for example, the Internet) (not shown). In the following, when it is not necessary to distinguish the executing server ₂₀ 1, 20 _2, and 20 _3, referred to as executing server 20. The number of user terminals 10 and execution servers 20 included in the verification system 1 is not limited to the example illustrated in FIG. For example, the verification system 1 may include a plurality of user terminals 10 and may include more execution servers 20.

  The user terminal 10 is an information processing device used by a user. Specifically, the user terminal 10 displays a UI (User Interface) of a verification program provided by the verification device 100 in order to verify the operation of the API to be verified, or issues a program execution instruction to the verification device 100. It is used for sending. The user terminal 10 is, for example, a mobile terminal such as a smartphone, a tablet terminal, or a PDA (Personal Digital Assistant), a desktop PC (Personal Computer), a notebook PC, or the like. The general user in the embodiment is, for example, an API provided by the execution server 20 and an API that provides a function for setting an advertisement to be distributed on the web network (hereinafter referred to as “API related to advertisement”). Advertisers who want to use this may apply.

The execution server 20 is a server device that executes an API. For example, the execution server 20 provides an API to the user via the Internet. Then, the execution server 20 executes the API according to predetermined input and control from the user. For example, the execution server 20 provides an API related to the advertisement to the user, accepts the setting of the bid related to the advertisement, saves the user from having to distribute the advertisement on the predetermined website, Execute the process to display the advertisement. In the embodiment, the execution server 20 receives a setting related to verification or executes an API by a verification program executed by the verification apparatus 100. The execution servers 20 ₁ , 20 ₂ , and 20 ₃ hold different APIs, and execute each process according to the control transmitted from the verification device 100. When using the API of each execution server 20, the user terminal 10 and the verification device 100 specify information that identifies the execution server 20. For example, the user terminal 10 and the verification apparatus 100 specify a URL (Uniform Resource Locator) that identifies the execution server 20.

  In the embodiment, the execution server 20 provides an API (hereinafter, may be referred to as “web API”) used in the web network. Web APIs may have mutual usage relationships among APIs. The mutual usage relationship means, for example, that the web APIs have a hierarchical relationship of upper and lower levels. For example, the lower layer web API may acquire a return value that is a result of a predetermined process of the upper layer web API, and perform processing using the acquired return value. In this case, if the lower layer web API cannot acquire the return value obtained from the upper layer web API, the processing may result in an error. In the following description, information such as a return value issued by the upper API and referred to for execution of the lower API may be referred to as “reference information”. The reference information is predetermined ID information issued as a processing result of the upper layer API, and can identify a process performed by the upper layer API. For example, the reference information is composed of numerical values of several digits.

  The verification apparatus 100 creates a test case for testing (verification) the operation of an API to be verified (hereinafter sometimes referred to as “verification target API”), and processes the API according to the test case. Device. The verification apparatus 100 can execute the verification target API test comprehensively and efficiently by executing the verification program. Hereinafter, the verification process by the verification system 1 and the verification apparatus 100 will be described along the flow with reference to FIG.

  First, in the example illustrated in FIG. 1, the verification apparatus 100 provides the user terminal 10 with a UI that is used by the user to set information related to an API test or the like in the verification program (step S11). That is, the user terminal 10 displays a UI related to the verification program on the display unit and accepts input from the user.

  Here, the user terminal 10 sets the hierarchy of the verification target API (step S12). As described above, when the verification target API is a web API, the APIs may have a hierarchical relationship. And if there is a hierarchical relationship, if an upper layer API is not set for the verification target API, there will be omissions in test items in the verification or errors will occur, so comprehensive verification will be possible. It may not be possible. Therefore, the user terminal 10 can cause the verification program to perform a test without omission by setting the hierarchy of the verification target API.

  Further, the user terminal 10 creates an API test case by selecting a predetermined test pattern provided by the verification program. Here, the test pattern indicates a predetermined operation to be executed by the API for verification. For example, in the API having an item that accepts an input from the user, the test pattern is input to the required input item provided, or whether a value exceeding the maximum value set in the input item is input. Specifically, the pattern is an action (behavior) predefined as an item that the API specifically performs as a test. Note that the user terminal 10 can also create a test pattern other than the predetermined test pattern provided by the verification program depending on the application. The test case is a collection of test patterns, and indicates a series of test items executed by the verification program for the API. The test case also includes information about the environment in which the test is performed.

In this way, the verification apparatus 100 accepts transmission of information set from the user terminal 10 such as an API hierarchy. And the verification apparatus 100 produces a test case based on the information set from the user terminal 10 (step S13). Then, the verification apparatus 100 tests the verification target API by executing a test case according to the control from the user terminal 10. Note that the verification apparatus 100 may access the execution servers 20 ₁ , 20 ₂ , and 20 ₃ as necessary to test the APIs held by each of them (step S14).

  Here, when executing the test, the verification program according to the verification apparatus 100 causes the upper API of the verification target API to issue reference information in order from the API located in the uppermost layer. That is, the verification program causes the test pattern to be executed by the verification target API to be tested in a state where all reference information issued from the API located in the upper layer is input. That is, according to the verification program, all the possible results are comprehensively included in the test pattern executed by the verification target API, even if the test pattern has a mutual relationship such that the operation result of another API is incorporated. Can be acquired.

  And the verification apparatus 100 notifies the user terminal 10 of the test result performed with respect to verification object API (step S15). For example, the verification apparatus 100 notifies the user terminal 10 of the result by displaying a list of test cases performed on the verification target API on the UI. In this case, the verification apparatus 100 may display a plurality of results for one test pattern. This is because the verification target API is executed for each different reference information in order to verify one test pattern because the reference information issued from the upper layer API is different in the API test. In other words, the verification apparatus 100 may dynamically create a plurality of test environments (test cases) for one test pattern based on the hierarchical relationship set in the verification target API.

  As described above, the verification apparatus 100 according to the embodiment creates a test case to be executed for the verification target API that is the verification target, and uses the verification target API and the reference used when the verification target API is executed. Setting is made for a hierarchical relationship with other APIs that have a relationship of returning information as a return value. Then, the verification apparatus 100 performs verification using a test case while acquiring reference information used for executing the verification target API by sequentially calling other APIs from the higher hierarchy based on the set hierarchical relationship. Validate the target API.

  Thereby, the verification apparatus 100 which concerns on embodiment can perform efficiently the verification process of API which has mutual relations, such as web API. That is, the user who performs the test can execute the test for the verification target API by setting the hierarchical relationship of the verification target API without being aware of the combination of the return values that can be taken by the higher-level API. . As described above, the verification apparatus 100 can efficiently perform exhaustive verification on an API having a hierarchical relationship.

[2. Configuration of verification device]
Next, the configuration of the verification apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration example of the verification apparatus 100 according to the embodiment. As illustrated in FIG. 2, the verification apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The verification apparatus 100 includes an input unit (for example, a keyboard and a mouse) that receives various operations from an administrator who uses the verification apparatus 100, and a display unit (for example, a liquid crystal display) that displays various types of information. You may have.

(About the communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card). The communication unit 110 is connected to a communication network in a wired or wireless manner, and transmits and receives information to and from the user terminal 10 and the execution server 20 via the communication network.

(About the storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 according to the embodiment includes an API information storage unit 121, a test case storage unit 122, and a test result storage unit 123. Hereinafter, each storage unit will be described in order.

(About the API information storage unit 121)
The API information storage unit 121 stores information related to the API that the verification apparatus 100 performs verification. For example, the API information storage unit 121 is described as a field in identification information (for example, API name and type) for identifying an API, information on API variables and information (for example, JAVA (registered trademark)), and the like. And the location where the API is held (for example, the URL of the execution server 20).

  For example, the user can refer to the API information stored in the API information storage unit 121 via the UI when creating a test case that is a test item for verification.

(About test case storage unit 122)
The test case storage unit 122 stores information on test cases used for API verification. For example, the test case storage unit 122 identifies the verification target API that is the verification target API, the hierarchical information set in the verification target API, the type of test pattern that is actually executed as a test, The type of method to be stored is stored.

  For example, when creating a test case for the verification target API, the user can refer to the information on the test case stored in the test case storage unit 122 via the UI. Thereby, the user can confirm the type of the test pattern executed in the test case.

(About test result storage unit 123)
The test result storage unit 123 stores a result after the test for the behavior of the verification target API using the test case. For example, the test result storage unit 123 determines whether information about the environment in which the test was performed, each test pattern, a state expected for each test pattern, and a state actually expected have occurred. The verified result is stored.

  Further, as described above, the test result may differ depending on the reference information issued by the upper API. In other words, since the verification target API receives the reference information issued by the upper layer API and executes the process, there are cases where a plurality of results are returned even if one test pattern is executed. In such a case, the test result storage unit 123 comprehensively stores the results of all the executed test patterns as the results for the executed test cases.

  For example, when a test is performed on the verification target API using a predetermined test case, the user can refer to the result of the test case stored in the test result storage unit 123 via the UI. .

(About the control unit 130)
The control unit 130 is configured such that, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like stores various programs (corresponding to an example of the verification program) stored in a storage device inside the verification apparatus 100 as a work area. This is realized by being executed as The control unit 130 is realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  As illustrated in FIG. 2, the control unit 130 according to the embodiment includes a UI providing unit 131, a creation unit 132, a reception unit 133, a setting unit 134, a verification unit 135, an execution unit 136, and a storage unit 137. And implements or executes the functions and operations of information processing described below. The internal configuration of the control unit 130 is not limited to the configuration illustrated in FIG. 2, and may be another configuration as long as the information processing described later is performed. In addition, the connection relationship between the processing units included in the control unit 130 is not limited to the connection relationship illustrated in FIG. 2, and may be another connection relationship.

(About the UI providing unit 131)
The UI providing unit 131 provides a UI related to the verification process. Specifically, the UI providing unit 131 provides a UI to be displayed on the display unit of the user terminal 10 so that the user can use the verification program provided by the verification device 100. As an example, the UI providing unit 131 provides a UI for displaying a result verified by the verification unit 135 described later.

  The user performs various settings related to the API verification process via the UI provided by the UI providing unit 131. Further, the user refers to the test pattern displayed on the UI, and selects the test pattern as an item to be actually tested. Then, the user creates a test case in which the selected test patterns are collected.

  In addition, the UI providing unit 131 provides a UI for displaying a result verified by the verification unit 135 described later. Accordingly, the user can refer to the test result via the UI after the API test is performed. At this time, the UI providing unit 131 can also provide information on other APIs used for verification of the verification target API together with the verification result of the verification target API via the UI. The other API used for verification of the verification target API specifically indicates an API in which a hierarchical relationship is set for the verification target API. For example, the UI providing unit 131 provides API identification information (for example, name) that is actually used as reference information in the executed script of the API to be verified.

  That is, the UI providing unit 131 exchanges information with the creation unit 132, the verification unit 135, the storage unit 120, and the like, which will be described later, to provide the user with test contents to be executed, test results, and the like. .

(About the creation unit 132)
The creation unit 132 creates a test case for the verification target API that is the verification target. For example, the creation unit 132 creates a test case desired by the user by receiving a setting regarding a test to be executed on the verification target API from the user. In creating a test case, the creation unit 132 acquires information related to the API. The creation unit 132 may acquire information related to the API using information transmitted from the user, or may acquire information related to the API from an external providing device. In this case, the creation unit 132 stores the acquired information in the API information storage unit 121. As illustrated in FIG. 2, the creation unit 132 includes a reception unit 133 and a setting unit 134.

(About the reception unit 133)
The accepting unit 133 accepts settings related to the test case. Specifically, the reception unit 133 receives a setting related to a test case transmitted from the user terminal 10 via a UI provided by the UI provision unit 131.

  For example, the accepting unit 133 accepts designation of a verification target API that is an API to be verified. The accepting unit 133 accepts setting of environment information (for example, fields) of arrays and variables in the verification target API.

  The accepting unit 133 accepts designation of a hierarchical relationship between the verification target API and another API that has a relationship of returning reference information used when the verification target API is executed as a return value. That is, when the verification target API designated by the user is a web API and there is an API having a hierarchical relationship for executing processing using reference information or the like, the accepting unit 133 accepts such designation from the user. Such a hierarchical relationship is set by the setting unit 134 described later.

  The accepting unit 133 accepts the type (method) of API processing executed as a test. For example, the reception unit 133 receives a specific method designation such as “add” or “set” as a method. The accepting unit 133 accepts an API reference and accepts a test pattern corresponding to the API reference.

  The test pattern is information input to the verification target API during the test, and some patterns are prepared in advance by the verification program. For example, as a test pattern, patterns such as “essential item × default value” and “essential item × minimum value” are prepared in advance as input information. As an example, if “required item × default value”, the test pattern indicates a situation in which a default value is input for an item that is required to be input in the API.

(About the setting unit 134)
The setting unit 134 performs settings for creating a test case based on the information received by the receiving unit 133. For example, the setting unit 134 sets the hierarchical relationship between the verification target API and another API that has a relationship of returning reference information used when the verification target API is executed as a return value.

  The test case in which the hierarchical relationship is set by the setting unit 134 can perform processing based on the hierarchical relationship when executing the test of the verification target API. For example, when the verification target API executes a script that uses reference information, if the reference information is not acquired, an error is returned as a result. On the other hand, when the verification target API is executed using the test case in which the hierarchical relationship is set by the setting unit 134, the reference information acquired by calling the higher-level API is input to the executed script. Thereby, the verification program can perform the process for the verification target API without causing an error.

  Further, the setting unit 134 can set a hierarchy level, which is information indicating the order of the hierarchical relationship with respect to the verification target API, for each API. For example, when there is a higher-level API that issues a return value referred to by the verification target API, the setting unit 134 sets the higher-level API to “hierarchy level 1” and returns the verification target API to reference. An API for issuing a value can be set as “hierarchy level 2”. That is, the setting unit 134 can set a multi-layer relationship for the verification target API.

  Thus, based on the processing by the reception unit 133 and the setting unit 134, the creation unit 132 creates a test case desired by the user. In addition, the creation unit 132 stores the created test case in the test case storage unit 122.

(About the verification unit 135)
The verification unit 135 verifies the API to be verified. For example, the verification unit 135 acquires reference information issued from an API in an upper layer by calling an API in an upper layer based on the hierarchical relationship set by the setting unit 134, and the acquired reference information The verification target API is verified using the test case created by the creation unit 132.

  Note that the verification unit 135 determines that the hierarchical level set by the setting unit 134 is higher when there is an upper-level API in which the hierarchical level that is information indicating the order of the hierarchical relationship with the verification target API is set by the setting unit 134. The reference information is acquired by calling the upper-level APIs in this order, and the verification target API is verified using the acquired reference information. Specifically, when a plurality of APIs that are higher layers are set for the verification target API, the verification unit 135 calls (executes) the top-level API at the hierarchical level to obtain the reference information. The API of the next hierarchy level is called using the acquired reference information. That is, the verification unit 135 calls other APIs in the descending order of the set hierarchy level, and registers in advance the upper settings required in the lower order as reference information. For example, the verification unit 135 registers (stores) reference information acquired from a higher-level API in the test case storage unit 122. Then, the verification unit 135 calls a lower-level API using the registered reference information. The verification unit 135 repeats such processing to finally acquire reference information used by the verification target API, and verifies the verification target API using the acquired reference information. As illustrated in FIG. 2, the verification unit 135 includes an execution unit 136 and a storage unit 137.

(About the execution unit 136)
The execution unit 136 executes a verification process for the verification target API. For example, the execution unit 136 performs a predetermined input to the verification target API according to the test case created by the creation unit 132. Then, the execution unit 136 obtains an output from the verification target API for the input information. In other words, the execution unit 136 executes a test defined for each test pattern on the verification target API and obtains a result.

  As described above, the execution unit 136 executes the test for the verification target API while obtaining the reference information from the upper layer API for the API for which the hierarchical relationship is set. Then, the execution unit 136 acquires a test result derived for each reference information from one test pattern. That is, according to the execution unit 136, it is possible to obtain the result of the output changing according to the hierarchical relationship, so that the user can comprehensively obtain the result regarding the behavior of the verification target API. Further, since the execution unit 136 can automatically call the upper layer API and obtain the reference information based on the hierarchical relationship set by the setting unit 134, the user can efficiently obtain a test result without omission. Can be obtained.

(About the storage unit 137)
The storage unit 137 stores the result of the verification process executed by the execution unit 136. Specifically, the storage unit 137 stores the result of each test pattern executed according to the test case in the test result storage unit 123.

[3. Example of processing)
The configuration of the verification apparatus 100 that executes the verification program has been described. Hereinafter, as an example of the verification process executed by the verification apparatus 100, a specific example in which the user uses an “advertisement API” that is an example of an API related to an advertisement is shown, and a process in which the verification of the advertisement API is performed Will be described together with the UI displayed on the user terminal 10. In the following description, it is assumed that a UI is provided from the verification apparatus 100 to the user terminal 10 and the user uses a verification program as a tool that can be executed on the user terminal 10 via the UI.

  First, with reference to FIGS. 3 to 6, an outline of an API related to an advertisement will be described as an example of a verification target API. An API related to an advertisement is an example of an API that provides a function used by a predetermined user via a web network. More specifically, an API that provides a function of accepting a setting of an advertisement distributed on the web network. is there. For example, an API related to an advertisement is a web API used when a user (for example, an advertiser) intends to distribute an advertisement, and has a predetermined hierarchical relationship.

  In the embodiment, the API related to the advertisement is an “account API” for setting an account, etc., a “campaign API” for setting an overall budget of the advertisement, and an “ad group API” for setting a detailed bid price and the like. Assume that there is a hierarchical relationship in the order of “advertisement API” for setting specific advertisement content. That is, the account API is higher than the campaign API. Further, the campaign API is higher than the advertisement group API. The advertisement group API is higher than the advertisement API.

  First, it is assumed that the user has set an account using the account API. Subsequently, the user uses a campaign API. FIG. 3 is a diagram illustrating the campaign API according to the embodiment. The campaign API is one of web APIs related to advertisements, and is provided from, for example, a predetermined execution server 20. When the user uses the campaign API, the user inputs information into items on the screen using a predetermined UI displayed on the display screen 31. Thereby, the user can set a campaign related to the advertisement. In the display screen 31, items for which “*” is displayed indicate essential items that must be input. For example, if “campaign name” is not input on the display screen 31, the campaign API returns an error. On the display screen 31, items for which “*” is not displayed indicate optional items that do not require input. In the case of an optional item, an error may not occur even if information is not entered in the item. In addition, the display screen 31 is provided with radio buttons for selecting “bid method” and “distribution method”. Depending on such selection, the reference information issued as a return value by the campaign API may change.

  Then, it is assumed that the user has set a campaign using the campaign API. Subsequently, the user uses the advertisement group API. FIG. 4 is a diagram illustrating the advertisement group API according to the embodiment. When using the advertisement group API, the user uses the predetermined UI displayed on the display screen 32 to input information in the items on the screen. Thereby, the user can create an advertisement group that is a superordinate concept of the advertisement content to be distributed. As shown in FIG. 4, the display screen 32 includes essential items, optional items, and selection items as in the display screen 31.

  Then, it is assumed that the user has set an advertisement group using the advertisement group API. Subsequently, the user uses an advertisement API. FIG. 5 is a diagram (1) illustrating the advertisement group API according to the embodiment. As shown in FIG. 5, the user can input a keyword related to the advertisement content via the UI such as the display screen 33 in the advertisement API. The keyword related to the advertisement content is used as a search query in the search process, for example.

  When the user sets a keyword, the user performs the following setting. FIG. 6 is a diagram (2) illustrating the advertisement group API according to the embodiment. As shown in FIG. 6, the user performs detailed settings related to the advertisement content via the UI such as the display screen 34 in the advertisement API. In the advertising API, the processing of the advertising API may change depending on the type of advertising content that the user wants to distribute. For example, in the advertising API, the user can select whether to create advertising content based on text or to create advertising content for an application that operates on a smartphone or the like. In this way, when there are two types of API processing, the test case described later may be affected.

  The API related to the advertisement has been described above with reference to FIGS. 3 to 6. Next, an example of processing in which the verification program executed by the verification apparatus 100 verifies the advertisement API will be described with reference to FIGS. 7 to 11.

  FIG. 7 is a diagram (1) illustrating a creation process according to the embodiment. FIG. 7 shows a flow of processing for creating a test case related to the advertisement API.

  First, the user selects a system for executing a test (step S21). Here, the system indicates a category for managing various APIs. In the embodiment, it is assumed that an API for verification is managed in the system “bltapi”.

  Subsequently, the user selects a service (API) to be verified (step S22). Here, the user selects an advertisement API (indicated as “adGroup” in FIG. 7) as an API to be verified among the four services.

  Then, the user appropriately selects a field related to the service (Field), a state related to the field (Value), and the like (steps S23 and S24). Such processing may be automatically performed when a service is selected. In other words, the verification device 100 acquires information regarding the API variables and arrangements at the time of acquiring information related to the API, and applies the information acquired at the time when the service is selected. The fields for the service are read.

  Then, the user sets an upper hierarchy regarding the service (step S25). In FIG. 7, the hierarchical relationship is indicated by an item “Chain”. That is, the user designates “acccount”, “campain”, and “adGroup” as the services of the upper hierarchy of the service “adGroup”, and sets the hierarchical relationship. Note that the notations “acccount”, “campain”, and “adGroup” in FIG. 7 correspond to the “account API”, “campaign API”, and “advertising group API” described as the API related to the above-described advertisement, respectively.

  There may be a plurality of methods that can be tested for a service to be verified (step S26). For example, in the service “adGrouppad”, “add”, “set”, “remove”, and “get” can be selected as methods. Such a method is selected by the user when the test is executed. Then, the user sets an API reference of the service “adGroup” (step S27). For such API reference, the verification apparatus 100 can acquire information set for each API in advance.

  Then, the user selects a test pattern to be executed in the verification process (step S28). As an example, the user selects input items such as “required item × default value” and “required item × minimum value” as test patterns for the service “adGroup” (in FIG. 7, “required × default”, “required” respectively). Xmin ”).

  Then, based on the information selected in the items of steps S21 to S28, the creation unit 132 creates a test case (step S29). As described above, the advertisement API can create two types of advertisements, a text advertisement and an application advertisement, and the process is performed by different processes. Therefore, the creation unit 132 automatically creates test cases corresponding to two types of processing, that is, text advertisement and application advertisement.

  Here, the process set by the user as described above will be described with reference to a diagram simulating a UI. FIG. 8 is a diagram (2) illustrating the creation process according to the embodiment. FIG. 8 illustrates an example in which the user specifies a hierarchical relationship for the service “adGrouppad” via the UI. For example, the UI is displayed on the display unit 41 included in the user terminal 10.

  The user can display the table 410 used for setting the upper layer by selecting the item “upper layer setting” of the service “adGrouppad”. As shown in Table 410, “Account” is set as “hierarchy level Level-1” for the service “adGrouppad”. Note that the item “hierarchy level” indicates a higher level as the number is smaller. Similarly, “campain” is set as “hierarchy level Level-2” and “adGroup” is set as “hierarchy level Level-3” for service “adGrouppad”. In this way, the user specifies a hierarchical relationship for the service “adGrouppad” via the UI. The accepting unit 133 accepts such designation, and the setting unit 134 sets a hierarchical relationship based on such designation.

  Next, an example of a test pattern will be described with reference to a diagram simulating a UI. FIG. 9 is a diagram (3) illustrating the creation process according to the embodiment. FIG. 9 shows an example in which the user designates a test pattern via the UI.

  FIG. 7 shows an example in which the user selects two test patterns of “essential item × default value” and “essential item × minimum value”. Actually, as shown in FIG. 9, the user can arbitrarily select a test pattern from a large number of prepared test patterns.

  Further, as shown in FIG. 9, the test pattern includes an item “expected result”. This is an item for selecting whether the API should return “success” or “failure” in response to input of information as a test pattern. For example, in the case of “essential item × default value”, it is desirable that the API returns “success” because a value that can be processed is input to the mandatory item. On the other hand, in the case of “required item × no input”, the API preferably returns “failure (error)” because a value is not input in the required item. As described above, the verification apparatus 100 stores the test pattern and the expected result that are assumed in advance. The user can efficiently perform a desired test by selecting such a test pattern. Note that the user may arbitrarily create a test pattern. In this case, the reception unit 133 receives the creation of the test pattern. The setting unit 134 sets a newly created test pattern.

  As described above, a test case is created by the creation unit 132 when the user selects an API to be tested and selects information related to the test.

  Next, a process for verifying a verification target API using a test case created by the creation unit 132 will be described with reference to FIGS. 10 to 12. First, a test environment designated by the user when performing a test will be described with reference to FIG.

  FIG. 10 is a diagram (1) illustrating the verification process according to the embodiment. In FIG. 10, items selected by the user when executing a test case will be described.

  First, the user selects a test environment (indicated as “Environment” in FIG. 10) (step S31). The test environment is, for example, a test environment setting prepared for each user who uses the verification program. Subsequently, the user selects a system for executing the test (step S32). The system corresponds to the items shown in FIG.

  Then, the user selects a service (API) for executing the test (step S33). Here, it is assumed that the user selects “adGroup” as the service. Subsequently, the user selects a method (indicated as “Execute” in FIG. 10) to be executed using the test case. Here, it is assumed that the user selects “add” as the method to be executed. Based on the user's selection, the execution unit 136 executes a verification process for the verification target API.

  Specifically, the execution unit 136 executes the specified method on the advertisement API that is the API to be verified for the two test cases created in the creation process, and whether the process was successful or failed. Get the result of what was. Here, when the reference information is used for the process to be executed, the execution unit 136 obtains the reference information by appropriately calling an upper layer API. The execution unit 136 prepares predetermined reference information in advance for reference information for the highest-level API. For example, the execution unit 136 executes the process by fitting predetermined reference information for testing to the account API that is the highest level with respect to the advertisement API.

  Then, the storage unit 137 stores the results executed by the execution unit 136 in the test result storage unit 123. The result is notified to the user via the UI provided by the UI providing unit 131. This point will be described with reference to FIG.

  FIG. 11 is a diagram (2) illustrating the verification process according to the embodiment. FIG. 11 illustrates an example in which an execution result of executing “add” of the advertising API is displayed via the UI. As shown in FIG. 11, the execution result performed by the execution unit 136 is displayed for each test pattern via the UI.

  Further, the execution unit 136 executes each reference information issued by the upper layer API with respect to one test pattern. This point will be described with reference to FIG. FIG. 12 is a diagram (3) illustrating the verification process according to the embodiment. FIG. 12 shows a state in which the details of the test pattern “REQUIRED PATTERN” are displayed via the UI.

  Table 411 in FIG. 12 shows how many test cases are actually verified when one test pattern is executed. In the above description, it has been described that the advertisement API verifies two types of test cases, that is, an application advertisement and a text advertisement. Since an account API, a campaign API, and an advertisement group API, which are upper-layer APIs, are set in the advertisement API, verification is performed for each reference information issued by each API.

  For example, in the example of Table 411 in FIG. 12, the test pattern “REQUIRED PATTERN” is executed in the test case “(account) required × default (campaign) AAA × aaa (adGroup) XXX >>> APP AD”. Is shown. For example, the notation “(campaign) AAA × aaa” indicates information that changes depending on the reference information issued by the campaign API. “AAA”, “aaa”, and the like conceptually indicate that reference information issued by the campaign API changes depending on a value input to the campaign API. As an example, in the example shown in FIG. 3, “AAA” due to a difference in the bidding method selected in the campaign API setting (a difference in selecting “manual setting” or “automatic bidding setting”) or the like. And items corresponding to “aaa” change.

  In the example of FIG. 12, it is shown that the campaign API may issue four types of different reference information, “AAA × aaa”, “AAA × bbb”, “BBB × aaa”, and “BBB × bbb”. Yes. Further, the advertisement group API indicates that two types of reference information “XXX” and “YYY” may be issued. Further, as described above, the advertisement API indicates that two types of test cases of “APP AD (application advertisement)” and “TEXT AD (text advertisement)” can be taken. As described above, the execution unit 136 can comprehensively verify the verification target API by executing the lower API while calling the upper API and obtaining the reference information even in one test case. it can.

  Note that the number of reference information that can be taken by the lower API may change depending on the reference information issued by the upper API. In the example of FIG. 12, for example, when the campaign API selects “AAA”, the advertisement group API may take “XXX” but may not take “YYY”. Such a situation is due to the relationship between APIs, and the execution unit 136 uses, for example, a test case created by selection that the APIs can derive by deriving such a relationship in the process of executing an upper layer API. And verify. Therefore, also in the example shown in FIG. 12, depending on the test pattern, eight or more test cases may be displayed in the detailed information, or fewer test cases may be displayed. In addition, although illustration in FIG. 12 is omitted, each processing result includes history information such as the time when the verification was performed. The user refers to such history information, and uses a verification program as a management tool to grasp information such as whether the API on the network is operating properly and when the API link breakage occurred. Can be used.

[4. (Verification procedure)
Next, a processing procedure performed by the verification apparatus 100 according to the embodiment will be described with reference to FIGS. 13 and 14. FIG. 13 is a flowchart illustrating a creation processing procedure performed by the verification apparatus 100 according to the embodiment.

  As illustrated in FIG. 13, the reception unit 133 receives an API setting from the user terminal 10 (step S101). Here, the API setting refers to, for example, API hierarchical relationship setting. In this case, the setting unit 134 sets the content received by the receiving unit 133. Furthermore, the reception unit 133 receives selection of a test pattern as content to be verified by the verification target API (step S102).

  Then, the creation unit 132 creates a test case for the verification target API based on the content received by the reception unit 133 (step S103). The creation unit 132 stores the created test case in the test case storage unit 122, and ends the creation process.

  Next, a verification processing procedure performed by the verification apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 14 is a flowchart illustrating a verification processing procedure performed by the verification apparatus 100 according to the embodiment.

  First, the verification unit 135 receives a selection of a test environment from the user (step S201). The selection of the test environment means specifying a system or the like to perform a test as shown in FIG. 10 or selecting a process to be executed by the API.

  Then, the execution unit 136 executes a test for the verification target API (step S202). Here, the execution unit 136 determines whether or not a hierarchical relationship is set for the verification target API being executed (step S203). Specifically, the execution unit 136 determines whether or not there is a blank area in the script to be executed in which the reference information is to be entered, and a return value from the upper layer API that can fill the blank (reference) Information).

  When the hierarchical relationship is set (step S203; Yes), the execution unit 136 calls the upper layer API and executes the upper layer API (step S204). Then, the execution unit 136 acquires a return value issued from the upper layer API as reference information (step S205).

  And the execution part 136 performs a test based on the acquired reference information (step S206). For example, the execution unit 136 substitutes reference information (for example, information issued as an ID such as a numerical value of several digits) in a script to be executed, and verifies the behavior of the verification target API.

  Then, the execution unit 136 acquires a test result (step S207). When the hierarchical relationship is not set in the verification target API (step S203; No), the execution unit 136 executes the test case related to the verification target API as it is and acquires the test result (step S207). Then, the storage unit 137 stores the test result in the test result storage unit 123, and ends the verification process.

[5. (Modification)
The verification program executed by the verification apparatus 100 according to the above-described embodiment may be executed in various different forms other than the above-described embodiment. Therefore, in the following, other embodiments of the verification apparatus 100 and the verification program will be described.

[5-1. API]
In the above-described embodiment, an example in which the verification process is executed for the API related to the advertisement has been described. However, the verification apparatus 100 can perform the above process without being limited to the API related to the advertisement. For example, according to the verification program related to the verification apparatus 100, the above verification process can be applied to any API that has a hierarchical relationship and is an API designed in a REST (Representational State Transfer) format. .

[5-2. Automatic setting)
In the above-described embodiment, the verification apparatus 100 has shown an example in which settings of variables and arrays such as fields related to APIs are manually received from the user. Here, the verification apparatus 100 may automatically perform the above setting by performing specific conversion on the API.

  For example, the verification apparatus 100 automatically sets an API by including a converter that performs conversion between predetermined languages. For example, an API creator creates an API created in a specific programming language (for example, JAVA or the like) and a predetermined IDL (Interface Description Language) file. The IDL file is a file that predefines fields and the like based on the described source code base. Then, the verification apparatus 100 uses the IDL converter to create a JAVA source API in which fields and the like are defined. As a result, it is possible to save the user (or a provider who sets API in advance in the verification program) from manually setting fields and the like.

[5-3. Execution process)
In the above embodiment, an example in which “add” is selected in the process executed by the execution unit 136 has been described. Here, when an instruction such as “set” is selected from the user, the execution unit 136 may execute the verification target API in advance and perform verification after obtaining a predetermined return value. This is because “set” is a process of replacing a predetermined value, and when there is no value to be replaced, the verification process cannot be performed. For this reason, the execution unit 136 can acquire the return value that is the target of “set” by operating the verification target API in advance, and can execute the verification process using the return value.

[5-4. (Superordinate relationship)
In the API, the number of processes that can be taken by the lower-level API may be determined depending on the upper level relationship. A specific description will be given using an API related to an advertisement in the embodiment. For example, in the campaign API, the number of campaigns that can be created by an API administrator or the like may be determined. For example, this is set in advance in the API so as not to hinder the actual operation of the API. For example, in the campaign API, it is assumed that an upper limit of “100” is set for the number of campaigns that can be created. In this case, when the test case to be executed exceeds “100”, an error occurs in the verification process. This is because, when the API is executed in the verification process, the campaign creation process is actually performed, and therefore the process exceeding the set upper limit value cannot be performed. However, since test cases increase due to a combination of reference information issued by upper-level APIs, it is difficult for a user to determine whether or not the above upper limit condition is satisfied when executing a test. .

  In this case, the execution unit 136 performs predetermined avoidance means and adjusts so that no error occurs in the verification process. As an example, the execution unit 136 causes the reference information issued by the account API above the campaign API to be issued so that the number of test cases can be satisfied. For example, when two types of reference information are issued with the account API, the execution unit 136 can perform processing for the campaign API, so the number of campaigns created with the campaign API is set to “200”. Can do. As described above, the execution unit 136 can perform the test without generating an error even if the number of test cases to be created increases by adjusting the reference information issued by the upper layer API. .

[6. Others]
In addition, among the processes described in the above embodiment, all or part of the processes described as being automatically performed can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the document and drawings can be arbitrarily changed unless otherwise specified.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  For example, the storage unit 120 illustrated in FIG. 2 may be stored in a predetermined storage server or the like without the verification device 100. In this case, the verification apparatus 100 acquires various pieces of stored information by accessing the storage server.

  Further, for example, the verification device 100 described above is distributed between a front-end server side that mainly performs transmission / reception of information with the user terminal 10 via a UI and a back-end server side that actually executes an API test. Also good. Moreover, the verification apparatus 100 and the execution server 20 may be comprised integrally. In this case, the verification apparatus 100 holds a predetermined API and causes the execution unit 136 to execute the API.

[7. Hardware configuration)
Further, the verification apparatus 100 according to the above-described embodiment is realized by a computer 1000 having a configuration as shown in FIG. 15, for example. FIG. 15 is a hardware configuration diagram illustrating an example of a computer 1000 that implements the functions of the verification apparatus 100. The computer 1000 includes a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I / F) 1500, input / output interface (I / F) 1600, and media interface (I / F) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each unit. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

  The HDD 1400 stores a program executed by the CPU 1100, data used by the program, and the like. The communication interface 1500 receives data from other devices via the communication network 500 (corresponding to the predetermined communication network in the embodiment) and sends the data to the CPU 1100, and the data generated by the CPU 1100 via the communication network 500. Send to other devices.

  The CPU 1100 controls an output device such as a display and a printer and an input device such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. Further, the CPU 1100 outputs the data generated via the input / output interface 1600 to the output device.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads such a program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. Etc.

  For example, when the computer 1000 functions as the verification apparatus 100, the CPU 1100 of the computer 1000 implements the function of the control unit 130 by executing a program loaded on the RAM 1200. In addition, each data in the storage unit 120 is stored in the HDD 1400. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them, but as another example, these programs may be acquired from other devices via the communication network 500.

[8. effect〕
As described above, the verification apparatus 100 according to the embodiment includes the creation unit 132, the setting unit 134, and the verification unit 135. The creation unit 132 creates a test case to be executed for the verification target API that is the verification target. The setting unit 134 sets a hierarchical relationship between the verification target API and another API having a relationship in which reference information, which is information used when the verification target API is executed, is returned as a return value. Based on the hierarchical relationship set by the setting unit 134, the verification unit 135 acquires reference information by calling other APIs in a higher level, and uses the acquired reference information and test cases to determine the verification target API. Validate. Note that the processing by the verification apparatus 100 is performed by causing the verification program according to the present application to execute each process related to the control unit 130 of the verification apparatus 100.

  As described above, the verification apparatus 100 according to the embodiment can efficiently perform verification processing of APIs having mutual relationships such as a hierarchical relationship. That is, the user who performs the test can execute the test for the verification target API by setting the hierarchical relationship of the verification target API without being aware of the combination of the return values that can be taken by the higher-level API. . As described above, the verification apparatus 100 can efficiently perform exhaustive verification on an API having a hierarchical relationship.

  In addition, the setting unit 134 sets, for each API, a hierarchy level that is information indicating the order of the hierarchical relationship with the verification target API. Then, the verification unit 135 calls other APIs in the order of higher hierarchy levels set by the setting unit 134, acquires reference information used by APIs corresponding to lower hierarchy levels, and acquires the acquired reference information. To verify the API to be verified.

  As described above, the verification apparatus 100 according to the embodiment can verify the verification target API by receiving the definition of the relationship even when the API has a multi-layer relationship. Thereby, according to the verification apparatus 100, since the test result derived | led-out for every reference information from one test pattern can be acquired comprehensively, the efficient process without a leak can be performed.

  The verification apparatus 100 according to the embodiment further includes a UI providing unit 131 (an example of a providing unit) that provides a UI for displaying a result verified by the verification unit 135. The UI providing unit 131 relates to another API used for verification of the verification target API (for example, an API in which a hierarchical relationship with the verification target API is set) together with the verification result of the verification target API via the UI. Provide information.

  As described above, the verification apparatus 100 according to the embodiment provides a UI that allows the user to browse the verification result. Thereby, according to the verification apparatus 100, even if it is API which requires the complicated verification which has a mutually related relationship, management can be performed based on visual information, Therefore The burden concerning a user's management Can be reduced. Further, according to the verification apparatus 100, information on other APIs can be provided as a verification result, so that it is possible to easily grasp the relevance between APIs that are difficult to manage manually.

  The verification target API may be an API that provides a function used by a predetermined user via a web network.

  According to the verification device 100, as described above, it is possible to perform comprehensive verification on the web API having a hierarchical relationship. Thereby, for example, the verification apparatus 100 can allow the user to easily grasp incomplete operation of the server holding the API, a broken link, etc., and improve the convenience for the API used via the web network. be able to.

  Further, the verification target API may be an API that provides a function of accepting a setting of an advertisement distributed on a web network.

  According to the verification apparatus 100, for example, coverage of APIs for setting types of advertisements that an advertiser intends to distribute (selection of automatic setting of bid price, manual setting, whether or not a text advertisement, etc.) Therefore, it is possible to efficiently manage APIs for advertisers and advertisement distribution companies.

  As described above, some of the embodiments of the present application have been described in detail with reference to the drawings. However, these are merely examples, and various modifications, including the aspects described in the disclosure section of the invention, based on the knowledge of those skilled in the art, It is possible to implement the present invention in other forms with improvements.

  The verification apparatus 100 described above may be realized by a plurality of server computers, and depending on the function, the configuration may be realized by calling an external platform or the like with an API (Application Programming Interface) or network computing. Can be changed flexibly.

  Further, “section (module, unit)” described in the claims can be read as “means”, “circuit”, and the like. For example, the verification unit can be read as verification means or a verification circuit.

DESCRIPTION OF SYMBOLS 1 Verification system 10 User terminal 20 Execution server 100 Verification apparatus 110 Communication part 120 Storage part 121 API information storage part 122 Test case storage part 123 Test result storage part 130 Control part 131 UI provision part 132 Creation part 133 Reception part 134 Setting part 135 Verification unit 136 Execution unit 137 Storage unit

Claims (7)

A creation procedure for creating a test case to be executed for a verification target API (Application Programming Interface),
A setting procedure for setting a hierarchical relationship between the verification target API and another API having a relationship of returning reference information, which is information used when the verification target API is executed, as a return value;
Based on the hierarchical relationship set by the setting procedure, the reference information is acquired by calling the other API in the higher hierarchy, and the verification target API is verified using the acquired reference information and the test case. Verification procedure to
The verification program characterized by making a computer run. The setting procedure is as follows:
A hierarchy level, which is information indicating the order of the hierarchy relationship for the verification target API, is set for each of the other APIs,
The verification procedure includes:
The other APIs are called in the order of the hierarchical level set by the setting procedure, the reference information used by the API corresponding to the lower hierarchical level is acquired, and the verification target API is used using the acquired reference information Verify
The verification program according to claim 1, wherein: A providing procedure for providing a UI (User Interface) for displaying a result verified by the verification procedure;
Further including
The providing procedure includes:
Via the UI, along with the verification result of the verification target API, providing information on the other API used for verification of the verification target API;
The verification program according to claim 1 or 2, characterized in that: The verification target API is an API that provides a function used by a predetermined user via a web network.
The verification program according to any one of claims 1 to 3. The verification target API is an API that provides a function of accepting a setting of an advertisement distributed on a web network.
The verification program according to any one of claims 1 to 4, wherein: A creation unit for creating a test case to be executed for a verification target API that is a verification target;
A setting unit configured to set a hierarchical relationship between the verification target API and another API having a relationship of returning reference information, which is information used when the verification target API is executed, as a return value;
Based on the hierarchical relationship set by the setting unit, the reference information is acquired by calling the other API in the higher hierarchy, and the verification target API is verified using the acquired reference information and the test case. A verification unit to
A verification apparatus comprising: A verification method performed by a computer,
A creation step for creating a test case to be executed for the verification target API that is the verification target;
A setting step for setting a hierarchical relationship between the verification target API and another API having a relationship of returning reference information, which is information used when the verification target API is executed, as a return value;
Based on the hierarchical relationship set by the setting step, the reference information is acquired by calling the other API in the higher hierarchy, and the verification target API is verified using the acquired reference information and the test case. A verification process to
The verification method characterized by including.

Images