JP2011250104A - Sequence test device and sequence test method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sequence test device and a sequence test method capable of easily specifying log information beyond the range of expectations.SOLUTION: A sequence test device 40 comprises: a reference scenario registration unit 41 for registering a reference scenario; a threshold setting unit 43 for setting a threshold of response time for each sequence defined in the reference scenario; a log information acquisition unit 45 for acquiring log information between a node A10, a node B20 and a node C30; a comparison unit 47 for comparing the response time of a predetermined sequence included in the log information with the range of expectations, which is a time range obtained by adding the response time included in the reference scenario and the threshold of the response time; and a display unit 48 for, by inputting comparison result data from the comparison unit 47, displaying in a distinguishable manner the log information related to a response message sent within the range of expectations and the log information related to a response message sent beyond the range of expectations.

Description

  The present invention relates to a sequence test apparatus and a sequence test method. For example, the present invention relates to a sequence test apparatus and a sequence test method for testing communication terminals such as mobile phones and mobile terminals in a sequence based on a predetermined communication protocol.

  When a communication terminal such as a mobile phone is newly developed, it is necessary to test whether or not the developed communication terminal operates normally in an environment where it is actually used. However, whether or not the newly developed communication terminal operates normally by transmitting / receiving various types of test communication information between the newly developed communication terminal and the actual base station already in operation. It is very difficult to test.

  Therefore, a newly developed communication terminal is connected to a pseudo base station device having the function of an actual base station, and various communication information is transmitted and received between the communication terminal and the pseudo base station device. There is known a test apparatus that performs a test of whether or not the developed communication terminal operates normally. As one of the functions in this type of test apparatus, log information of various communication information transmitted / received between a communication terminal as a terminal under test and a pseudo base station apparatus is acquired and displayed as trace information on a display device. There is a trace function. By using this trace function to display and output a plurality of detailed time-series log information of communication information as trace information, it is possible to investigate the cause of abnormalities when the terminal under test does not operate normally. It becomes easy.

  In general, since test items for the terminal under test are diverse, there is a huge amount of communication information transmitted and received for testing between the terminal under test and the pseudo base station apparatus, and the log information of this communication information is also huge. Number. However, since the conventional test apparatus only outputs a large amount of log information of communication information transmitted and received for testing in time series, a log that is regarded as abnormal when the terminal under test does not operate normally There was a problem that information could not be easily identified.

  As a solution to this problem, for example, a pseudo base station apparatus disclosed in Patent Document 1 is known. The pseudo base station apparatus includes: extraction information specifying means for specifying extraction information included in at least a part of log information; log information extracting means for extracting log information having the specified extraction information from trace information; And a filter marker display means for adding a filter marker to the log information extracted by the log information extraction means among the plurality of log information constituting the trace information displayed in FIG. With this configuration, the conventional pseudo base station apparatus can easily identify abnormal log information from among a plurality of log information constituting the trace information displayed on the display.

JP 2006-174173 A

  However, although disclosed in Patent Document 1 can easily identify abnormal log information, it cannot be said to be abnormal log information, but a response time range for a certain message is a predetermined expected value range. Log information indicating that it is outside (hereinafter referred to as “log information outside the expected value range”) could not be easily specified, and its improvement was demanded.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to provide a sequence test apparatus and a sequence test method that can easily specify log information outside the expected value range.

  A sequence test apparatus according to claim 1 of the present invention is a sequence test apparatus (40) for performing tests on communication means (20, 30) for executing a sequence for transmitting and receiving a message of a predetermined communication protocol. A reception message received by the communication unit, a response message transmitted in response to the reception message, and information on an ideal value of a response time from when the communication unit receives the reception message to when the response message is transmitted Based on the reference scenario registered by the reference scenario registration means (41), threshold setting means (43) for setting a threshold of the response time, and reference scenario registration means Log information including the response time when causing the communication means to execute a sequence for transmitting and receiving a message A log information acquisition unit (45) to be acquired, and a set response indicating a time range obtained by adding the response time threshold set by the threshold setting unit to the response time registered by the reference scenario registration unit Based on the comparison result of the response time comparison means (47) for comparing the time range with the response time included in the log information acquired by the log information acquisition means, and the response time comparison means, the set response time A display means (48) for identifying and displaying log information relating to a response message transmitted within the range and log information relating to a response message transmitted outside the set response time range; Yes.

  With this configuration, the sequence test apparatus according to claim 1 of the present invention, based on the comparison result of the response time comparison means, the log information related to the response message transmitted within the set response time range, and outside the set response time range. The log information related to the transmitted response message is identified and displayed. Therefore, the sequence test apparatus according to claim 1 of the present invention can easily specify log information outside the expected value range by setting the threshold value of the response time as the expected value in advance.

  A sequence test method according to a second aspect of the present invention is a sequence test method for performing a test on a communication unit that executes a sequence for transmitting and receiving a message of a predetermined communication protocol, the reception being received by the communication unit. A reference for registering a message and a response message to be transmitted in response to the received message and information on an ideal value of a response time from when the communication means receives the received message until the response message is transmitted as a reference scenario A scenario registration step; a threshold setting step for setting a threshold value for the response time; and a sequence for transmitting and receiving a message based on the reference scenario registered in the reference scenario registration step. Log information acquisition step for acquiring log information including the response time at the time of And a set response time range indicating a time range obtained by adding the response time threshold value set in the threshold value setting step to the response time registered in the reference scenario registration step, and the log information acquisition Log information relating to response messages transmitted within the set response time range based on a comparison result in the response time comparison step comparing the response time included in the log information acquired in the step And a display step of identifying and displaying log information related to a response message transmitted outside the set response time range.

  With this configuration, the sequence test method according to claim 2 of the present invention is based on the comparison result in the response time comparison step, and the log information regarding the response message transmitted within the set response time range, and outside the set response time range. The log information related to the transmitted response message is identified and displayed. Therefore, the sequence test method according to claim 2 of the present invention can easily specify log information outside the expected value range by setting the threshold value of the response time as the expected value in advance.

  The present invention can provide a sequence test apparatus and a sequence test method having an effect that log information outside an expected value range can be easily specified.

It is a block block diagram of the sequence test system in the embodiment of the present invention. It is a figure which shows the message and expectation value example of the reference | standard scenario in embodiment of this invention. It is a figure which shows the sequence of the reference | standard scenario in embodiment of this invention. It is a figure which shows the 1st operation result example based on the reference | standard scenario in embodiment of this invention. It is a figure which shows the 2nd operation result example based on the reference | standard scenario in embodiment of this invention. It is a figure which shows the example of a display of the 2nd operation result example based on the reference | standard scenario in embodiment of this invention. It is a figure which shows the 3rd example of operation result based on the reference | standard scenario in embodiment of this invention. It is a block block diagram of the sequence test system in the other aspect of embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. An example in which the sequence test apparatus of the present invention is applied to a sequence test system including a plurality of nodes will be described. For example, when testing a communication terminal such as a mobile phone or a mobile terminal in a sequence based on a predetermined communication protocol, this sequence test system performs a call connection test such as location registration, transmission, incoming call, and handover of the communication terminal. It is something that can be done. In the present embodiment, the sequence test system will be described as performing a call connection test on a plurality of nodes connected to a wired communication network in a sequence based on a communication protocol.

  First, the configuration of the sequence test system in the present embodiment will be described.

  As shown in FIG. 1, the sequence test system 100 in this embodiment includes a node A10, a node B20, and a node C30. Although not shown, the node A10, the node B20, and the node C30 include a CPU that controls the operation of each node and a control circuit that includes a ROM, a RAM, and the like that store a program for causing the CPU to function. I have.

  A communication cable 11 is connected between the node A10 and the node B20. Between the node B20 and the node C30, a tap 60 for taking out a communication signal between the two is provided. A communication cable 12 is connected between the node B20 and the tap 60, and a communication cable 13 is connected between the tap 60 and the node C30. Here, the node B20 and the node C30 constitute a communication unit according to the present invention.

  The node A10 includes a sequence test device 40 and a communication device 50. The sequence test apparatus 40 includes a reference scenario registration unit 41, a reference scenario storage unit 42, a threshold setting unit 43, a threshold storage unit 44, a log information acquisition unit 45, a log information storage unit 46, a comparison unit 47, and a display unit. 48 is provided. The sequence test apparatus 40 performs a sequence test based on a predetermined communication protocol for the node B 20 and the node C 30.

  The reference scenario registration unit 41 is information such as a communication test procedure, a communication condition, a response time from when a certain node receives a received message in a predetermined sequence until a response message is transmitted, and serves as a reference for the sequence test. (Hereinafter referred to as “reference scenario”) is operated by the user for registration. The reference scenario registration unit 41 includes, for example, a display that displays a registration screen for creating and registering a reference scenario, an input device such as a keyboard, a dial, or a mouse, a control circuit that controls these devices, and the like. . The reference scenario registration unit 41 constitutes reference scenario registration means according to the present invention.

  The reference scenario storage unit 42 is composed of, for example, a hard disk drive, and stores a reference scenario registered by the user.

  The threshold value setting unit 43 sets a threshold value for determining whether or not the actual response time is within the expected value range with respect to the response time for each sequence determined in the reference scenario. The user operates. The threshold setting unit 43 includes, for example, a display that displays a setting screen for setting a threshold, an input device such as a keyboard, a dial, or a mouse, a control circuit that controls these devices, and the like. . The expected value range is determined by the response time registered by the reference scenario registration unit 41 and the threshold value set by the threshold value setting unit 43. Data for determining the range of the expected value is prepared in advance based on the specification contents of the nodes B20 and C30 to be tested, the results of simulation, and the like. The threshold setting unit 43 constitutes threshold setting means according to the present invention.

  The threshold value storage unit 44 is constituted by a hard disk drive, for example, and stores threshold value data set by the threshold value setting unit 43.

  Here, in this embodiment, the reference scenario registered by the reference scenario registration unit 41 and the threshold value set by the threshold value setting unit 43 will be specifically described with reference to FIGS. FIG. 2A is an example of a reference scenario. The scenario describes a test procedure and a communication sequence for the test, and corresponds to the portion (a) in FIG. The reference scenario is obtained by adding an elapsed time or a response time to the scenario as a reference time which is time information serving as a reference, and a portion (b) in FIG. 2A is the elapsed time or the response time. This elapsed time or response time is an ideal value when the sequence test system 100 is ideally operated, or a value expected by the reference scenario creator for the sequence test system 100. As for the elapsed time, the time at which each message should be transmitted is described as the elapsed time for each message on the basis of the transmission time of the first message. Regarding the response time, the time from when one message is transmitted until the next message is transmitted is described as the response time for each message. The elapsed time or the response time is described in the reference scenario. FIG. 2B is a conceptual diagram combining the reference scenario and the above-described threshold value, and the expected value derived from the elapsed time or response time of the reference scenario and the threshold value is shown for each message. ing.

  As shown in FIGS. 2 and 3, in order to explain the invention in an easy-to-understand manner, in this embodiment, the number of messages (hereinafter referred to as “MSG”) transmitted and received between nodes is four. Specifically, MSG1 transmitted from the node A10 to the node B20 is “Update Location Request”, and MSG2 transmitted from the node B20 to the node C30 in response to this MSG1 is “Authentication Information”. In addition, MSG3 transmitted from the node C30 to the node B20 in response to the MSG2 is “Authentication Information Acknowledge”, and MSG4 transmitted from the node B20 to the node A10 in response to the MSG3 is “Authentication Information Request”. In FIG. 3, “Req” represents “Request”, “Info” represents “Information”, and “Ack” represents “Acknowledge”.

  FIG. 2B illustrates two expected values. Example 1 of the expected value is expressed using the time when the node A10 transmits MSG1 to the node B20 as “0” as a reference value, and the time when the node B20 and the node C30 transmit a message and a threshold value. . Specifically, the time when the node B20 transmits MSG2 to the node C30 is 0.100 seconds ± 0.05 seconds, the time when the node C30 transmits MSG3 to the node B20 is 0.400 seconds ± 0.05 seconds, The time when B20 transmits MSG4 to node A10 is 0.500 seconds ± 0.05 seconds. Note that the threshold values 5 to 7 shown in FIG. 3 are expressed using Example 1 of this expected value.

  On the other hand, the expected value example 2 is expressed by using a time required for each node and a threshold value. Specifically, for example, the time from when the Node B 20 receives MSG 1 until it transmits MSG 2 to the node C 30 is set to 0.100 seconds ± 0.05 seconds.

  In addition, in order to simplify the description, it is assumed that the transmission time is the same as the reception time in the expected value example 1 and example 2 shown in FIG. Here, the direct operation based on this reference scenario is only that the communication device 50 transmits MSG1. As described above, the node B20 and the node C30, which are test targets, operate according to the communication protocol. Therefore, MSG2 to MSG4 described in the reference scenario are described as messages that should be transmitted according to the communication protocol.

  Returning to FIG. 1, the log information acquisition unit 45 acquires log information between the node A 10 and the node B 20 via the communication device 50. This log information includes time stamp information created by the communication device 50 or the Node B 20 at the time of transmission. In addition, the log information acquisition unit 45 acquires log information between the node B 20 and the node C 30 via the tap 60. This log information includes time stamp information created at the time of transmission by the node B20 or the node C30. The log information acquisition unit 45 constitutes log information acquisition means according to the present invention.

  The log information storage unit 46 is constituted by, for example, a hard disk drive, and stores the log information acquired by the log information acquisition unit 45.

  The comparison unit 47 reads the reference scenario from the reference scenario storage unit 42 and the threshold value information of the response time from the threshold storage unit 44. The comparison unit 47 reads log information from the log information storage unit 46. Then, the comparison unit 47 sets the time range obtained by adding the response time threshold to the response time included in the reference scenario as the expected value range, and the response time of the predetermined sequence included in the log information and the expected value range Are to be compared. Further, the comparison unit 47 compares the message set in the reference scenario with the message included in the log information stored in the log information storage unit 46. The comparison unit 47 outputs the comparison result data to the display unit 48. The comparison unit 47 constitutes response time comparison means according to the present invention.

  The display unit 48 includes a display processing circuit and a display, and inputs and displays the log information stored in the log information storage unit 46 via the comparison unit 47. Further, the display unit 48 receives the comparison result data from the comparison unit 47, and identifies and displays log information that is within the expected value range and log information that is not within the expected value range. It has become. The display unit 48 constitutes display means according to the present invention.

  The communication device 50 communicates with the Node B 20 based on the reference scenario stored in the reference scenario storage unit 42. The communication device 50 acquires log information between the node A 10 and the node B 20 and outputs the log information to the log information acquisition unit 45.

  Next, the operation of the sequence test system 100 in the present embodiment will be described.

  When the user operates the reference scenario registration unit 41, the reference scenario is registered (reference scenario registration step). The reference scenario registered by the reference scenario registration unit 41 is stored by the reference scenario storage unit 42.

  In addition, the user operates the threshold setting unit 43 to determine whether or not the actual response time is within the expected value range with respect to the response time for each sequence determined in the reference scenario. Threshold is set (threshold setting step). The threshold value set by the threshold value setting unit 43 is stored in the threshold value storage unit 44.

  The node A10, the node B20, and the node C30 provided with the communication device 50 each transmit / receive a message based on the reference scenario stored in the reference scenario storage unit 42. During transmission / reception of this message, the communication device 50 acquires log information between the node A 10 and the node B 20 and outputs the log information to the log information acquisition unit 45, and the tap 60 displays log information between the node B 20 and the node C 30. It acquires and outputs to the log information acquisition part 45 (log information acquisition step).

  The log information storage unit 46 stores the log information acquired by the log information acquisition unit 45.

  The comparison unit 47 reads the reference scenario from the reference scenario storage unit 42 and the threshold value information of the response time from the threshold storage unit 44. The comparison unit 47 reads log information from the log information storage unit 46. Then, the comparison unit 47 sets the time range obtained by adding the response time threshold to the response time included in the reference scenario as the expected value range, and the response time of the predetermined sequence included in the log information and the expected value range Are compared (response time comparison step). Further, the comparison unit 47 compares the message set in the reference scenario with the message included in the log information stored in the log information storage unit 46. Then, the comparison unit 47 outputs the comparison result data to the display unit 48.

  The display unit 48 inputs and displays the log information stored in the log information storage unit 46 via the comparison unit 47. Further, the display unit 48 receives comparison result data from the comparison unit 47, and identifies and displays log information that is within the expected value range and log information that is not within the expected value range (display). Step).

Hereinafter, it is assumed that the reference scenario shown in FIGS. 2 and 3 is registered by the reference scenario registration unit 41, and three operation results of the node A10, the node B20, and the node C30 are illustrated and described with reference to FIGS. To do.
(First operation result example)
A first operation result example is shown in FIG. As illustrated in FIG. 4, the node A10 transmits MSG1 “Update Location Request” to the node B20 at the transmission time “0.000”.

  Subsequently, the node B20 transmits MSG2 "Authentication Information" to the node C30 in response to the received MSG1. The transmission time when this MSG2 is transmitted is “0.105”, which is within the range of the expected value “0.100 ± 0.05” of the transmission time shown in FIGS.

  Subsequently, in response to the received MSG2, the node C30 transmits MSG3 “Authentication Information Acknowledge” to the node B20. The transmission time when this MSG3 is transmitted is “0.440”, which is within the range of the expected value “0.400 ± 0.05” of the transmission time shown in FIG. 2B and FIG.

  Subsequently, the node B20 transmits MSG4 “Authentication Information Request” to the node A10 in response to the received MSG3. The transmission time when this MSG4 is transmitted is “0.460”, which is within the range of the expected value “0.500 ± 0.05” of the transmission time shown in FIG. 2B and FIG.

As described above, in the first operation result example, since the response time of each node is within the range of the expected value, the display unit 48 displays the log information regarding MSG1 to MSG4 on the display without particular identification. .
(Second operation result example)
A second operation result example is shown in FIG. As shown in FIG. 5, in this example, the transmission time of the MSG 4 is different from that in the first operation result example (see FIG. 4).

  That is, the node B 20 transmits MSG 4 at the transmission time “0.560” in response to the received MSG 3. This transmission time is outside the range of the expected value “0.500 ± 0.05” of the transmission time shown in FIGS.

Therefore, in the second operation result example, the display unit 48 distinguishes the log information related to MSG4 from the log information related to MSG1 to 3 and displays it on the display. Specifically, the display unit 48 displays, for example, as shown in FIG. FIG. 6 shows a state in which the display unit 48 performs display in the second operation result example. As shown in FIG. 6, the display unit 48 displays only the log information related to MSG4 “Authentication Information Request”, for example, by coloring, and displays the log information related to MSG4 and the log information related to MSG1 to MSG1 to 3 so that the user can easily identify them. is doing. By this display, the MSG 4 allows the user to easily grasp that the message is transmitted from the node B 20 to the node C 30 and that the transmission time is “0.560” and is outside the expected value range. be able to. In FIG. 6, for example, an expected value range may be displayed adjacent to the transmission time column.
(Example of third operation result)
FIG. 7 shows a third operation result example. As shown in FIG. 7, in this example, MSG4 in the first operation result example (see FIG. 4) is MSG8.

  That is, in response to the received MSG3, the node B20 transmits MSG8 “Update Location Request” to the node C30 at the transmission time “0.460”. Although this transmission time is within the range of the expected value “0.500 ± 0.05” of the transmission time shown in FIGS. 2B and 3, the message destination and content are different from the reference scenario. That is, if the sequence described in the reference scenario is correct, there is a suspicion that a failure has occurred in the Node B 20 that has performed an operation different from the reference scenario.

  Therefore, in the third operation result example, the display unit 48 identifies the log information related to MSG4 from the log information related to MSG1 to MSG3 and displays it on the display.

  As described above, the sequence test system 100 according to the present embodiment is transmitted based on the comparison result of the comparison unit 47 and the log information related to the response message transmitted within the range of the expected value and the range outside the range of the expected value. Identify and display log information related to response messages. Therefore, the sequence test system 100 can easily specify log information outside the expected value range.

(Other aspects 1)
The present invention is not limited to the above-described embodiment, and various modifications can be made. For example, in the above-described embodiment, the sequence test is performed on a plurality of nodes connected to a wired communication network. However, the present invention is not limited to this. For example, a sequence test can be performed with a configuration as shown in FIG.

  The sequence test system 200 shown in FIG. 8 includes a node A 70 and a node B 80. The node A 70 includes a sequence test device 40 and a communication device 90. The description of the configuration of the sequence test apparatus 40 has already been given and will be omitted. Node B 80 constitutes a communication means according to the present invention.

  The node B 80 and the communication device 90 are configured to perform wireless communication with each other using RF signals, and include antennas 81 and 91, respectively. Instead of the antennas 81 and 91, the node B 80 and the communication device 90 may be connected by a coaxial cable to exchange RF signals. In the case of this configuration example, specifically, the communication device 90 is a pseudo base station that simulates a base station of a mobile communication system. Further, the Node B 80 to be tested is a mobile communication terminal of the mobile communication system, for example, a mobile phone or a mobile terminal.

  With the above-described configuration, the sequence test system 200 performs a sequence test based on the wireless communication protocol with respect to the Node B 80, and log information regarding the response message transmitted within the expected value range, and the expected value range. Log information related to response messages transmitted outside can be identified and displayed. Therefore, the sequence test system 200 can easily specify log information outside the expected value range.

(Other aspect 2)
In the above-described embodiment, the node B20 and the node C30 are exemplified as the communication unit according to the present invention. However, the present invention is not limited to this, and can be applied to those operating in a predetermined sequence. For example, the present invention uses each layer (physical layer, medium access control layer, radio link control layer, etc.) of a multi-layered protocol stack based on the 3GPP (Third Generation Partnership Project) standard as a communication means. The same effects as those of the above-described embodiment can be obtained.

  As described above, the sequence test apparatus and the sequence test method according to the present invention have an effect that log information outside the expected value range can be easily specified, and communication terminals such as mobile phones and mobile terminals are specified. It is useful as a sequence test apparatus and a sequence test method for testing with a sequence based on the communication protocol.

10, 70 Node A
11, 12, 13 Communication cable 20, 80 Node B (communication means)
30 Node C (communication means)
40 Sequence test equipment 41 Reference scenario registration unit (reference scenario registration means)
42 Reference scenario storage unit 43 Threshold setting unit (threshold setting means)
44 threshold storage unit 45 log information acquisition unit (log information acquisition means)
46 log information storage unit 47 comparison unit (response time comparison means)
48 Display section (display means)
50, 90 Communication device 60 Tap 81, 91 Antenna 100, 200 Sequence test system

Claims (2)

A sequence test apparatus (40) for testing a communication means (20, 30) for executing a sequence for transmitting and receiving a message of a predetermined communication protocol,
A received message received by the communication means and a response message transmitted in response to the received message; information on an ideal value of a response time from when the communication means receives the received message to when the response message is transmitted; A reference scenario registration means (41) for registering as a reference scenario;
Threshold setting means (43) for setting a threshold of the response time;
Log information acquisition means (45) for acquiring log information including the response time when causing the communication means to execute a sequence for transmitting and receiving a message based on the reference scenario registered by the reference scenario registration means;
A set response time range indicating a time range obtained by adding a threshold value of the response time set by the threshold value setting means to the response time registered by the reference scenario registration means; and acquired by the log information acquisition means Response time comparison means (47) for comparing the response time included in the log information,
Based on the comparison result of the response time comparison means, the log information regarding the response message transmitted within the set response time range and the log information regarding the response message transmitted outside the set response time range are identified and displayed. Display means (48) for providing a sequence test apparatus. A sequence test method for testing a communication means for executing a sequence for transmitting and receiving a message of a predetermined communication protocol,
A received message received by the communication means and a response message transmitted in response to the received message; information on an ideal value of a response time from when the communication means receives the received message to when the response message is transmitted; A reference scenario registration step for registering as a reference scenario,
A threshold setting step for setting a threshold for the response time;
A log information acquisition step of acquiring log information including the response time when the communication unit executes a sequence of transmitting and receiving a message based on the reference scenario registered in the reference scenario registration step;
A set response time range indicating a time range obtained by adding the response time threshold value set in the threshold value setting step to the response time registered in the reference scenario registration step; and acquired in the log information acquisition step A response time comparison step for comparing the response time included in the logged log information;
Based on the comparison result in the response time comparison step, the log information regarding the response message transmitted within the set response time range and the log information regarding the response message transmitted outside the set response time range are identified and displayed. And a display test step.

Images