JP2011087078A - Verification device and verification method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve quality by shortening the period of verification work of a communication apparatus and a network constituted of a plurality of communication apparatuses. <P>SOLUTION: All communication apparatuses are connected in a form of a star around a connection change switch 20, and an environment constructing section 11 operates the connection change switch 20 to connect communication apparatuses. This eliminates physical wiring work when constructing a verification environment and can easily construct a verification environment. A state control section 13 can verify a state change in the verification environment, such as power source ON/OFF and Link UP/DOWN without a person in charge of verification going up to the installation place of the communication apparatuses by controlling a power supply converter 40 a link switch 50 to change states of the verification environment. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technology for verifying the quality of service of a network composed of a communication device and a plurality of communication devices.

  In commercialization of a large-scale network service such as a next generation network (NGN: Next Generation Network), service stability and resistance to a load are strongly required. For this reason, verification of communication devices used in the network is important. However, since high quality and early provision of services are required, efficient verification work is essential. Patent Document 1 is known as a method for improving the efficiency of verification between nodes of a network.

  Here, a conventional verification process will be described. FIG. 5 is a schematic configuration diagram showing a configuration of a conventional verification environment. As shown in the figure, a network to which various communication devices such as a measuring device that transmits and receives frames and a verification device to be verified is connected is constructed as a verification environment, and verification work is performed. When starting the verification work, first, the verification person performs the physical wiring work while looking at the verification configuration diagram, and sets each communication device. After that, the person in charge of verification operates the measuring device using the terminal, and transmits / receives a frame to / from the network according to the verification procedure manual. For abnormal system tests such as link disconnection and power OFF, the person in charge of verification goes to the installation location of the communication device, searches for and operates the port for LinkUP / DOWN, or to the installation location of the communication device that turns the power ON / OFF. To turn on / off the power. Finally, the work of collecting the information of each communication device and collecting it as a verification result is performed.

Japanese Patent Laid-Open No. 10-98468

  The verification work is performed in a very large number of patterns while changing the network configuration, device settings, frame type, abnormal system status, and the like. Therefore, long-term verification work was necessary with a large number of people as the scale of the network increased.

  In addition, because verification work is performed using a limited number of verification machines, changing to a different verification environment or returning to a previous verification environment often occurs, but it is easy to change the verification environment as described above. However, there is a problem that the verification environment cannot be constructed immediately even when urgent is required. In Patent Document 1, although verification efficiency is improved by automation, the verification environment cannot be automatically changed.

  When performing large-scale verification work, the verification work is distributed to multiple workers, but when the verification machines assigned to each verification officer overlap and the verification environment competes, the verification personnel's skill differences, etc. Therefore, even if there are a plurality of verification personnel, efficient verification may not proceed. In addition, there are many repetitive verification operations, and in addition, there is a problem that the burden on the verification person is heavy, such as redoing due to a verification error.

  The present invention has been made in view of the above, and an object of the present invention is to shorten the period of verification work of a network composed of a communication device and a plurality of communication devices and improve the quality.

  A verification apparatus according to a first aspect of the present invention includes one or more verification machines, a measuring instrument that transmits and receives a frame, a status changing apparatus that changes the status of a verification machine or a connection between verification machines, a verification machine, and a measuring instrument. And a verification scenario in which the state change device is connected in a star shape, and a switch that internally connects these connected communication devices is used to verify the network configuration and the verification work content Storage means for storing data, environment construction means for controlling a switch based on a verification scenario and constructing a desired network, and instrument control means for controlling a measuring instrument based on the verification scenario and transmitting and receiving frames in the network , A state control means for controlling the state change device based on the verification scenario and changing the state of the network, and the verification by the measuring instrument control means and the state control means Get the results obtained by the work, and having a a result determination means for performing acceptance judgment on the basis of the criteria described in the verification scenario for each verification item.

  The verification apparatus includes a result output unit that outputs a verification work log and a verification result by the result determination unit.

  In the verification apparatus, after the network is constructed, the environment construction means transmits a message to each of the verification machines connected to the network, receives the model type information of the verification machine, and stores the model type information and the database. It is characterized by judging whether or not the network constructed by comparing the response message of the stored verifier is appropriate.

  In the above verification apparatus, the environment construction means sets a use flag in a verifier used when constructing a network, and does not start verification for a verification scenario using the verifier in which the use flag is set. It is characterized by that.

  The verification method according to the second aspect of the present invention includes one or more verification devices, a measuring device that transmits and receives a frame, a state change device that changes a connection state between the verification device or the verification devices, a verification device, and a measurement device. And a verification method for performing verification using a switch that connects the state change devices in a star shape and internally connects these connected communication devices, and describes a network configuration and verification work contents Reading the verification scenario from the storage means for storing the data, controlling the switch based on the verification scenario, building a desired network, controlling the measuring device based on the verification scenario, and transmitting / receiving frames in the network; Controlling the state change device based on the verification scenario, changing the state of the network, sending and receiving frames, and the network Get the results obtained by the verification by the state change over click, and having a step of performing acceptance judgment, the on the basis of the criteria described in the verification scenario for each verification item.

  By connecting communication devices in a star shape and providing a switch in the verification environment that internally connects these connected communication devices, and the environment construction means controls the switch based on the verification scenario and constructs the desired network. Since the person in charge of verification does not need to construct a network by connecting communication devices with cables, a network to be verified can be easily constructed. In addition, a state change device that changes the state of the network, for example, a device that controls power ON / OFF or a device that controls LinkUP / DOWN is provided in the verification environment, and the state control means sets the state change device based on the verification scenario. By controlling and changing the state of the network, the person in charge of verification does not have to go to the communication device and change the state of the power supply ON / OFF. In addition, since the verification results are output and stored for each item, it is not necessary to collect the verification results.

  As described above, since the verification apparatus can perform a process that conventionally requires a manual operation by a verification person, a series of verification operations can be continuously performed.

  ADVANTAGE OF THE INVENTION According to this invention, the period of the verification operation | work of the network comprised with the communication apparatus and the some communication apparatus can be shortened, and quality can be improved.

It is a schematic block diagram which shows the structure of the verification environment in this Embodiment. It is a block diagram which shows the structure of the network constructed | assembled as the verification environment of a unit test. It is a block diagram which shows the structure of the network constructed | assembled as the verification environment of a composite test. It is a flowchart which shows the flow of a process of the verification apparatus in this Embodiment. It is a schematic block diagram which shows the structure of the conventional verification environment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic configuration diagram showing the configuration of the verification environment in the present embodiment. The verification environment 1 shown in the figure includes a verification device 10, a connection change switch 20, a measuring instrument 30, a power converter 40, a link switch 50, a log server 60, an external terminal 70, a hub 80, and verification machines 90A, 90B, and 90C. Prepare. In this verification environment 1, a desired network configuration is constructed using the connection change switch 20, and frames are transmitted / received using the measuring device 30 to verify the verifiers 90A, 90B, 90C and the constructed network. . Further, the state of the verification environment 1 such as power ON / OFF, LinkUP / DOWN, etc. is changed by the power converter 40, the link switch 50, etc., and the influence at the time of failure occurrence is confirmed.

  The verification environment 1 is for verifying not only the verification machines 90A, 90B, and 90C but also the entire constructed network. In the case of a unit test for verifying individual communication devices, it is verified whether the specifications announced by the vendor have been issued, whether the system has rebooted normally, and whether an alarm is appropriately issued in the event of an abnormality. In the case of a combined test that verifies the entire network, it is verified whether the redundancy protocol is functioning normally, whether there is no problem in connection between different devices, and whether an error such as a link failure is appropriate. Is called.

  The verification apparatus 10 is an apparatus that comprehensively performs verification work by operating each communication device in the verification environment 1 based on a verification scenario created by a verification person. Details and processing of each unit of the verification apparatus 10 will be described later.

  The connection change switch 20 is a device capable of “connecting”, “disconnecting”, and “switching” mechanically or electrically between arbitrary ports. Each communication device in the verification environment 1 is connected in a star shape around the connection change switch 20. The connection change switch 20 mechanically or electrically connects each port to which the communication device is connected, so that the communication device is communicably connected and a desired network can be flexibly constructed.

  The measuring device 30 is a device that transmits and receives frames in a network constructed as a verification target and collects data such as the number of received frames and delay values.

  The power converter 40 changes the power ON / OFF and the voltage / current state of the communication device connected to the power converter 40.

  The link switch 50 controls connection and disconnection of the cable to change the LinkUP / DOWN state, and simulates a pseudo failure of a predetermined cable.

  The log server 60 collects and accumulates logs transmitted from each communication device.

  The external terminal 70 is, for example, a so-called personal computer, and is used for GUI verification such as a Web server.

  The hub 80 connects the verification device 10 and each communication device to enable communication, and enables each communication device to be operated from the verification device 10.

  The verification machines 90A, 90B, and 90C are communication devices to be verified. A plurality of the same types may be arranged, and of course, communication devices from different vendors may be arranged.

  Next, the verification apparatus 10 will be described. As shown in FIG. 1, the verification device 10 includes an environment construction unit 11, a measuring instrument control unit 12, a state control unit 13, a result determination unit 14, a storage unit 15, and a result output unit 16. The environment construction unit 11, the measuring instrument control unit 12, the state control unit 13, and the result determination unit 14 operate based on the verification scenario stored in the storage unit 15. The verification scenario describes a network configuration to be built as a verification target, communication device setting information, measuring device operation details, verification environment state control details, and the like. The verification scenario is created by the worker and stored in the storage unit 15.

  In addition, each part with which the verification apparatus 10 is provided is good also as what is comprised by the computer provided with the arithmetic processing unit, the memory | storage device, memory, etc., and the process of each part is performed by a program. This program is stored in a storage device included in the verification device 10, and can be recorded on a recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, or provided through a network. Hereinafter, each part will be described in detail.

  The environment construction unit 11 configures a network to be verified and sets each communication device. More specifically, the environment construction unit 11 constructs a desired network by sending a control command to the connection change switch 20 based on the network configuration described in the verification scenario, and describes the verification scenario. Based on the setting information of the communication device, the setting information is transmitted to each communication device to set the communication device. Since all the communication devices are preliminarily connected around the connection change switch 20, a desired network configuration can be easily established only by the environment construction unit 11 controlling the connection change switch 20.

  FIG. 2A is a configuration diagram showing the configuration of the network when the verification of the verification machine 90A is performed. In this case, the connection change switch 20 is controlled so as to be connected to “measuring instrument 30—verifier 90A—measuring instrument 30”. FIG. 2B is a configuration diagram showing a configuration of another network connected to “measuring instrument 30 -verifier 90B-measuring instrument 30”. When changing from the configuration shown in FIG. 2A to the configuration shown in FIG. 2B, the network can be changed by simply controlling the connection change switch 20 and changing the connection inside the connection change switch 20 without performing physical wiring work. You can change the configuration. As shown in FIGS. 3A and 3B, even if the configuration of the network is more complicated, the environment construction unit 11 only controls the connection change switch 20, and FIGS. The configuration of the network can be easily changed to (b) or from FIG. 3 (b) to FIG. 3 (a).

  The network configuration and communication device setting information described in the verification scenario can be used, for example, by changing the network configuration to “measurement device 30, verification device 90A, measurement device” so that it can cope with future configuration changes or addition of new models. 30 ”, the data is converted into a data that can be easily identified. As a result, the network configuration can be corrected with a simple operation. It is also easy to create a network configuration in which a part of the previously used network configuration is changed.

  Since the environment construction unit 11 can read the verification scenario and change the network configuration simply by controlling the connection change switch 20, even if a highly urgent verification work interruption occurs, the verification environment is changed to change the urgency. It is possible to easily implement a high verification operation temporarily.

  The measuring instrument control unit 12 controls the measuring instrument 30 to perform verification. More specifically, the measuring instrument control unit 12 transmits a control command to the measuring instrument 30 based on the operation contents of the measuring instrument described in the verification scenario, sets the frame format to be transmitted, and arbitrarily The measuring device 30 executes transmission / reception of frames at timing, and collects results such as the number of received frames and delay values.

  Since the measuring instrument 30 can be controlled using a script language, the measuring instrument control unit 12 divides the work contents of the measuring instrument 30 into parts by the script language. The work contents of the measuring device 30 include, for example, frame generation, frame transmission, frame transmission stop, and frame reception result acquisition. Since the work contents of the measuring instrument 30 necessary for verification are pre-packaged and mounted, the person in charge of the verification simply records the necessary parts (work contents), frame format, etc. in the verification scenario in text format. The device 30 can be operated appropriately. Even when a new control method for the measuring instrument 30 is required, by defining (programming) a new part corresponding to the control method, the control method can be described and used in the verification scenario.

  As described above, the measuring instrument control unit 12 controls the measuring instrument 30 based on the operation contents of the measuring instrument described in the verification scenario and executes the verification work. It is possible to verify such as changing the frame length from the minimum to the maximum with the minimum step size, which has been difficult due to the time restriction. In addition, when the person in charge repeats simple work as in the past, work omissions, leaks, or errors occur, but the work contents described in the verification scenario are executed reliably, so accurate verification is performed. It becomes possible to improve the verification quality.

  The state control unit 13 controls the state of the verification environment 1 and confirms the influence on each communication device and the entire network. More specifically, the state control unit 13 controls the power converter 40 to change the power ON / OFF state of the communication device, and controls the link switch 50 to change the LinkUP / DOWN state between the communication devices. Let Alternatively, the communication device setting is changed. Then, in the verification environment 1, when a change in state such as power ON / OFF, LinkUP / DOWN, or a change in setting of the communication device occurs, it is confirmed how the communication device or the entire network is affected.

  Since there are a large number of verification patterns, such as the order of setting communication devices in the network, the LinkUP / DOWN pattern between communication devices, or the power ON / OFF order of communication devices, all the states have been conventionally known. Although it was difficult to realize the verification, the state control unit 13 calculates all combinations in each state of the connection between the communication device and the communication device, and sequentially changes the combination of the states for each combination. Collect and verify the status and logs of communication devices.

  Further, the state control unit 13 mounts control contents such as LinkUP, LinkDOWN, power ON, and power OFF as components. Therefore, when the person in charge of verification selects these components (control contents) and describes them in the verification scenario, the state control unit 13 operates the power converter 40, the link switch 50, and the like based on the control contents described in the verification scenario. . The state of the verification environment can be easily controlled at an arbitrary timing.

  The result determination unit 14 obtains the log regarding all setting operations performed during the verification work, the alarm information of each communication device, and the information obtained by the verification work from the log server 60 and stores them in units of verification items. It is determined whether or not each piece of information indicates a correct state for each verification item. Then, the determination result is output as alert information. Judgment criteria for pass / fail of each verification item is described in the verification scenario. In addition, complicated verification automation is realized by branching subsequent verification scenarios according to the pass / fail determination result.

  The result output unit 16 outputs a verification pass / fail result list and detailed verification data in time series, verification item units, and the like based on the verification results and information stored in the storage unit 15. Since the result can be browsed in the HTML format, it can be browsed freely from any terminal. The result output unit 16 can also notify the person in charge of the verification by e-mail.

  As described above, each unit operates based on the description of the verification scenario, and executes the verification work. In the present embodiment, the contents of work performed by each unit are converted into parts, and a verification scenario is created in units of parts. Therefore, a new verification scenario can be created simply by selecting and combining already created parts. When completely new verification work occurs and it is difficult to divert parts, a more complete parts group can be constructed by defining and creating new parts.

  Next, the processing flow of the verification apparatus will be described.

  FIG. 4 is a flowchart showing a processing flow of the verification apparatus 10. When the verification device 10 receives an instruction to start verification work, the environment construction unit 11 reads the verification scenario from the storage unit 15 and constructs a verification environment based on the verification scenario (step S11). In constructing the verification environment, network construction and communication device settings are performed.

  The environment construction unit 11 constructs a network by transmitting a control command to the connection change switch 20 based on the network configuration described in the verification scenario, and based on the setting contents of the communication device described in the verification scenario. Then, the setting information is transmitted to each communication device to set the communication device.

  When the verification environment is constructed, the environment construction unit 11 transmits an inquiry message to each communication device in the constructed verification environment, and receives model type information of each communication device as a response to the message. Then, based on the received model type information, it is determined whether or not the network configuration and the setting contents of the communication device are appropriate. The environment construction unit 11 has a database storing response messages of communication devices of each vendor, and whether the setting is appropriate by comparing the received model type information with the response message stored in the database. Determine. As a result of the determination, if an error or an unexpected situation occurs in the verification environment, the verification person is notified of the location and contents. By having a database storing response messages of each communication device, it becomes possible to support communication devices of a plurality of different vendors.

  Subsequently, the measuring instrument control unit 12 reads the verification scenario from the storage unit 15 and controls the measuring instrument based on the verification scenario (step S12). When the measuring instrument control unit 12 transmits a control command to the measuring instrument 30 based on the operation contents of the measuring instrument described in the verification scenario, the measuring instrument 30 transmits and receives a frame based on the control command, and the verification environment Service quality and performance verification of each communication device.

  Subsequently, the state control unit 13 reads the verification scenario from the storage unit 15, and controls the state of the verification environment based on the verification scenario (step S13). When the state control unit 13 transmits a control command to the connection change switch 20, the power converter 40, the link switch 50, and the verifiers 90A, 90B, and 90C based on the state control content of the verification environment described in the verification scenario, Based on the control command, the power converter 40 changes the power ON / OFF of the communication device, the connection change switch 20 and the link switch 50 change the LinkUP / DOWN that connects the communication devices, and verifiers 90A, 90B, 90C changes the state of the device such as reboot or port shutdown.

  The measuring instrument control unit 12 and the state control unit 13 perform various verification operations while changing the state of the verification environment while changing the settings of the verifiers 90A, 90B, and 90C while changing the type and setting of the transmitted and received frames. To do.

  The result determination unit 14 receives various logs and information from the log server 60, and calculates a verification result for each verification item (step S14).

  Finally, the result output unit 15 outputs the verification result. By referring to the output verification result, the worker in charge can determine whether the verification is successful or not without connecting to each communication device and checking the status (step S15).

  When the verification work for one verification scenario is completed, the process may return to step S11 to start execution of another verification scenario. Also, by executing a plurality of verification scenarios in parallel, the verification time can be greatly shortened. In this case, the environment construction unit 11 sets a use flag for each communication device used in the verification scenario or each function of the communication device, and refers to the use flag to determine whether the verification scenario can be executed in parallel. Judging. Verification for a verification scenario using a communication device in which the use flag is set is not started. When the verification work of the verification scenario is completed, the setting of the use flag is canceled. For example, the unit tests of the verification machines 90A, 90B, and 90C can be performed simultaneously.

  As described above, when the verification work is started by the verification device 10, the verification work is sequentially performed according to the verification scenario, so that the person in charge of the verification does not need to be resident and the verification work is performed at night or on weekends. It becomes possible. As a result, the verification period is shortened, and early provision of new services can be expected.

  Further, remote verification can be performed from a place away from the verification environment by remotely operating the verification device 10.

  In this embodiment, the verification environment is constructed and verified independently of the operating network. However, it is also possible to construct the verification environment within the commercial network environment and perform the verification work. is there.

  As described above, according to the present embodiment, all communication devices are connected in a star shape around the connection change switch 20, and the environment construction unit 11 operates the connection change switch 20 to connect the communication devices. By connecting, the physical wiring work for constructing the verification environment is eliminated, and the verification environment can be easily constructed. In addition, the state controller 13 controls the connection change switch 20, the power converter 40, the link switch 50, and the verification machines 90A, 90B, and 90C to change the state of the verification environment, so that the verification person installs the communication device. Without going to the place, it becomes possible to verify the state change of the verification environment such as power ON / OFF, LinkUP / DOWN, and the like. As described above, since the verification apparatus 10 can execute processes that have been manually performed by a person in charge of verification in the past, a series of verification operations from the construction of the verification environment to the execution of verification and the aggregation of verification results are continuously performed. Implementation became possible.

  According to the present embodiment, it is possible to easily create a verification scenario by converting the work contents of each part into parts and creating a verification scenario for each part. Since all of the verification work is performed based on the verification scenario, the difference in skills among the persons in charge of verification is less likely to appear in the verification result, and improvement in verification quality can be expected.

DESCRIPTION OF SYMBOLS 1 ... Verification environment 10 ... Verification apparatus 11 ... Environment construction part 12 ... Measuring instrument control part 13 ... State control part 14 ... Result judgment part 15 ... Accumulation part 16 ... Result output part 20 ... Connection change switch 30 ... Measuring instrument 40 ... Power supply Converter 50 ... Link switch 60 ... Log server 70 ... External terminal 80 ... Hub 90A, 90B, 90C ... Verifier

Claims (5)

One or more verifiers, a measuring device that transmits and receives a frame, a state changing device that changes the state of the verification device or a connection between the verifiers, the verification device, the measuring device, and the state changing device A verification device that performs verification using a switch connected to a mold and internally connected to these communication devices,
Storage means for storing a verification scenario describing the network configuration and verification work contents;
An environment construction means for controlling the switch based on the verification scenario and constructing a desired network;
Instrument control means for controlling the instrument based on the verification scenario and transmitting and receiving frames in the network;
State control means for controlling the state change device based on the verification scenario and changing the state of the network;
A result determination unit that obtains a result obtained by the verification operation by the measuring instrument control unit and the state control unit, and performs pass / fail determination for each verification item based on a determination criterion described in the verification scenario;
The verification apparatus characterized by having.   The verification apparatus according to claim 1, further comprising a result output unit that outputs a verification result by the result determination unit and a log of verification work.   The environment construction means, after construction of the network, sends a message to each of the verification machines connected to the network, receives model type information of the verification machine, and is stored in the model type information and the database. 3. The verification apparatus according to claim 1, wherein the verification apparatus determines whether or not the network constructed by comparing the response message of the verification machine is appropriate.   The environment construction means sets a use flag in the verifier used when constructing the network, and does not start verification for the verification scenario using the verifier in which the use flag is set. The verification apparatus according to claim 1, wherein One or more verifiers, a measuring device that transmits and receives a frame, a state changing device that changes the state of the verification device or a connection between the verifiers, the verification device, the measuring device, and the state changing device A verification method for performing verification using a switch connected to a mold and connecting these connected communication devices internally,
Reading the verification scenario from the storage means for storing the verification scenario describing the network configuration and the verification work content, controlling the switch based on the verification scenario, and building a desired network;
Controlling the measuring device based on the verification scenario and transmitting and receiving frames in the network;
Controlling the state change device based on the verification scenario and changing the state of the network;
Obtaining a result obtained by a verification operation by transmitting and receiving the frame and changing the state of the network, and performing pass / fail determination for each verification item based on a determination criterion described in the verification scenario;
A verification method characterized by comprising:

Images