CN110601917A

CN110601917A - Network performance testing method and device based on multi-system multiplexing network card design

Info

Publication number: CN110601917A
Application number: CN201910785430.1A
Authority: CN
Inventors: 周磊
Original assignee: Suzhou Wave Intelligent Technology Co Ltd
Current assignee: Suzhou Wave Intelligent Technology Co Ltd
Priority date: 2019-08-23
Filing date: 2019-08-23
Publication date: 2019-12-20
Anticipated expiration: 2039-08-23
Also published as: CN110601917B

Abstract

The invention provides a network performance testing method based on multi-system multiplexing network card design, which comprises the following steps: one subsystem of a multi-subsystem server is designated as a control end, and an instruction is sent to all the subsystems through the control end to open ports, wherein the ports correspond to ports opened by a network sending end one by one; each subsystem simultaneously receives network data from the network sending end to carry out bandwidth test and sends a bandwidth data value tested in each monitoring duration to the control end in real time, so that the control end compares the bandwidth data value with a bandwidth expected value of a single subsystem; and the control end adds the bandwidth data values of all the subsystems tested in each monitoring time length, and compares the obtained total bandwidth data value with the total bandwidth expected value of the whole server. The invention can realize the synchronization of a plurality of systems and the simultaneous test, and more accurately reflect whether the network equipment per se meets the design requirements.

Description

Network performance testing method and device based on multi-system multiplexing network card design

Technical Field

The invention relates to the field of computers, in particular to a network performance testing method and device based on multi-system multiplexing network card design.

Background

With the continuous increase of data to be processed, the form of server equipment changes rapidly, and in order to meet the larger calculation requirements under the condition of a limited computer room, a server design with a bias calculation appears at present, that is, under a complete machine system with power supply and heat dissipation of a server, not only one calculation and a storage system used by the calculation but also 2 or more independent calculation systems are integrated. The design is different from a two-way or four-way server, each CPU independently corresponds to one system, a plurality of systems under the same whole machine are mutually independent, different services are processed, and the on-off state of each system is not influenced. In the whole system, under the condition that the heat dissipation is exerted to the limit, the processing speed of the server can be greatly improved. Meanwhile, for users with low computing requirements, more processing service types and low correlation among the processing service types, different services can be deployed on a plurality of systems, and the purchasing cost of the server can be saved for the client. For a server enterprise, the profit of the whole integrated system is high, the integration level of the functions of the main board is high, the assembly and the maintenance are more convenient, and the profit of the enterprise is a better development direction.

Because the integration level of the whole machine system with the design is very high, besides the heat dissipation system and the power supply system are provided by the whole machine, more services are built on the mainboard in a shared mode, such as a server management control module (BMC), a mainboard logic CPLD service and the like. In addition, the network device may be a system in which a plurality of systems are multiplexed. Under the condition of the design, network card equipment with higher bandwidth, such as a 50G/100G network card, can be selected, and when the network card is multiplexed, the network bandwidth is evenly distributed to a plurality of systems, so that cost saving can be realized, and each independently operated system can be ensured to share higher network bandwidth. However, for test verification, the bandwidth in actual use may fall below an acceptable range due to non-time-sharing multiplexing of the network bandwidth and the network card device with a larger bandwidth due to its own reasons and system design. Therefore, in the test validation phase, the bandwidth of the network device must be tested. The design of common multiplexing makes the network bandwidth test unable to reach the nominal value under an independent system, and in order to ensure the availability of the customer deployment service, the network card performance must reach the following standards, namely: the single system bandwidth value does not result in the data being less than the lowest expected value of the total bandwidth and the lowest expected value of the single system bandwidth allocated to the system, and when the whole machine runs, the network bandwidth of the whole machine system is not less than the lowest expected value of the total bandwidth of the used network card equipment. The conventional test method used by the current test team cannot realize monitoring, checking and testing of the network bandwidth test under the design, so that the performance monitoring of the network part under the design is very difficult.

Disclosure of Invention

In view of this, an object of the embodiments of the present invention is to provide a method and an apparatus for testing network performance based on a multi-system multiplexing network card design, which can effectively test the performance of network bandwidth of a single system in the above-mentioned design of the whole multi-system multiplexing network card, integrate the test conditions of the whole network, obtain a conclusion whether the design requirement between the network card itself and the motherboard design can be met, and obtain a level whether compatibility of each part meets the expected requirement or not at the fastest speed, thereby laying a foundation for improvement of subsequent design and realization of the requirement of the customer on the conclusion.

Based on the above object, an aspect of the embodiments of the present invention provides a network performance testing method based on a multi-system multiplexing network card design, including the following steps:

one subsystem of a multi-subsystem server is designated as a control end, and an instruction is sent to all the subsystems through the control end to open ports, wherein the ports correspond to ports opened by a network sending end one by one;

each subsystem simultaneously receives network data from the network sending end to carry out bandwidth test and sends a bandwidth data value tested in each monitoring duration to the control end in real time, so that the control end compares the bandwidth data value with a bandwidth expected value of a single subsystem;

and the control end adds the bandwidth data values of all the subsystems tested in each monitoring time length, and compares the obtained total bandwidth data value with the total bandwidth expected value of the whole server.

In some embodiments, the method further comprises:

before executing the network performance test, the network sending end starts a bandwidth sending service, and all subsystems of the server carry out time synchronization.

In some embodiments, the method further comprises:

responding to the situation that the bandwidth expectation value of a single subsystem is not reached, displaying an alarm word on a screen of a server and recording a log of the subsystem; and

and in response to the total bandwidth expected value of the whole server is not reached, displaying an alarm word on a screen of the server and recording logs of all subsystems.

In some embodiments, the method further comprises:

and in response to reaching the bandwidth expectation value of the single subsystem and reaching the total bandwidth expectation value of the whole server, continuing to monitor the bandwidth data value in the next monitoring time period.

In some embodiments, the method initiates a network performance test using iperf or netperf.

In some embodiments, the designating a subsystem of a multi-system server as a control end and sending an instruction to all the subsystems through the control end to open a port, where the port corresponds to a port opened by the network sending end one to one includes:

and the control end sends an instruction for opening the port to all the subsystems and also sends an instruction for specifying the total testing time length, the length of each monitoring time length, the size of a received data block and the thread number of each subsystem bandwidth test.

In some embodiments, the number of threads tested per subsystem bandwidth is greater than 1.

In some embodiments, a network time protocol server is used for time synchronization of all the subsystems.

Another aspect of the embodiments of the present invention provides a network performance testing apparatus based on a multi-system multiplexing network card design, including:

at least one processor; and

a memory storing program code executable by the processor, the program code implementing the following steps when executed by the processor:

In some embodiments, the steps further comprise:

The invention has the following beneficial technical effects: the network performance testing method and the device based on the design of the multi-system multiplexing network card can effectively test the network bandwidth performance of a single system in the complete machine design of the multi-system multiplexing network card, integrate the test condition of the complete machine and obtain the conclusion whether the design requirement between the network card and a mainboard can be met, can reduce the deviation and delay caused by a test method to the maximum extent for the single system and the complete machine system, more accurately embody whether the network equipment meets the design requirement, and lay a foundation for the improvement of subsequent design and the realization of the conclusion requirement by a client.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.

FIG. 1 is a flow chart of a network performance testing method based on a multi-system multiplexing network card design according to the present invention;

fig. 2 is a schematic flowchart of a network performance testing method based on a multi-system multiplexing network card design according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a hardware structure of a network performance testing apparatus designed based on a multi-system multiplexing network card according to the present invention.

Detailed Description

Embodiments of the present invention are described below. However, it is to be understood that the disclosed embodiments are merely examples and that other embodiments may take various and alternative forms. The figures are not necessarily to scale; certain features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present invention. As one of ordinary skill in the art will appreciate, various features illustrated and described with reference to any one of the figures may be combined with features illustrated in one or more other figures to produce embodiments that are not explicitly illustrated or described. The combination of features shown provides a representative embodiment for a typical application. However, various combinations and modifications of the features consistent with the teachings of the present invention may be desired for certain specific applications or implementations.

In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.

Based on the above purpose, an embodiment of the present invention provides a network performance testing method based on a multi-system multiplexing network card design, as shown in fig. 1, including the following steps:

step S101: one subsystem of a multi-subsystem server is designated as a control end, and an instruction is sent to all the subsystems through the control end to open ports, wherein the ports correspond to ports opened by a network sending end one by one;

step S102: each subsystem simultaneously receives network data from the network sending end to carry out bandwidth test and sends a bandwidth data value tested in each monitoring duration to the control end in real time, so that the control end compares the bandwidth data value with a bandwidth expected value of a single subsystem;

step S103: and the control end adds the bandwidth data values of all the subsystems tested in each monitoring time length, and compares the obtained total bandwidth data value with the total bandwidth expected value of the whole server.

In some embodiments, the method further comprises: before executing the network performance test, the network sending end starts a bandwidth sending service, and all subsystems of the server carry out time synchronization. The whole machine of the network sending end starts the bandwidth sending service, so that the subsystem(s) of the sending end are in a state of constantly and stably outputting data blocks. The whole machine of the sending end only realizes the function of sending data blocks and does not have other work. In one embodiment, as shown in fig. 2, the multiple subsystems of the server (i.e., the receiving end) first synchronize time through a network time protocol server (NTP server). After synchronization is completed, acquiring system time from a plurality of subsystems and comparing the acquired time, if synchronization fails (the time of each subsystem is different), performing time synchronization operation again until synchronization is completed; if the synchronization is successful (the time of each subsystem is the same), one subsystem is designated as a control end in a plurality of subsystems of the receiving end server, and meanwhile, the subsystem also realizes the monitoring and alarm output functions.

In some embodiments, the method initiates a network performance test using iperf or netperf. The sending and receiving module of the network bandwidth is realized based on iperf or netperf software under Linux, and the software can continuously and stably output extremely many data blocks which fully occupy the network bandwidth at the bandwidth output end so as to fully run the bandwidth on a network card link.

In some embodiments, the designating a subsystem of a multi-system server as a control end, and sending an instruction to all the subsystems through the control end to open a port, where the port corresponds to a port opened by the network sending end one to one includes: and the control end sends an instruction for opening the port to all the subsystems and also sends an instruction for specifying the total testing time length, the length of each monitoring time length, the size of a received data block and the thread number of each subsystem bandwidth test. The control end sends a command to the receiving end server complete machine, and the realization function is as follows: and opening a receiving port of the network bandwidth test opened by the sending end, wherein the opened multiple ports correspond to the ports opened by the sending end one by one, and the corresponding mode can be a unique fixed IP (Internet protocol) used for the sending port. In addition, the command to open a port will include a specification of the total test duration, the length of each monitor duration (time space), the size of the received data block (where the data block size is specified to ensure the stability of the total bandwidth using the same size data block), and the number of threads tested per subsystem bandwidth. In one embodiment according to the invention, more than 1 thread number is used, with the aim that for bandwidths above 1Gbit/s, normally one thread cannot reach the desired bandwidth value, as in the following command example:

#iperf-c 10.11.12.13-i 2-w 512k-t 86400-P 8

represents: data is received from the 10.11.12.13 interface, the length of each time space is 2 seconds, the size of each received data block is 512k, the total test time is 86400s, and each test uses 8 threads for testing. Then, the test result is stored, the result in each subsystem is independently stored, and the result is sent to the control end, so that data processing and monitoring are realized. Because the last line of each test result is the bandwidth data of the last time space, that is, the total bandwidth data of all threads in one time space, it is only necessary to monitor the last line of data of all subsystems of the server after each time space is finished, and perform data processing on the last line of data.

In some embodiments, the method further comprises: and in response to the bandwidth expectation value of the whole server not being reached, displaying the alarm word on a screen of the server and recording the logs of all the subsystems. In some embodiments, the method further comprises: and in response to reaching the bandwidth expectation value of the single subsystem and reaching the total bandwidth expectation value of the whole server, continuing to monitor the bandwidth data value in the next monitoring time period.

In an embodiment of the present invention, as shown in fig. 2, a method for implementing bandwidth detection, data analysis and alarm of a single system is as follows: after each time space is finished, capturing the last line (namely the bandwidth data of the previous time space) under each subsystem, extracting the bandwidth data in the line through the shell language of the Linux system, comparing the bandwidth data with the expected bandwidth value of the single system, and displaying a WARNING typeface on a screen and capturing all logs in the subsystem for recording when the expected bandwidth value of the single system is not reached; and if the bandwidth expected value of the single system is reached, continuously monitoring the next time space, and entering the next step of bandwidth detection, data analysis and alarm of the whole system. The method for realizing bandwidth detection, data analysis and alarm of the whole system comprises the following steps: after each time space is finished, capturing the last line (namely the bandwidth data of the previous time space) under each subsystem, extracting the bandwidth data in the line through the shell language of a Linux system, adding the bandwidth data extracted in all the single subsystems in the time space, comparing the obtained total bandwidth data value with the bandwidth expected value of the whole system, and displaying WARNING characters on a screen and capturing all logs in all the systems for recording when the bandwidth expected value of the system is not reached; and if the bandwidth expected value of the whole system is reached, continuing to monitor the next time space.

Where technically feasible, the technical features listed above for the different embodiments may be combined with each other or changed, added, omitted, etc. to form further embodiments within the scope of the invention.

It can be seen from the foregoing embodiments that the network performance testing method and apparatus based on the design of the multi-system multiplexing network card according to the embodiments of the present invention can effectively test the performance of the network bandwidth of a single system in the overall design of the multi-system multiplexing network card, integrate the test conditions of the overall system, and obtain the conclusion whether the design requirement between the network card itself and the motherboard design can be met, so that the deviation and delay caused by the test method can be minimized for both the single system and the overall system, and the design requirement whether the network device itself meets the design requirement can be more accurately reflected, thereby laying a foundation for the improvement of the subsequent design and the realization of the discussion requirement by the customer.

The bandwidth conclusion obtained by using the method provided by the invention is data which is accepted and referred by the manufacturers of network card equipment in the industry. For the network card device itself, since the network card device is related to, for example, a frequency division multiplexing network card device, but the test performance condition of a single card needs to be tested, at this time, a test script needs to be synchronously operated under a plurality of systems, and the performance of a single component can be stably and accurately embodied only by the test result. The invention can realize the synchronous and simultaneous test of a plurality of systems, can reduce the deviation and delay caused by the test method to the maximum extent for a single system or a whole system, and more accurately reflects whether the network equipment per se meets the design requirements.

Based on the above object, in another aspect of the embodiments of the present invention, a network performance testing apparatus based on a multi-system multiplexing network card design is provided, which includes:

at least one processor; and

In some embodiments, the steps further comprise: before executing the network performance test, the network sending end starts a bandwidth sending service, and all subsystems of the server carry out time synchronization.

Fig. 3 is a schematic diagram of a hardware structure of an embodiment of a network performance testing apparatus designed based on a multi-system multiplexing network card according to the present invention.

Taking the computer device shown in fig. 3 as an example, the computer device includes a processor 301 and a memory 302, and may further include: an input device 303 and an output device 304.

The processor 301, the memory 302, the input device 303 and the output device 304 may be connected by a bus or other means, and fig. 3 illustrates the connection by a bus as an example.

The memory 302 is used as a non-volatile computer-readable storage medium, and can be used to store a non-volatile software program, a non-volatile computer-executable program, and modules, such as program instructions/modules corresponding to the network performance testing method based on the multi-system multiplexing network card design in this embodiment of the application. The processor 301 executes various functional applications and data processing of the server by running the nonvolatile software program, instructions and modules stored in the memory 302, that is, implements the network performance testing method based on the multisystem multiplexing network card design according to the above method embodiment.

The memory 302 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area can store data and the like created by a network performance testing method based on the multi-system multiplexing network card design. Further, the memory 302 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 302 optionally includes memory located remotely from processor 301, which may be connected to a local module via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input device 303 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the computer apparatus of the network performance test method based on the multisystem multiplexing network card design. The output means 304 may comprise a display device such as a display screen.

The program instructions/modules corresponding to the one or more network performance testing methods based on the multi-system multiplexing network card design are stored in the memory 302, and when being executed by the processor 301, the network performance testing method based on the multi-system multiplexing network card design in any of the above method embodiments is executed.

Any embodiment of the computer device executing the network performance testing method based on the multisystem multiplexing network card design can achieve the same or similar effects as any corresponding method embodiment.

Finally, it should be noted that, as will be understood by those skilled in the art, all or part of the processes in the methods of the above embodiments may be implemented by a computer program, which may be stored in a computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a Random Access Memory (RAM), or the like.

In addition, the apparatuses, devices and the like disclosed in the embodiments of the present invention may be various electronic terminal devices, such as a mobile phone, a Personal Digital Assistant (PDA), a tablet computer (PAD), a smart television and the like, or may be a large terminal device, such as a server and the like, and therefore the scope of protection disclosed in the embodiments of the present invention should not be limited to a specific type of apparatus, device. The client disclosed in the embodiment of the present invention may be applied to any one of the above electronic terminal devices in the form of electronic hardware, computer software, or a combination of both.

Furthermore, the method disclosed according to an embodiment of the present invention may also be implemented as a computer program executed by a CPU, and the computer program may be stored in a computer-readable storage medium. The computer program, when executed by the CPU, performs the above-described functions defined in the method disclosed in the embodiments of the present invention.

Further, the above method steps and system elements may also be implemented using a controller and a computer readable storage medium for storing a computer program for causing the controller to implement the functions of the above steps or elements.

Further, it should be appreciated that the computer-readable storage media (e.g., memory) described herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of example, and not limitation, nonvolatile memory can include Read Only Memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which can act as external cache memory. By way of example and not limitation, RAM is available in a variety of forms such as synchronous RAM (DRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to comprise, without being limited to, these and other suitable types of memory.

Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.

The various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein may be implemented or performed with the following components designed to perform the functions described herein: a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP, and/or any other such configuration.

The steps of a method or algorithm described in connection with the disclosure herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.

In one or more exemplary designs, the functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.

The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disk, or the like.

The above-described embodiments are possible examples of implementations and are presented merely for a clear understanding of the principles of the invention. Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of an embodiment of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.

Claims

1. A network performance test method based on multi-system multiplexing network card design is characterized by comprising the following steps:

2. The method of claim 1, further comprising:

3. The method of claim 2, further comprising:

4. The method of claim 3, further comprising:

5. The method of claim 1, wherein the method initiates a network performance test using iperf or netperf.

6. The method of claim 5, wherein the designating a subsystem of a multisystem server as a control end and sending a command to all the subsystems through the control end to open ports, wherein the ports are in one-to-one correspondence with the ports opened by the network sending end includes:

7. The method of claim 6, wherein the number of threads per subsystem bandwidth test is greater than 1.

8. The method of claim 2, further comprising: and performing time synchronization of all the subsystems by using a network time protocol server.

9. A network performance testing device based on multi-system multiplexing network card design is characterized by comprising:

at least one processor; and

10. The apparatus of claim 9, wherein the steps further comprise: