CN106789446B - Node peer-to-peer cluster distributed test system and method - Google Patents
Node peer-to-peer cluster distributed test system and method Download PDFInfo
- Publication number
- CN106789446B CN106789446B CN201710087509.8A CN201710087509A CN106789446B CN 106789446 B CN106789446 B CN 106789446B CN 201710087509 A CN201710087509 A CN 201710087509A CN 106789446 B CN106789446 B CN 106789446B
- Authority
- CN
- China
- Prior art keywords
- node
- test
- zmq
- socket
- cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/50—Testing arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
Abstract
The invention discloses a node-peer cluster distributed test system, which comprises at least one test task sender node, a plurality of test task receiver nodes, a plurality of test result sender nodes and at least one test result collector node, wherein all the nodes are connected through a network; also discloses a cluster distributed test method of the node peer, which is applied to the cluster distributed test system of the node peer. The invention relates to a cluster distributed test system and a method of node peer, wherein the system makes full use of the distributed characteristic to remove centralization, each node in the cluster is not only the publisher of the test task, but also the receiving executor of the test task; the test result collecting and storing device is not only a sender of the test result, but also a collector of the test result; and an efficient and reliable distributed testing method for node peer is also provided. The invention is used as a node peer-to-peer cluster distributed test system and method, and can be widely applied to the field of test data transmission.
Description
Technical Field
The invention relates to the field of test data transmission, in particular to a node peer-to-peer cluster distributed test system and a node peer-to-peer cluster distributed test method.
Background
The IOZONE is a benchmark tool of a file system, and can test the read-write performance of the file system in different operating systems.
SANERGY is a memory sharing, bandwidth testing software.
IOPS (Input/Output Operations Per Second), i.e., the number of read/write (I/O) Operations Per Second, is often used in databases and other applications to measure the performance of random access.
ZeroMQ (hereinafter ZMQ for short) is a generic message layer, where Zero means Zero latency, also defined as the "fastest history message queue". ZeroMQ provides 3 basic communication modes: Request-Reply, Publish-Subscribe, Parallel Pipeline. The distributed architecture can be conveniently and quickly constructed by using the zeroMQ.
Along with the increasing of the number of hosts in the cluster and the uncertainty of the directions in the distributed layout of the hosts, the original test method for synchronizing the test tasks of all the hosts in the cluster by using scripts has the defects of complexity, low efficiency and the like. If the number of the hosts is too large, the test result of a single host can be still checked once after the test task is executed, and the analysis, the evaluation and the optimization of the overall performance of the cluster are not facilitated.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide a distributed test system and method for a peer-to-peer cluster, which solve the problems that the number of nodes in the cluster is increased and the orientation in the distributed layout of the nodes is uncertain, which results in the complicated test tasks for all the nodes in the synchronous cluster, and the test results of all the nodes in the cluster cannot be checked at the same time after the test tasks are executed.
The technical scheme adopted by the invention is as follows: a node-peer cluster distributed test system comprises at least one test task sender node, a plurality of test task receiver nodes, a plurality of test result sender nodes and at least one test result collector node, wherein all the nodes are connected through a network, each node is based on ZMQ _ DEALER sockets to send test tasks to a corresponding node, each node is based on ZMQ _ ROUTER sockets to receive test tasks sent by the corresponding node, the test result sender node is based on ZMQ _ PUB sockets to send test results to the test result collector node, and the test result collector node sets ZMQ _ SUB sockets to collect test results sent by the test result sender node.
The other technical scheme adopted by the invention is as follows: a distributed test method of a node peer cluster is applied to the distributed test system of the node peer cluster, and comprises the following steps:
s1, starting all nodes, and creating a ZMQ _ Router socket for each node and binding to receive messages;
s2, each node discovers other nodes in the cluster through the network, and each discovered node creates a ZMQ _ DEALER socket to be connected with the ZMQ _ ROUTER socket of the discovered node;
s3, judging whether the node is a test task sender node, if so, sending test tasks to a plurality of opposite end nodes by the node through a plurality of ZMQ _ DEALER sockets, executing the test tasks and storing test results; otherwise, the node receives the test task through the ZMQ _ route socket, executes the test task and stores the test result;
s4, judging whether the node is a test result sender node or not, if so, establishing ZMQ _ PUB socket by the node, and connecting the node with ZMQ _ SUB socket through ZMQ _ PUB socket to send a test result; otherwise, the node creates ZMQ _ SUB socket, receiving the test results through ZMQ _ SUB socket.
Further, the step between the step S2 and the step S3 further includes the steps of:
and each node detects whether the opposite node leaves at preset time intervals, and closes an ZMQ _ DEALER socket corresponding to the leaving opposite node if the opposite node leaves.
Further, after the node creates ZMQ _ deasler socket in the step S2, setting the attribute of ZMQ _ deasler socket as the identification information of the node; and after the peer node is connected through the ZMQ _ route socket, the identification information and the address information of the node are sent to the peer node.
Further, in the step S4, the node creates ZMQ _ SUB socket, sets subscription filtering and binds ZMQ _ SUB socket, and receives the test result sent by ZMQ _ PUB socket through ZMQ _ SUB socket.
Furthermore, the node encrypts and stores the test result after executing the test task, and the node receives the test result and then encrypts and stores the test result.
Further, the method for each node to discover other nodes in the cluster through the network includes broadcasting a UDP packet or Gossip discovery protocol.
Further, the test tool performing the test task includes IOZONE or SANERGY.
Further, the test result comprises any combination of read speed, write speed, IOPS, access delay, and frame loss number.
Further, the test task includes a file IO operation.
The invention has the beneficial effects that: the invention relates to a cluster distributed test system and a method of node peer, wherein the system makes full use of the distributed characteristic to remove centralization, each node in the cluster is not only the publisher of the test task, but also the receiving executor of the test task; the test result collecting and storing device is not only a sender of the test result, but also a collector of the test result; the same test task can be synchronously executed through all the nodes in the system cluster; if one node in the cluster is paralyzed, other nodes can be selected to issue test tasks or collect test results, the test of the whole cluster is not influenced, and the test reliability is improved; in the system, each part adopts the zeroMQ Msg to encode data, and the efficient transmission performance of the zeroMQ is fully utilized by combining the PUB-SUB mode and the ROUTER/DEALER socket to construct an efficient and reliable node peer-to-peer distributed test system, so that the efficient and reliable node peer-to-peer distributed test method is provided.
Drawings
The following further describes embodiments of the present invention with reference to the accompanying drawings:
FIG. 1 is a schematic diagram of data transmission of a peer-to-peer cluster distributed test system according to the present invention;
FIG. 2 is a flow chart of steps of a distributed testing method for peer-to-peer cluster of nodes in the present invention;
fig. 3 is a flowchart illustrating steps of a distributed testing method for peer-to-peer cluster of nodes according to an embodiment of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
A cluster distributed test system of node peer, the cluster distributed test system includes at least one test task sender node, several test task receiver nodes, several test result sender nodes and at least one test result collector node, all nodes are connected through the network, refer to FIG. 1, FIG. 1 is the data transmission schematic diagram of a cluster distributed test system of node peer in the invention, all nodes interconnect and use route/DEAER socket in ZeroMQ technology, each node is based on ZMQ _ DEAER socket to send the test task to the opposite end node, each node is based on ZMQ _ route socket to receive the test task sent by the opposite end node, the test result collection uses PUB-SUB mode in ZeroMQ technology, the test result sender node is based on ZMQ _ PUB socket to send the test result to the test result collector node, like nodes 1 and 2 in fig. 1, the test result collector node sets ZMQ _ SUB socket to collect the test results sent by the test result sender node, like node N in fig. 1. In addition, network connection can be established among a plurality of nodes through a local area network or a wide area network.
A distributed test system of peer-to-peer cluster of node, the system makes full use of the distributed characteristic, decentralized, every node in the cluster is both a publisher of the test task, also is the receiving executor of the test task; it is both the sender of the test results and the collection store of the test results.
A distributed testing method for a peer-to-peer cluster, which is applied to the distributed testing system for a peer-to-peer cluster, and referring to fig. 2 and fig. 3, fig. 2 is a flow chart of steps of a distributed testing method for a peer-to-peer cluster of the present invention, fig. 3 is a flow chart of steps of a specific embodiment of a distributed testing method for a peer-to-peer cluster of the present invention, fig. 3 illustrates a data transmission situation of a single node, and the distributed testing method for a peer-to-peer cluster of a node includes the following steps:
s1, starting all nodes, and creating a ZMQ _ Router socket for each node and binding to receive messages;
s2, each node discovers other nodes in the cluster through the network, each discovered node creates a ZMQ _ DEALER socket to be connected with the ZMQ _ ROUTER socket of the discovered node, each opposite end node corresponds to a ZMQ _ DEALER socket, and fan-out data to a plurality of opposite end nodes is achieved; further, the method for each node to discover other nodes in the cluster through the network includes broadcasting UDP packets or Gossip discovery protocol; after the node creates ZMQ _ deasler socket in step S2, setting the attribute of ZMQ _ deasler socket as the unique identification information of the node; after connecting to the correspondent node through the ZMQ _ route socket, the identification information and address information of the node are sent to the correspondent node so that the correspondent node can be connected back to the node. In this embodiment, the node in the present invention is the host, and the identification information of the node is the UUID code of the node, or may be other information that is fixed and can uniquely identify the node;
as a further improvement of the technical scheme, each node detects whether the opposite node leaves at preset time intervals, and if so, closes an ZMQ _ DEALER socket corresponding to the leaving opposite node; specifically, the method for detecting whether the opposite node leaves comprises heartbeat detection in a Ping-Pong mode;
s3, judging whether the node is a test task sender node, if so, the node sends test tasks to a plurality of opposite end nodes through a plurality of ZMQ _ DEALER sockets respectively, and according to the characteristics of the ZMQ _ ROUTER sockets, the opposite end nodes can obtain UUID codes of the nodes sending the messages, execute the test tasks and encrypt and store test results; otherwise, the node receives the test task through the ZMQ _ ROUTER socket, executes the test task and encrypts and stores the test result; further, the test tool performing the test task includes IOZONE or SANERGY; the test result comprises any combination of reading speed, writing speed, IOPS, access delay and frame loss number;
s4, judging whether the node is a test result sender node, if so, establishing ZMQ _ PUB socket by the node, and connecting the ZMQ _ SUB socket with the ZMQ _ PUB socket to send a test result, wherein the ZMQ _ PUB socket is used for sending data in a fan-out mode; otherwise, the node creates ZMQ _ SUB socket, receives the test results through ZMQ _ SUB socket, encrypts and stores, and is further analyzed by the parser. Further, in step S4, the node creates ZMQ _ SUB socket, sets subscription filtering and binds ZMQ _ SUB socket, and receives the test result sent by ZMQ _ PUB socket through ZMQ _ SUB socket, so that the test results of each node arrive in a cross way without mutual influence. And finally, finishing data transmission and closing all sockets.
Due to the cross-platform and portability of the zeroMQ technology, the operating system is not limited by the method and the device. ZeroMQ can be called by most languages, including C, C + +, C #, Clojure, CL, Erlang, F #, Felix, Go, Haskell, Java, Lua, node. js, Objective-C, Perl, PHP, Python, Ruby, Scala, Tcl, Ada, Basic, Haxe, ooc.
All the nodes in the cluster can synchronously execute the same test task; if one node in the cluster is paralyzed, other nodes can be selected to issue test tasks or collect test results, the test of the whole cluster is not influenced, and the test reliability is improved; in the system, each part adopts the zeroMQ Msg to encode data, and the efficient transmission performance of the zeroMQ is fully utilized by combining the PUB-SUB mode and the ROUTER/DEALER socket to construct an efficient and reliable node peer-to-peer distributed test system, so that the efficient and reliable node peer-to-peer distributed test method is provided.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A cluster distributed test system of node peer, characterized by that, the said cluster distributed test system includes at least one test task sender node, several test task receiver nodes, several test result sender nodes and at least one test result collector node, all nodes are connected through the network, each node is based on ZMQ _ DEALER socket in order to send the test task to the correspondent node, each node is based on ZMQ _ ROUTER socket in order to receive the test task that the correspondent node sent, the test result sender node is based on ZMQ _ PUB socket in order to send the test result to the test result collector node, the test result collector node sets up ZMQ _ SUB socket in order to collect the test result that the sender node of test result sends; each node in the cluster is both a publisher and a receiver and executor of the test task, and is both a sender and a collector of the test result.
2. A distributed testing method for a cluster of node peers, applied to the distributed testing system for a cluster of node peers of claim 1, comprising the steps of:
s1, starting all nodes, and creating a ZMQ _ Router socket for each node and binding to receive messages;
s2, each node discovers other nodes in the cluster through the network, and each discovered node creates a ZMQ _ DEALER socket to be connected with the ZMQ _ ROUTER socket of the discovered node;
s3, judging whether the node is a test task sender node, if so, sending test tasks to a plurality of opposite end nodes by the node through a plurality of ZMQ _ DEALER sockets, executing the test tasks and storing test results; otherwise, the node receives the test task through the ZMQ _ route socket, executes the test task and stores the test result;
s4, judging whether the node is a test result sender node or not, if so, establishing ZMQ _ PUB socket by the node, and connecting the node with ZMQ _ SUB socket through ZMQ _ PUB socket to send a test result; otherwise, the node creates ZMQ _ SUB socket, receiving the test results through ZMQ _ SUB socket.
3. The cluster distributed test method of claim 2, further comprising, between the step S2 and the step S3, the steps of:
and each node detects whether the opposite node leaves at preset time intervals, and closes an ZMQ _ DEALER socket corresponding to the leaving opposite node if the opposite node leaves.
4. The cluster distributed test method of claim 2 or 3, wherein after the node creates ZMQ _ DEALER socket in step S2, the attribute of ZMQ _ DEALER socket is set as the identification information of the node; and after the peer node is connected through the ZMQ _ route socket, the identification information and the address information of the node are sent to the peer node.
5. The cluster distributed test method of claim 4, wherein in the step S4, the node creates ZMQ _ SUB socket, sets subscription filter and binds ZMQ _ SUB socket, and receives test results sent by ZMQ _ PUB socket through ZMQ _ SUB socket.
6. The cluster distributed test method of claim 5, wherein the nodes encrypt and store the test results after executing the test tasks, and the nodes encrypt and store the test results after receiving the test results.
7. The method of claim 6, wherein the method for each node to discover other nodes in the cluster over the network comprises broadcasting a UDP packet or a Gossip discovery protocol.
8. The clustered distributed test method of claim 7, wherein the test tools performing the test tasks comprise IOZONE or SANERGY.
9. The cluster distributed test method of claim 8, wherein the test results comprise any combination of read speed, write speed, IOPS, access latency, number of lost frames.
10. The cluster distributed test method of claim 9, wherein the test tasks include file IO operations.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710087509.8A CN106789446B (en) | 2017-02-17 | 2017-02-17 | Node peer-to-peer cluster distributed test system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710087509.8A CN106789446B (en) | 2017-02-17 | 2017-02-17 | Node peer-to-peer cluster distributed test system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106789446A CN106789446A (en) | 2017-05-31 |
| CN106789446B true CN106789446B (en) | 2020-06-02 |
Family
ID=58958571
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710087509.8A Active CN106789446B (en) | 2017-02-17 | 2017-02-17 | Node peer-to-peer cluster distributed test system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106789446B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111382737A (en) * | 2018-12-29 | 2020-07-07 | 深圳光启空间技术有限公司 | Multi-path load balancing asynchronous target detection method, storage medium and processor |
| CN109885495B (en) * | 2019-02-20 | 2022-04-05 | 重庆邮电大学 | Group testing method and system for decentralized system |
| CN109903178B (en) * | 2019-04-04 | 2021-08-20 | 腾讯科技（深圳）有限公司 | Method, device and system for determining common social contact object and computing equipment |
| CN111682961B (en) * | 2020-05-18 | 2023-03-07 | 杜晓楠 | Method for eliminating low-bandwidth nodes in I2P network, computer readable storage medium and I2P network |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101447892A (en) * | 2008-11-24 | 2009-06-03 | 中兴通讯股份有限公司 | Distributed test method, system and test server |
| CN102141962A (en) * | 2011-04-07 | 2011-08-03 | 北京航空航天大学 | Safety distributed test framework system and test method thereof |
| CN102420727A (en) * | 2012-01-05 | 2012-04-18 | 北京邮电大学 | Distributed protocol test system and method |
| CN102970176A (en) * | 2012-09-19 | 2013-03-13 | 东莞中山大学研究院 | Peer-to-peer (P2P) streaming media simulation system based on network simulator version 2 (NS2) |
| CN103297286A (en) * | 2012-02-23 | 2013-09-11 | 百度在线网络技术（北京）有限公司 | Distributed cluster reliability test system, method and device |
| CN104077212A (en) * | 2013-03-26 | 2014-10-01 | 阿里巴巴集团控股有限公司 | Pressure test system and method |
| CN104850491A (en) * | 2014-02-19 | 2015-08-19 | 深圳中兴网信科技有限公司 | Software test method, equipment and system |
-
2017
- 2017-02-17 CN CN201710087509.8A patent/CN106789446B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101447892A (en) * | 2008-11-24 | 2009-06-03 | 中兴通讯股份有限公司 | Distributed test method, system and test server |
| CN102141962A (en) * | 2011-04-07 | 2011-08-03 | 北京航空航天大学 | Safety distributed test framework system and test method thereof |
| CN102420727A (en) * | 2012-01-05 | 2012-04-18 | 北京邮电大学 | Distributed protocol test system and method |
| CN103297286A (en) * | 2012-02-23 | 2013-09-11 | 百度在线网络技术（北京）有限公司 | Distributed cluster reliability test system, method and device |
| CN102970176A (en) * | 2012-09-19 | 2013-03-13 | 东莞中山大学研究院 | Peer-to-peer (P2P) streaming media simulation system based on network simulator version 2 (NS2) |
| CN104077212A (en) * | 2013-03-26 | 2014-10-01 | 阿里巴巴集团控股有限公司 | Pressure test system and method |
| CN104850491A (en) * | 2014-02-19 | 2015-08-19 | 深圳中兴网信科技有限公司 | Software test method, equipment and system |
Non-Patent Citations (1)
| Title |
|---|
| 分布式自动化软件测试实现技术研究;杨秋辉等;《四川大学学报（工程科学版）》;20110930;第43卷(第5期);正文第127-128页、附图1-3 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106789446A (en) | 2017-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106789446B (en) | Node peer-to-peer cluster distributed test system and method | |
| RU2684192C1 (en) | Method and apparatus for transmitting service stream in flexible ethernet | |
| US10044581B1 (en) | Network traffic tracking using encapsulation protocol | |
| US8493871B2 (en) | End-to end analysis of transactions in networks with traffic-altering devices | |
| CN101841441B (en) | Test method and system for flow control device and data stream playback device | |
| US10033602B1 (en) | Network health management using metrics from encapsulation protocol endpoints | |
| US9825855B2 (en) | Information processing apparatus and route setting method | |
| US10616378B2 (en) | Adaptive session intelligence extender | |
| WO2021164261A1 (en) | Method for testing cloud network device, and storage medium and computer device | |
| WO2018171641A1 (en) | Method and device for transmitting and receiving network management information, transmitting apparatus and receiving apparatus | |
| CN104079571A (en) | Method and device for recognizing Android simulator | |
| CN113783800B (en) | Data packet processing method and device, computer equipment and readable storage medium | |
| CN106411924B (en) | A kind of method creating session forwarding-table item, the method and device that E-Packets | |
| CN107852344B (en) | Storage network element discovery method and device | |
| CN107528747B (en) | Method and device for diagnosing communication state of master station and slave station and computer readable storage medium | |
| CN114553799B (en) | Multicast forwarding method, device, equipment and medium based on programmable data plane | |
| CN110784339B (en) | LACP message overtime fault detection method and device, and electronic equipment | |
| US9716673B2 (en) | Packet storage method and packet storage apparatus | |
| CN110225133A (en) | Message method, node, device, system and relevant device | |
| CN114500633A (en) | Data forwarding method, related device, program product and data transmission system | |
| US8145698B1 (en) | Self organizing peer-to-peer system, method, and/or apparatus | |
| CN112468363B (en) | Method and device for detecting link connectivity | |
| CN112436982B (en) | Network flow automatic mixed running test method, system, terminal and storage medium | |
| CN115174414A (en) | Method, system and electronic device for automatically identifying devices and device paths in session | |
| CN111106977B (en) | Data stream detection method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190912 Address after: 100089 Floor 1-4, No. 2 Building, No. 9 Courtyard, Dijin Road, Haidian District, Beijing Applicant after: Beijing Toyou Feiji Electronics Co., Ltd. Address before: 518000 Guangdong city of Shenzhen province Nanshan District South Road Fiyta Technology Building Room 1402 Applicant before: Shenzhen City Rui Bo deposit information technology Co. Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |