CN116095195B - Industrial Ethernet data packet generation method and device based on domestic platform - Google Patents
Industrial Ethernet data packet generation method and device based on domestic platform Download PDFInfo
- Publication number
- CN116095195B CN116095195B CN202310354423.2A CN202310354423A CN116095195B CN 116095195 B CN116095195 B CN 116095195B CN 202310354423 A CN202310354423 A CN 202310354423A CN 116095195 B CN116095195 B CN 116095195B
- Authority
- CN
- China
- Prior art keywords
- data
- serialization
- industrial ethernet
- record
- records
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/12—Protocol engines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
- G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses an industrial Ethernet data packet generation method and device based on a domestic platform, which can cover continuous generation of typical industrial Ethernet protocol characteristic data packets, provide direct support for industrial Ethernet flow analysis and research, industrial Ethernet safety analysis and test verification, and have good application prospects. Meanwhile, the Time Stamp Counter (TSC) is used for high-resolution time measurement to solve the problem that the operation system cannot accurately transmit according to a record given time point due to low timing precision.
Description
Technical Field
The invention relates to the technical field of network testing, in particular to a method and a device for generating industrial Ethernet data packets based on a domestic platform, which can cover continuous generation of typical industrial Ethernet protocol characteristic data packets.
Background
Network security is information security on a network, that is, hardware, software and data in a network system are protected from being damaged, changed and leaked due to accidental or malicious reasons, the system continuously and reliably operates normally, and network service is not interrupted.
Along with the deep fusion of national informatization and industrialization, a large amount of informatization and intelligent products and advanced technical means are applied to various industrial control systems, and the original physical isolation state of the traditional industrial enterprise network is broken. Various technologies are introduced, and meanwhile, information technology security risks are also introduced, so that great challenges are brought to industrial Ethernet security protection. Therefore, a large number of domestic information security manufacturers also increase the investment, and numerous industrial Ethernet security devices and software are introduced.
After the development of the industrial Ethernet safety equipment and the software is finished, the performance of the industrial Ethernet safety equipment and the software needs to be verified, and in order to ensure the accuracy of the performance verification, the industrial Ethernet safety equipment and the software usually need to be connected into the corresponding industrial Ethernet which normally operates. However, since the industrial ethernet belongs to a high-value network, any operation interruption directly affects the production rhythm of enterprises, and the conventional security device test and verification means can analyze network data after monitoring, since the security device and software access to the ethernet which is normally operated may affect the normal time sequence of the ethernet, the real-time performance and time sequence of the normal data of the industrial network are affected, the operation of the industrial device is unexpectedly affected, and the industrial ethernet security device lacking verification cannot embody the validity of the industrial ethernet security device.
In order to solve the above-mentioned problems, a network data packet generator (also called a network tester, which may be simply called a data packet generator) is proposed, and the network data packet generator generates a specific network data packet, sends the network data packet to a network security device to be tested, and receives a data packet returned by the network device to be tested, so as to implement functions such as performance test, function test, and network connectivity test. However, the network data packet generator in the prior art has single function generally, and the conventional network data packet generator also has the problems of slow data packet generation speed, interruption and blockage of data transmission due to large data volume of the simulated industrial Ethernet.
In addition, most of the network data packet generating devices used in China are based on foreign hardware platforms and developed by foreign engineers, and the data security and technical availability of the network data packet generating devices cannot be guaranteed in view of the current international situation. With the continuous development of the localization technology, the network data packet generating device based on the localization platform needs to be studied in depth in the future. Therefore, a device based on a localization platform and capable of fully covering and simulating the generation of the characteristic data of the industrial Ethernet is needed, and a basic support is provided for the development of the safety technology of the industrial Ethernet and the detection of the performance of the industrial Ethernet.
Disclosure of Invention
In view of the above, the present invention provides a method and apparatus for generating an industrial ethernet packet based on a localization platform, which is used to overcome or at least partially solve the above problems. The method can cover continuous generation of typical industrial Ethernet protocol characteristic data packets, provides direct support for industrial Ethernet flow analysis and research, industrial Ethernet security analysis and test verification, and has good application prospect.
The invention provides the following scheme:
an industrial Ethernet data packet generation method based on a localization platform comprises the following steps:
acquiring a plurality of data files stored in a file system, wherein each data file comprises industrial Ethernet data to be simulated of a data frame type;
storing the data frame loads contained in a plurality of data files in a plurality of annular buffer areas in a one-to-one correspondence manner;
carrying out one-to-one serialization on the data frame loads in a plurality of annular buffer areas by adopting a multi-core parallel serialization model to obtain a plurality of serialization records, and pushing the plurality of serialization records into a multi-producer multi-consumer lock-free queue; the multi-core parallel serialization model comprises a plurality of first parallel working threads, and the serialization record comprises a target serialization format;
taking out a plurality of serialized records in the multi-producer multi-consumer lock-free queue by adopting a parallel transmission model, and transmitting the serialized records outwards through a network card by adopting a direct memory access mode; the parallel transmission model comprises a plurality of second parallel working threads.
Preferably: the target serialization format comprises a length domain, a timestamp domain and a data payload domain which are sequentially arranged from left to right; the length field is used for representing the length of a data packet load, the timestamp field is used for representing the future sending time of the data packet, and the data payload field is used for representing the data load directly sent when the data packet is sent.
Preferably: the content stored in the data file comprises all data frame loads above a physical layer in an OSI seven-layer protocol in the industrial Ethernet data to be simulated of the same data frame type.
Preferably: and creating and obtaining a plurality of ring buffers by adopting a data creation thread, wherein the number of the ring buffers is the same as the number of the data files, the number of nodes contained in each ring buffer is the same as the number of data contained in each data file, and the contents of the nodes comprise data load contents, data load lengths and data sending speeds.
Preferably: and each first parallel working thread sequentially takes out the data frame load in the annular buffer area according to the serial sequence of the annular buffer area linked list and at the consumption speed which is multiple times of that of the multi-producer multi-consumer lock-free queue, and the serialization record is obtained.
Preferably: the single-thread working method of the multi-core parallel serialization model comprises the following steps:
after the multi-producer multi-consumer lock-free queue empty semaphore is determined, data load content, data load length and data sending speed related information are taken out from the corresponding annular buffer zone;
and allocating memory from a memory pool to create the serialization record, and filling the data length into the serialization record, wherein the memory pool is created and obtained by the data creation thread.
Preferably: determining the serialization record as a first serialization record generated by the first parallel working thread;
acquiring a system timestamp counter value, calculating a first sending timestamp of the first serialization record, and storing the first sending timestamp into a global variable as a datum reference point for generating all the serialization records subsequently;
calculating and obtaining the future transmission time of the first data frame of the serialization record according to the future transmission time stamp of the first data frame of the serialization record, the system time stamp counter value and the first transmission time stamp, and filling the future transmission time of the data frame into the serialization record;
copying data frame load content to the serialization record;
pushing the serialized records into a multi-producer multi-consumer lock-free queue.
Preferably: the parallel sending model is adopted to take out a plurality of serialized records in the lock-free queues of multiple producers and multiple consumers, and the serialized records are sent outwards through a network card in a direct memory access mode, and the method comprises the following steps:
determining the current transmission as the first transmission;
recording a second sending time stamp of the data, and taking the second sending time stamp and the system time stamp counter value as a reference point of all subsequent single-thread sending data.
Preferably: the file system is arranged in a Feiteng platform based on localization, and the network card is a DMA network card based on the Feiteng platform.
An industrial ethernet data packet generating device based on a localization platform, comprising:
a data file acquisition unit for acquiring a plurality of data files stored in a file system, wherein each data file comprises industrial Ethernet data to be simulated of a data frame type;
the buffer unit is used for storing the data frame loads contained in the data files in a plurality of annular buffer areas in a one-to-one correspondence manner; creating and obtaining a plurality of annular buffer areas by adopting a data creation thread, wherein the number of the annular buffer areas is the same as the number of the data files, the number of nodes contained in each annular buffer area is the same as the number of data contained in each data file, and the contents of the nodes comprise data load contents, data load lengths and data sending speeds;
the serialization recording unit is used for serializing the data frame loads in the plurality of annular buffer areas one by adopting a multi-core parallel serialization model to obtain a plurality of serialization records, and pushing the plurality of serialization records into a multi-producer multi-consumer lock-free queue; the multi-core parallel serialization model comprises a plurality of first parallel working threads, the serialization records comprise a target serialization format, and each first parallel working thread sequentially takes out data frame loads in the annular buffer area according to the serial sequence of the annular buffer area linked list at the consumption speed which is multiple times that of the multi-producer multi-consumer lock-free queue to obtain the serialization records; the single-thread working method of the multi-core parallel serialization model comprises the following steps:
after the multi-producer multi-consumer lock-free queue empty semaphore is determined, load content, data load length and data sending speed related information are taken out from the corresponding annular buffer zone;
allocating memory from a memory pool to create the serialization record and filling the data length into the serialization record, wherein the memory pool is created and obtained by the data creation thread;
the sending unit is used for taking out a plurality of serialized records in the multi-producer multi-consumer lock-free queue by adopting a parallel sending model and sending the serialized records outwards through a network card by adopting a direct memory access mode; the parallel transmission model comprises a plurality of second parallel working threads. According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the industrial Ethernet data packet generation method and device based on the domestic platform can cover continuous generation of typical industrial Ethernet protocol characteristic data packets, provide direct support for industrial Ethernet flow analysis and research, industrial Ethernet safety analysis and test verification, and have good application prospects. Meanwhile, the Time Stamp Counter (TSC) is used for high-resolution time measurement to solve the problem that the operation system cannot accurately transmit according to a record given time point due to low timing precision.
The serialization thread uses the first sending timestamp Tg generated by combining TSCs as a time reference of the subsequent production record, and all sending data records are generated by the same time reference across a plurality of production threads, so that the problem of consistency of record time generated across the plurality of threads is solved.
The TSC is used in the single-thread transmission in combination with the first transmission time stamp T0 as a subsequent transmission record time reference, and the transmission time point is expressed by the relative quantity, so that the problem of synchronization of all transmission record times in the single-thread transmission is solved, meanwhile, the problem that the conventional flow generator cannot continuously transmit the network flow with variable speed is solved, and the method has higher application value and covers wider industrial Ethernet real network flow application scenes.
The data packet generator can cover the application scene generated by the full protocol, full packet length, full data content, full data frame mixing proportion and full traffic size custom industrial Ethernet data frame based on the Ethernet by storing the generated data, the transmission traffic size and the transmission rate in the file system based on the domestic platform and adopting a mode that the data content is fully opened to be editable.
Of course, it is not necessary for any one product to practice the invention to achieve all of the advantages set forth above at the same time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments will be briefly described below. It is evident that the drawings in the following description are only some embodiments of the present invention and that other drawings may be obtained from these drawings by those of ordinary skill in the art without inventive effort.
Fig. 1 is a flowchart of an industrial ethernet packet generation method based on a localization platform according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a data generator architecture according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a serialized recording format according to an embodiment of the present invention;
FIG. 4 is a single-threaded workflow diagram of a multi-core parallel serialization model provided by an embodiment of the present invention;
FIG. 5 is a flow chart of a parallel send model workflow provided by an embodiment of the present invention;
FIG. 6 is a schematic diagram of a Feiteng hardware platform provided by an embodiment of the present invention;
fig. 7 is a schematic diagram of an industrial ethernet packet generating device based on a localization platform according to an embodiment of the present invention;
fig. 8 is a schematic diagram of an industrial ethernet packet generating device based on a localization platform according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which are derived by a person skilled in the art based on the embodiments of the invention, fall within the scope of protection of the invention.
Referring to fig. 1, a method for generating an industrial ethernet packet based on a localization platform according to an embodiment of the present invention, as shown in fig. 1, may include:
s101: acquiring a plurality of data files stored in a file system, wherein each data file comprises industrial Ethernet data to be simulated of a data frame type; specifically, the content stored in the data file includes all data frame loads above a physical layer in an OSI seven-layer protocol in the industrial ethernet data to be simulated of the same data frame type.
S102: storing the data frame loads contained in a plurality of data files in a plurality of annular buffer areas in a one-to-one correspondence manner; specifically, a plurality of ring buffers are created and obtained by adopting a data creation thread, the number of the ring buffers is the same as the number of the data files, the number of nodes contained in each ring buffer is the same as the number of data contained in each data file, and the contents of the nodes comprise data load contents, data load lengths and data sending speeds.
S103: carrying out one-to-one serialization on the data frame loads in a plurality of annular buffer areas by adopting a multi-core parallel serialization model to obtain a plurality of serialization records, and pushing the plurality of serialization records into a multi-producer multi-consumer lock-free queue; the multi-core parallel serialization model comprises a plurality of first parallel working threads, and the serialization record comprises a target serialization format; specifically, the target serialization format includes a length field, a timestamp field and a data payload field which are sequentially arranged from left to right; the length field is used for representing the length of a data packet load, the timestamp field is used for representing the future sending time of the data packet, and the data payload field is used for representing the data load directly sent when the data packet is sent.
In practical application, each first parallel working thread sequentially takes out the data frame load in the annular buffer area according to the serial sequence of the annular buffer area linked list and at the consumption speed which is multiple times of that of the multi-producer multi-consumer lock-free queue, and serialization records are obtained.
The single-thread working method of the multi-core parallel serialization model comprises the following steps:
after the multi-producer multi-consumer lock-free queue empty semaphore is determined, data load content, data load length and data sending speed related information are taken out from the corresponding annular buffer zone;
and allocating memory from a memory pool to create the serialization record, and filling the data length into the serialization record, wherein the memory pool is created and obtained by the data creation thread.
Determining the serialization record as a first serialization record generated by the first parallel working thread;
acquiring a system timestamp counter value, calculating a first sending timestamp of the first serialization record, and storing the first sending timestamp into a global variable as a datum reference point for generating all the serialization records subsequently;
calculating and obtaining the future transmission time of the first data frame of the serialization record according to the future transmission time stamp of the first data frame of the serialization record, the system time stamp counter value and the first transmission time stamp, and filling the future transmission time of the data frame into the serialization record;
copying data frame load content to the serialization record;
pushing the serialized records into a multi-producer multi-consumer lock-free queue.
S104: taking out a plurality of serialized records in the multi-producer multi-consumer lock-free queue by adopting a parallel transmission model, and transmitting the serialized records outwards through a network card by adopting a direct memory access mode; the parallel transmission model comprises a plurality of second parallel working threads. Specifically, determining the current transmission as the first transmission;
recording a second sending time stamp of the data, and taking the second sending time stamp and the system time stamp counter value as a reference point of all subsequent single-thread sending data.
In order to enable the industrial ethernet packet generating method based on the localization platform provided in the embodiments of the present application to be used based on localization software, the embodiments of the present application may also provide that the file system is arranged in a localization-based flying platform, and the network card is a DMA network card based on the flying platform.
According to the industrial Ethernet data packet generation method based on the domestic platform, the industrial Ethernet characteristic data can be generated in a full-coverage simulation mode, the safety equipment is not required to be directly connected into the operated industrial Ethernet during the test equipment, the data packet for simulating the related industrial Ethernet can be rapidly generated by the method provided by the embodiment of the application, and the high-simulation test on the network safety equipment is realized on the premise that the normal instantaneity and the time sequence of the operated industrial Ethernet are not damaged. Providing basic support for industrial Ethernet safety technology development and industrial Ethernet performance detection.
It can be understood that the industrial ethernet packet generating method based on the localization platform provided in the embodiments of the present application may be used in a network packet generator (also called a network tester, which may be abbreviated as a packet generator) to implement functions such as performance test, functional test, and network connectivity test by generating a specific network packet, sending the network packet to a network device to be tested, and receiving a packet returned by the network device to be tested. The data packet generator plays an important role in network research, development, operation and maintenance and the like. Network researchers can utilize a data packet generator to test the performance and the function of a network prototype system and find the system bottleneck; network operators can use the data packets to debug network faults and detect network service quality. The data packet generator is an indispensable network tool, and has important social and economic benefits and academic research value.
The following describes in detail a method provided by the embodiment of the present application based on the Feiteng platform.
The embodiment of the application can adopt a fully domestic Feiteng hardware platform design, as shown in fig. 6, the FT-2000/4 general computing processor chip adopted by the hardware platform integrates 4 Feiteng independently developed processor cores FTC663, is compatible with a 64-bit ARMv8 instruction set, is in a 16nm process, has the highest main frequency of 3.0GHz, is provided with two 72-bit (ECC-supporting) DDR4-3200 storage interfaces, is provided with two paths of 16 lanes (each of which is divided into 2 x 8), is provided with two paths of 1lane PCIE3.0, and is provided with at most 6 PCIE3.0 interfaces and two gigabit Ethernet interfaces. FCBGA package, pin count 1144, package size 35mmx35mm, maximum power consumption 10W.
Fig. 2 shows the architecture of a data packet generator, where the generator is composed of a data production module, a multi-core parallel serialization model and a multi-core parallel serialization model, and these four components are described in detail below.
The data production module consists of a data file, a data creation thread and a ring buffer.
The data files are stored in the data files of the file system based on the domestic Feiteng platform, each data file represents an industrial Ethernet data frame type, and the stored content of the data files is all data frame loads above a physical layer in an OSI seven-layer protocol. When different types of industrial Ethernet data need to be generated, the data of different frame types are respectively stored in different files. When it is necessary to generate industrial Ethernet data frames of the same type, different lengths and different contents, the data frame loads are stored in the same file.
The data creation thread is responsible for creating a memory pool for the MPMC queue to allocate memory when the generator is initialized, and the design can greatly accelerate the creation speed of the MPMC queue node. While creating a ring buffer, reading the data file contents and copying them to the ring buffer.
The ring buffer stores a data frame payload extracted from a data file, consisting of data blocks based on a doubly-circular linked list. The ring buffers are created by the data creation threads, the number depends on the number of data files, the number of nodes of each ring buffer depends on the number of data files, and the node content comprises data load content, data load length and data sending speed.
The multi-core parallel serialization model consists of a plurality of first parallel working threads, wherein the number of the first parallel working threads depends on the number of the annular buffer areas. Each thread takes out the data in the serial order of the ring buffer linked list and at the consumption speed of the MPMC lock-free queue in turn, then lists the data as records according to the target serialization format shown in fig. 3, and pushes the records into the MPMC lock-free queue. The data is fetched at the speed which is multiple times of that of the sending thread and is serialized into the record, so that the production rate of the MPMC lock-free queue is ensured to be always larger than the consumption rate, and the sending data is not interrupted or blocked.
The target serialization format comprises a length domain, a timestamp domain and a data payload domain which are arranged in sequence from left to right.
length domain: 2 bytes, representing the length of the packet payload.
timestamp domain: 8 bytes, the high four bytes are seconds, the low four bytes are microseconds, and the method is compatible with a struct time structure body under a linux system and represents the future sending moment of a data packet.
data payload field, which is the actual data load, and data directly transmitted when the data packet is transmitted.
The single-thread workflow of the multi-core parallel serialization model is shown in fig. 4:
and step 1, clearing the length of the global variable MPMC queue, waiting for the completion of initialization of a sending thread, informing the MPMC non-lock queue of being empty through the signal quantity, entering step 2 after receiving the empty signal quantity of the MPMC non-lock queue, and otherwise, continuing to execute step 1.
And 2, taking out the data, the data length and the sending speed related information from the corresponding annular buffer area, distributing the memory from the memory pool to create a serialization record, and filling the data length into the serialization record.
And 3, detecting whether the record is a first record generated by all the serialization threads, if so, executing the step 3-1, then executing the step 3-2, and if not, directly executing the step 3-2.
Step 3-1, acquiring a system timestamp counter value TSC (Ch) of the timestamp counter, calculating a first transmission timestamp Tg (first transmission timestamp) and storing the first transmission timestamp Tg into a global variable as a base reference point for generating all the serialization records subsequently.
Step 3-2, calculating the future transmission time of the data frame according to the future transmission time stamp Tn of the data frame, the system time stamp counter value TSC (Ch) and the first transmission time stamp Tg, and filling the transmission time into the serialization record.
And 4, copying the data frame load content to the serialization record.
And 5, pushing the serialized records into the MPMC lock-free queue.
And 6, detecting whether the number of the queues created at the time reaches the maximum number of the queues, returning to the step 1 if the number of the queues reaches the maximum number of the queues, and returning to the step 2 if the number of the queues does not reach the maximum number of the queues.
A linked list based lock-free queue created by the serialization thread, each node of the queue containing a serialization record created by the serialization thread. The number of lock-free queues is determined by the number of threads in the serialization, and the maximum length of the queues is configurable. Consumers of the lock-free data structure are sending threads, which are fetched and released by the sending threads.
The multi-core parallel serialization model consists of a plurality of second parallel working threads, is responsible for taking out data in an MPMC queue and sending the data out through a Feiteng platform network card in a Direct Memory Access (DMA) mode, and is a consumer of a lock-free data structure. The workflow of the multi-core parallel serialization model is shown in fig. 5:
and 1, clearing the number of the sending records of the round, sending a signal quantity that the lock-free queue is empty to the serialization thread, and informing the serialization thread to push the data to be sent to the MPMC lock-free queue.
And 2, detecting whether the MPMC lock-free queue is empty, if not, executing the step 3, and if so, sleeping for 10ms, and continuously executing the step 2.
And 3, taking out the data in the head of the MPMC lock-free queue, and increasing the transmission frame count of the round by 1.
And 4, detecting whether the nodes in the MPMC lock-free queue to be transmitted in the round are about to be exhausted, wherein the exhaustion condition is that the transmission record count in the round reaches the maximum queue length number. If the current state is exhausted, the step 4-1 is executed, and if the current state is not exhausted, the step 5 is executed.
And 4-1, clearing the transmission record count of the round, and transmitting a signal quantity that the lock-free queue is empty to the serialization thread to inform the serialization thread to push the data to be transmitted in the next round to the MPMC lock-free queue.
And 5, extracting the data load to be transmitted and the future transmission time stamp Tn of the data from the transmission record, and releasing the memory used by the data record to a memory pool.
And 6, detecting whether the transmission is the first transmission after the data generator is started, if so, executing the step 6-1, and if not, executing the step 7.
In step 6-1, the data transmission time stamp is recorded as T0 (second transmission time stamp), and the system time stamp counter value TSC (Ch) is acquired and recorded, wherein the two values are used as the datum reference point of all subsequent single-thread transmission data.
And 7, sending a time stamp Tn in the future according to the data extracted in the step 5, calculating a theoretical sending time point TSC (Cd) =Tn-T0 by taking the T0 as a datum reference point, and acquiring a current host time stamp counter value TSC (Cn).
if Cn > Ch+Cd, executing step 8-1;
if cn= > ch+cd, step 8-2 is performed;
if cn=ch+cd, step 8-3 is performed.
Step 8-1, not reaching the transmission time point, jumping to step 7 and waiting for Cn= > Ch+Cd.
And 8-2, beyond the sending time point, the data packet is illegal, the frame is discarded, and the step 3 is skipped.
And 8-3, transmitting the data frame to the DMA network card based on the Feiteng platform after the time point is reached.
Therefore, according to the method provided by the embodiment of the application, the problem that the transmission cannot be accurately performed according to the recorded given time point caused by low timing precision of the operating system is solved by using the Time Stamp Counter (TSC) to perform high-resolution time measurement.
The TSC is used in the single-thread transmission in combination with the first transmission time stamp T0 as a subsequent transmission record time reference, and the transmission time point is expressed by the relative quantity, so that the problem of synchronization of all transmission record times in the single-thread transmission is solved, meanwhile, the problem that the conventional flow generator cannot continuously transmit the network flow with variable speed is solved, and the method has higher application value and covers wider industrial Ethernet real network flow application scenes.
The serialization thread uses the first sending timestamp Tg generated by combining TSCs as a time reference of the subsequent production record, and all sending data records are generated by the same time reference across a plurality of production threads, so that the problem of consistency of record time generated across the plurality of threads is solved.
The data packet generator can cover the application scene generated by the full protocol, full packet length, full data content, full data frame mixing proportion and full traffic size custom industrial Ethernet data frame based on the Ethernet by storing the generated data, the transmission traffic size and the transmission rate in the file system based on the domestic platform and adopting a mode that the data content is fully opened to be editable.
Referring to fig. 7, the embodiment of the present application may further provide an industrial ethernet packet generating device based on a localization platform, as shown in fig. 7, the device may include:
a data file obtaining unit 701, configured to obtain a plurality of data files stored in a file system, where each data file includes industrial ethernet data to be simulated of a data frame type;
a buffer unit 702, configured to store data frame loads contained in a plurality of data files in a plurality of ring buffers in a one-to-one correspondence; creating and obtaining a plurality of annular buffer areas by adopting a data creation thread, wherein the number of the annular buffer areas is the same as the number of the data files, the number of nodes contained in each annular buffer area is the same as the number of data contained in each data file, and the contents of the nodes comprise data load contents, data load lengths and data sending speeds;
the serialization recording unit 703 is configured to perform serialization on the data frame loads in the plurality of ring buffers one by using a multi-core parallel serialization model to obtain a plurality of serialization records, and push the plurality of serialization records into a multi-producer multi-consumer lock-free queue; the multi-core parallel serialization model comprises a plurality of first parallel working threads, the serialization records comprise a target serialization format, and each first parallel working thread sequentially takes out data frame loads in the annular buffer area according to the serial sequence of the annular buffer area linked list at the consumption speed which is multiple times that of the multi-producer multi-consumer lock-free queue to obtain the serialization records; the single-thread working method of the multi-core parallel serialization model comprises the following steps:
after the multi-producer multi-consumer lock-free queue empty semaphore is determined, load content, data load length and data sending speed related information are taken out from the corresponding annular buffer zone;
allocating memory from a memory pool to create the serialization record and filling the data length into the serialization record, wherein the memory pool is created and obtained by the data creation thread;
a sending unit 704, configured to take out a plurality of the serialized records in the lock-free queue of multiple producers and multiple consumers by using a parallel sending model, and send the serialized records outwards through a network card by using a direct memory access manner; the parallel transmission model comprises a plurality of second parallel working threads.
The embodiment of the application can also provide industrial Ethernet data packet generating equipment based on the localization platform, and the equipment comprises a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is used for executing the steps of the industrial Ethernet data packet generation method based on the localization platform according to the instructions in the program codes.
As shown in fig. 8, an industrial ethernet packet generating device based on a localization platform according to an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all complete communication with each other through a communication bus 13.
In the present embodiment, the processor 10 may be a central processing unit (Central Processing Unit, CPU), an asic, a dsp, a field programmable gate array, or other programmable logic device, etc.
The processor 10 may call a program stored in the memory 11, and in particular, the processor 10 may perform operations in an embodiment of an industrial ethernet packet generation method based on a localization platform.
The memory 11 is used for storing one or more programs, and the programs may include program codes, where the program codes include computer operation instructions, and in this embodiment, at least the programs for implementing the following functions are stored in the memory 11:
acquiring a plurality of data files stored in a file system, wherein each data file comprises industrial Ethernet data to be simulated of a data frame type;
storing the data frame loads contained in a plurality of data files in a plurality of annular buffer areas in a one-to-one correspondence manner;
carrying out one-to-one serialization on the data frame loads in a plurality of annular buffer areas by adopting a multi-core parallel serialization model to obtain a plurality of serialization records, and pushing the plurality of serialization records into a multi-producer multi-consumer lock-free queue; the multi-core parallel serialization model comprises a plurality of first parallel working threads, and the serialization record comprises a target serialization format;
taking out a plurality of serialized records in the multi-producer multi-consumer lock-free queue by adopting a parallel transmission model, and transmitting the serialized records outwards through a network card by adopting a direct memory access mode; the parallel transmission model comprises a plurality of second parallel working threads.
In one possible implementation, the memory 11 may include a storage program area and a storage data area, where the storage program area may store an operating system, and application programs required for at least one function (such as a file creation function, a data read-write function), and the like; the store data area may store data created during use, such as initialization data, etc.
In addition, the memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid-state storage device.
The communication interface 12 may be an interface of a communication module for interfacing with other devices or systems.
Of course, it should be noted that the structure shown in fig. 8 does not limit the industrial ethernet packet generating device based on the localization platform in the embodiment of the present application, and in practical application, the industrial ethernet packet generating device based on the localization platform may include more or fewer components than those shown in fig. 8, or may combine some components.
Embodiments of the present application may also provide a computer readable storage medium storing program code for executing the steps of the above-described industrial ethernet packet generation method based on a localization platform.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
From the description of the embodiments above, it will be apparent to those skilled in the art that the present application may be implemented in software plus the necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in the embodiments or some parts of the embodiments of the present application.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for a system or system embodiment, since it is substantially similar to a method embodiment, the description is relatively simple, with reference to the description of the method embodiment being made in part. The systems and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (7)
1. The industrial Ethernet data packet generation method based on the localization platform is characterized by comprising the following steps:
acquiring a plurality of data files stored in a file system, wherein each data file comprises industrial Ethernet data to be simulated of a data frame type;
storing the data frame loads contained in a plurality of data files in a plurality of annular buffer areas in a one-to-one correspondence manner; creating and obtaining a plurality of annular buffer areas by adopting a data creation thread, wherein the number of the annular buffer areas is the same as the number of the data files, the number of nodes contained in each annular buffer area is the same as the number of data contained in each data file, and the contents of the nodes comprise data load contents, data load lengths and data sending speeds;
carrying out one-to-one serialization on the data frame loads in a plurality of annular buffer areas by adopting a multi-core parallel serialization model to obtain a plurality of serialization records, and pushing the plurality of serialization records into a multi-producer multi-consumer lock-free queue; the multi-core parallel serialization model comprises a plurality of first parallel working threads, and the serialization record comprises a target serialization format; each first parallel working thread sequentially takes out the data frame load in the annular buffer area according to the serial sequence of the annular buffer area linked list and at the consumption speed which is multiple times of that of the multi-producer multi-consumer lock-free queue, and sequences to obtain a serialized record; the single-thread working method of the multi-core parallel serialization model comprises the following steps:
after the multi-producer multi-consumer lock-free queue empty semaphore is determined, data load content, data load length and data sending speed related information are taken out from the corresponding annular buffer zone;
allocating memory from a memory pool to create the serialization record and filling the data length into the serialization record, wherein the memory pool is created and obtained by the data creation thread;
taking out a plurality of serialized records in the multi-producer multi-consumer lock-free queue by adopting a parallel transmission model, and transmitting the serialized records outwards through a network card by adopting a direct memory access mode; the parallel transmission model comprises a plurality of second parallel working threads.
2. The method for generating industrial ethernet packets based on a localization platform according to claim 1, wherein the target serialization format comprises a length field, a timestamp field, and a data payload field sequentially arranged from left to right; the length field is used for representing the length of a data packet load, the timestamp field is used for representing the future sending time of the data packet, and the data payload field is used for representing the data load directly sent when the data packet is sent.
3. The method for generating industrial ethernet packets based on a localization platform according to claim 1, wherein the content stored in the data file comprises all data frame loads above a physical layer in OSI seven layer protocols in the industrial ethernet data to be simulated of the same data frame type.
4. The method for generating industrial ethernet packets based on a localization platform as recited in claim 1, wherein determining the serialization record is a first serialization record generated by the first parallel working thread;
acquiring a system timestamp counter value, calculating a first sending timestamp of the first serialization record, and storing the first sending timestamp into a global variable as a datum reference point for generating all the serialization records subsequently;
calculating and obtaining the future transmission time of the first data frame of the serialization record according to the future transmission time stamp of the first data frame of the serialization record, the system time stamp counter value and the first transmission time stamp, and filling the future transmission time of the data frame into the serialization record;
copying data frame load content to the serialization record;
pushing the serialized records into a multi-producer multi-consumer lock-free queue.
5. The method for generating industrial ethernet packets based on a localization platform according to claim 4, wherein said taking out a plurality of said serialized records in said multi-producer multi-consumer lock-free queue using a parallel transmission model and transmitting the serialized records outwards through a network card using a direct memory access method comprises:
determining the current transmission as the first transmission;
recording a second sending time stamp of the data, and taking the second sending time stamp and the system time stamp counter value as a reference point of all subsequent single-thread sending data.
6. The method for generating industrial ethernet data packets based on a localization platform as claimed in claim 5, wherein the file system is arranged in a localization platform, and the network card is a DMA network card based on the localization platform.
7. An industrial ethernet data packet generating device based on a localization platform, which is characterized by comprising:
a data file acquisition unit for acquiring a plurality of data files stored in a file system, wherein each data file comprises industrial Ethernet data to be simulated of a data frame type;
the buffer unit is used for storing the data frame loads contained in the data files in a plurality of annular buffer areas in a one-to-one correspondence manner; creating and obtaining a plurality of annular buffer areas by adopting a data creation thread, wherein the number of the annular buffer areas is the same as the number of the data files, the number of nodes contained in each annular buffer area is the same as the number of data contained in each data file, and the contents of the nodes comprise data load contents, data load lengths and data sending speeds;
the serialization recording unit is used for serializing the data frame loads in the plurality of annular buffer areas one by adopting a multi-core parallel serialization model to obtain a plurality of serialization records, and pushing the plurality of serialization records into a multi-producer multi-consumer lock-free queue; the multi-core parallel serialization model comprises a plurality of first parallel working threads, the serialization records comprise a target serialization format, and each first parallel working thread sequentially takes out data frame loads in the annular buffer area according to the serial sequence of the annular buffer area linked list at the consumption speed which is multiple times that of the multi-producer multi-consumer lock-free queue to obtain the serialization records; the single-thread working method of the multi-core parallel serialization model comprises the following steps:
after the multi-producer multi-consumer lock-free queue empty semaphore is determined, load content, data load length and data sending speed related information are taken out from the corresponding annular buffer zone;
allocating memory from a memory pool to create the serialization record and filling the data length into the serialization record, wherein the memory pool is created and obtained by the data creation thread;
the sending unit is used for taking out a plurality of serialized records in the multi-producer multi-consumer lock-free queue by adopting a parallel sending model and sending the serialized records outwards through a network card by adopting a direct memory access mode; the parallel transmission model comprises a plurality of second parallel working threads.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310354423.2A CN116095195B (en) | 2023-04-06 | 2023-04-06 | Industrial Ethernet data packet generation method and device based on domestic platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310354423.2A CN116095195B (en) | 2023-04-06 | 2023-04-06 | Industrial Ethernet data packet generation method and device based on domestic platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116095195A CN116095195A (en) | 2023-05-09 |
| CN116095195B true CN116095195B (en) | 2023-06-20 |
Family
ID=86202908
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310354423.2A Active CN116095195B (en) | 2023-04-06 | 2023-04-06 | Industrial Ethernet data packet generation method and device based on domestic platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116095195B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114050979A (en) * | 2021-11-19 | 2022-02-15 | 成都卓源网络科技有限公司 | Industrial control protocol safety test system and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102531645B1 (en) * | 2020-02-24 | 2023-05-11 | 주식회사 마키나락스 | Computer program for performance testing of models |
| CN111488219B (en) * | 2020-04-07 | 2022-08-02 | 中国科学院自动化研究所 | Ethernet data flow recording method for high-speed data acquisition system |
-
2023
- 2023-04-06 CN CN202310354423.2A patent/CN116095195B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114050979A (en) * | 2021-11-19 | 2022-02-15 | 成都卓源网络科技有限公司 | Industrial control protocol safety test system and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116095195A (en) | 2023-05-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2016090908A1 (en) | Memory simulation testing board system and testing method for embedded application of intelligent electric meter | |
| CN102946616B (en) | Internet of things middleware performance testing system and testing method | |
| CN104615477A (en) | Cycle-accurate replay and debugging of running fpga systems | |
| CN103441861B (en) | A kind of data record generation method and device | |
| CN111600760B (en) | Quality detection system, method and equipment of power grid operation monitoring system | |
| CN103198001B (en) | Storage system capable of self-testing peripheral component interface express (PCIE) interface and test method | |
| CN105763394B (en) | A kind of network link detection method, device and system | |
| CN103593271A (en) | Method and device for chip tracking debugging of system on chip | |
| CN103188116A (en) | Network performance detection method and system of blade server | |
| CN102929769A (en) | Virtual machine internal-data acquisition method based on agency service | |
| CN105528546B (en) | Vulnerability mining method and device and electronic equipment | |
| CN110632461A (en) | Test method, test terminal and test system for synchrophasor data concentrator | |
| CN105162649A (en) | Testing method and system for nuclear power station safety level network performance index | |
| CN105243037A (en) | FPGA based multi-channel data recording and playback method | |
| CN104731702B (en) | Test system and its service end | |
| CN108897695A (en) | A kind of the interconnection test method and system of demand side apparatus | |
| CN116095195B (en) | Industrial Ethernet data packet generation method and device based on domestic platform | |
| CN109905261A (en) | Method for diagnosing faults and device | |
| CN101572713A (en) | Method for detecting worm and system thereof | |
| CN104780123B (en) | A kind of network pack receiving and transmitting processing unit and its design method | |
| CN106776329A (en) | The adjustment method and commissioning device of energy accumulation current converter | |
| CN106788842A (en) | The processing method and SOC of a kind of PTP messages | |
| CN204392274U (en) | A kind of intelligent substation test system | |
| CN109412939A (en) | Communication gateway for recording industrial network communication period process data and working method | |
| CN103457996B (en) | A kind of Protocol Conformance Test System internal data store method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |