CN117762556A - Low false alarm rate data automatic matching method for distributed simulation DDS bus - Google Patents
Low false alarm rate data automatic matching method for distributed simulation DDS bus Download PDFInfo
- Publication number
- CN117762556A CN117762556A CN202311561319.7A CN202311561319A CN117762556A CN 117762556 A CN117762556 A CN 117762556A CN 202311561319 A CN202311561319 A CN 202311561319A CN 117762556 A CN117762556 A CN 117762556A
- Authority
- CN
- China
- Prior art keywords
- data
- vector
- simulation
- false alarm
- bit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 70
- 238000000034 method Methods 0.000 title claims abstract description 33
- 239000013598 vector Substances 0.000 claims abstract description 140
- 238000004891 communication Methods 0.000 claims abstract description 15
- 238000013507 mapping Methods 0.000 claims abstract description 11
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 9
- 238000007906 compression Methods 0.000 claims description 7
- 230000006835 compression Effects 0.000 claims description 7
- 230000008569 process Effects 0.000 abstract description 10
- 230000005540 biological transmission Effects 0.000 abstract description 8
- 230000007246 mechanism Effects 0.000 abstract description 3
- 230000006872 improvement Effects 0.000 description 7
- 239000003550 marker Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000013144 data compression Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a low false alarm rate data automatic matching method for a distributed simulation DDS bus, which comprises the steps of adopting an FBF marked bloom filter to store the description information of all participant endpoints in each simulation node, compressing and mapping the description information into the FBF marked bloom filter consisting of a data vector DV and a marked vector FV through hash operation, modulo operation and marking operation, and acquiring the information in the FBF marked bloom filter through a corresponding query algorithm, namely the invention is realized by adopting the SDP_FBF based on the marked bloom filter and a data automatic matching algorithm of a simple discovery mechanism SDP, and can ensure the data matching of the low false alarm rate in the data subscription process while reducing the data transmission quantity among the simulation nodes, thereby ensuring the real-time performance of the data communication among the nodes and improving the simulation efficiency.
Description
Technical Field
The invention relates to the technical field of data distribution service in large-scale distributed simulation, in particular to a low false alarm rate data automatic matching method for a distributed simulation DDS bus.
Background
Distributed simulation is widely applied to the fields of military, traffic, electronic systems, medical treatment and the like. The method can divide a huge simulation calculation task aiming at a complex system into a plurality of small tasks, and the small tasks are simultaneously executed by a plurality of simulation calculation nodes. In order to ensure smooth progress of simulation tasks, good data communication between the various simulated computing nodes is necessary. However, as the simulation object becomes more complex, the simulation scale is continuously enlarged, the number of simulation computers required in the distributed simulation environment is also continuously increased, and the amount of data transmitted between each computing node in the simulation process is more huge, which provides challenges for the data communication capability in the distributed simulation environment.
Currently, data distribution services (Data Distribution Service, DDS) have become one of the main solutions to the data communication problem in distributed emulation applications. The DDS is a data center publish/subscribe communication model specification proposed by an object management organization (Object Management Group, OMG), and the data communication mechanism adopted by the DDS is implemented based on a simple discovery protocol (Simple Discovery Protocol, SDP). In the simulation process, each simulation computing node transmits all data of the simulation computing node to other nodes, and simultaneously receives all data transmitted by the nodes. However, when the system scale is large and more data needs to be frequently exchanged, the communication efficiency is greatly affected by the extremely high network data transmission quantity, so the invention provides a low false alarm rate data automatic matching method for a distributed simulation DDS bus to solve the problems in the prior art.
Disclosure of Invention
In view of the above problems, the present invention aims to provide a method for automatically matching low false alarm rate data of a distributed simulation DDS bus, which has the advantages of reducing network transmission capacity and improving data communication efficiency between simulation nodes in large-scale distributed simulation application, and solves the problems existing in the prior art.
In order to achieve the purpose of the invention, the invention is realized by the following technical scheme: the low false alarm rate data automatic matching method for the distributed simulation DDS bus comprises the following steps:
step one: when data communication is carried out among a plurality of groups of simulation nodes, each simulation node respectively passes through an FBF (fiber-film) mark bloom filter, the description information of all participant endpoints contained in the simulation nodes is compressed and mapped into two vectors which are respectively a data vector DV and a mark vector FV, and the initial value of each bit of the data vector DV and the mark vector FV is set to be 0;
step two: the information compression mapping is to describe information of each participant endpoint, so that the description information carries out hash operation for k times through k hash functions respectively, then the k hash operation results are limited between 0 and (m-1) through modulo operation, the modulo operation results are mapped to corresponding bits in a data vector DV, and then the corresponding bits in the obtained data vector DV are in one-to-one correspondence with bits in a flag vector FV;
step three: each simulation node mutually transmits a data vector DV and a marking vector FV, any simulation node searches the two vectors in the first step through a query algorithm, and if the two vectors contain the needed participant endpoint description information, the simulation nodes corresponding to the simulation nodes are subjected to data subscription, so that data communication is realized;
step four: the simulation node searches the data vector DV and the marking vector FV through a query algorithm, when the simulation node contains the needed participant endpoint description information, the corresponding simulation node is subjected to data subscription, the topic name of certain data to be subscribed by the simulation node is mapped into the query vector through the same k hash functions and modulo operation, then the query vector and all the data vectors DV sent by other simulation nodes are respectively subjected to dot product, and subsequent subscription operation is carried out according to the obtained dot product value.
The further improvement is that: in the first step, the data vector DV is a one-dimensional vector having m bits, and the flag vector FV is a multidimensional vector having m bits, and the dimension of each bit is determined by the flag value corresponding to the bit.
The further improvement is that: in the second step, a certain bit in the data vector DV is mapped by the i-th element in the data set, and the corresponding bit in the flag vector FV is set to i.
The further improvement is that: in the second step, when a certain bit in the data vector DV is mapped multiple times by multiple elements, the corresponding bit in the flag vector FV is restored to 0.
The further improvement is that: in the second step, the hash operation is specifically that the data element sets to be compressed are mapped into machine words through hash functions respectively.
The further improvement is that: in the second step, the modulo operation is to modulo the machine word to the size of the data vector DV.
The further improvement is that: in the fourth step, if the obtained dot product value is equal to k and the value of the corresponding bit in the flag vector FV is not more than one, the corresponding simulation node is considered to be able to provide the data, so as to subscribe.
The further improvement is that: in the fourth step, the query vector is also a one-dimensional vector having m bits.
The beneficial effects of the invention are as follows:
(1) When the SDP_FBF data automatic matching method provided by the invention adopts the mark bloom filter to store the description information of all participant endpoints in each simulation node, a plurality of hash operations and corresponding modulo operations are used for compressing and mapping a large amount of data into one-dimensional data vector and one multi-dimensional mark vector for transmission, inquiry and matching subscription, so that the problem that the SDP of the existing simple discovery mechanism needs to transmit all data is solved, and the network transmission quantity is effectively reduced.
(2) The SDP_FBF data automatic matching method provided by the invention adopts the marking vector to enrich the compressed data information, and can effectively reduce the false positive rate while keeping the true positive rate to be 100%, thereby reducing the transmission of invalid data and further reducing the network transmission quantity.
Drawings
Fig. 1 is a schematic diagram of an automatic SDP-FBF based data matching process of the present invention.
Fig. 2 is a schematic diagram of the FBF marker bloom filter of the present invention compression storing 3 data information elements.
Fig. 3 is a schematic diagram of the data query process of the present invention based on the FBF flag bloom filter.
Description of the embodiments
The present invention will be further described in detail with reference to the following examples, which are only for the purpose of illustrating the invention and are not to be construed as limiting the scope of the invention.
According to the first embodiment, as shown in fig. 1, the present embodiment proposes a method for automatically matching low false alarm rate data of a distributed simulation DDS bus, including the following steps:
step one: when data communication is carried out among a plurality of groups of simulation nodes, each simulation node respectively passes through an FBF (fiber-film-filter) mark bloom filter, the description information of all participant endpoints contained in the simulation nodes is compressed and mapped into two vectors which are respectively a data vector DV and a mark vector FV, and the initial value of each bit of the data vector DV and the mark vector FV is set to be 0, wherein the data vector DV is a one-dimensional vector with m bits, the mark vector FV is a multidimensional vector with m bits, and the dimension of each bit is determined by a mark value corresponding to the bit;
step two: the information compression mapping is to describe information of each participant endpoint, so that the description information carries out hash operation for k times through k hash functions respectively, then the k hash operation results are limited between 0 and (m-1) through modulo operation, the modulo operation results are mapped to corresponding bits in a data vector DV, then the corresponding bits in the obtained data vector DV are in one-to-one correspondence with bits in a flag vector FV, wherein a certain bit in the data vector DV is mapped by an ith element in a data set, the corresponding bit in the flag vector FV is set as i, a certain bit in the data vector DV is mapped for a plurality of times by a plurality of elements, and the corresponding bit in the flag vector FV is restored to 0;
the hash operation is to map the data element set to be compressed into machine words through hash functions respectively, and the modulo operation is to modulo the size of the data vector DV by the machine words respectively.
Step three: each simulation node mutually transmits a data vector DV and a marking vector FV, any simulation node searches the two vectors in the first step through a query algorithm, and if the two vectors contain the needed participant endpoint description information, the simulation nodes corresponding to the simulation nodes are subjected to data subscription, so that data communication is realized;
step four: the simulation node searches the data vector DV and the mark vector FV through a query algorithm, when the simulation node contains the needed participant endpoint description information, the corresponding simulation node is subjected to data subscription, the topic name of certain data to be subscribed by the simulation node is mapped into the query vector through the same k hash functions and modulo arithmetic, the query vector and all data vectors DV sent by other simulation nodes are respectively subjected to dot product, the subsequent subscription operation is carried out according to the obtained dot product value, wherein the query vector is also a one-dimensional vector with m bits, and if the obtained dot product value is equal to k and the value of the corresponding bit in the mark vector FV is not more than one, the corresponding simulation node can be considered to provide the data, so that the subscription is carried out.
In a second embodiment, as shown in fig. 1 to 3, the present embodiment proposes a low false alarm rate data automatic matching method for a distributed simulation DDS bus, and the present invention is described in detail from three parts, namely an sdp_fbf data automatic matching process, FBF-based data compression storage, and FBF-based data query subscription, respectively, in conjunction with fig. 1 to 3, and includes the following steps:
1. SDP_FBF data automatic matching procedure
As shown in fig. 1, in this embodiment, taking a data transmission process between two simulation computing nodes (a first computing node and a second computing node) as an example, data is sent from the first computing node to the second computing node, where the first computing node and the second computing node are respectively defined as a local participant and a remote participant, and an execution process of the sdp_fbf data automatic matching method when data communication is performed between the two is divided into two phases: a participant discovery phase (SPD) and an endpoint discovery phase (SED).
In the SPD phase, the local participant first compresses and stores the endpoint description information thereof through the FBF marker bloom filter, wherein the endpoint description information is a unique key word of each local participant, namely the subject names of all data which can be provided by the local participant, then the local participant sends the data vector DV and the marker vector FV contained in the FBF marker bloom filter to the remote participant, and after the remote participant receives and stores the data vector DV and the marker vector FV, the remote participant starts the SED phase.
In the SED phase, the remote participant queries the FBF flag bloom filter sent from the local participant, if one or more topic names subscribed by the remote participant are judged to exist in the FBF, the remote participant sends subscription information of the topic to the local participant, and after the local participant receives the subscription information, the local participant sends a quality of service data packet QoS of the subscribed topic to the remote participant for further matching. If the matching is successful, the data of the subscribed theme starts to be transmitted between the two.
2. FBF-based data compression storage
In the present embodiment, the FBF Flag bloom filter includes two vectors, which are a Data Vector (Data Vector, DV) which is a one-dimensional Vector having m (m Σ1) bits and a Flag Vector (FV) which is a multidimensional Vector also having m bits, the dimension of each bit of which is determined by a Flag value corresponding to the bit. The initial value of each bit of the data vector DV and the flag vector FV is set to 0, corresponding to step one in the first embodiment.
As shown in fig. 2, taking the example of compressing three data information elements by the FBF flag bloom filter, compressed storage is divided into three phases: the hash phase, the modulus-taking phase, and the marking phase correspond to the second step in the first embodiment.
Wherein the hash phase: mapping the data element sets to be compressed into machine words z through hash functions ij As shown in fig. 2, three hash functions h are respectively passed through j (1. Ltoreq.j.ltoreq.3) each element x in the set i (1.ltoreq.i.ltoreq.3) into machine word z ij 。
And (3) a mould taking stage: machine word z ij The magnitudes of the data vectors DV are respectively modulo, i.e., |z ij |（z ij mod m). For any data element, the result of the mapping machine word modulo the size of the data vector DV will make the position corresponding to the modulo result plus 1 in the data vector DV be assigned 1, when storing multiple elements, if two or more elementsAre mapped to the same bit of the data vector DV, the value of which bit remains 1.
Marking: according to the compression mapping process of the data element set in the data vector DV, each bit in the flag vector FV is assigned, in fig. 2, the flag vector FV and the data vector DV are consistent in size, and each bit corresponds to one another, if a certain bit in the data vector DV is mapped by the ith element in the data set, the corresponding bit in the flag vector FV will be set to i, however, in the compression mapping process of multiple elements, the bit in the data vector DV mapped only by a certain element can retain the value of the corresponding bit in the flag vector FV, i.e. if a certain bit in the data vector DV is mapped by multiple elements, the corresponding bit in the flag vector FV will be restored to 0.
In fig. 2, the data element set T includes three elements (i.e., t= { x 1 ，x 2 ，x 3 The number of bits of both the data vector DV and the flag vector FV is 20 (i.e., m=20), and the hash stage uses three hash functions (i.e., k=3) in terms of element x 3 For example, first in the hash phase, x is taken as 3 Respectively calculating through three hash functions to obtain three machine words z 31 、z 32 And z 33 Let z be 31 =1306，z 32 =0892，z 33 In the modulo phase, the above three and their children are modulo-calculated according to the size of the data vector DV, and then: (z) 31 mod m)=6，(z 32 mod m)=12，(z 33 mod m) =16. The 7 th, 13 th and 17 th bits of the data vector DV are correspondingly set to 1, finally in the marking phase, according to the element x 1 And x 2 Is mapped by compression, only bit 17 has element x only 3 Mapping is thus performed such that bit 17 of the label vector FV is set to 3, bit 1 of the label vector FV is set to 1, and bit 10 is set to 2.
3. FBF-based data query subscription
In this embodiment, the query algorithm based on the FBF tag bloom filter is implemented by determining whether the data vector DV and the tag vector FV sent by each participant contain the information of the required data topic name, and the implementation process is as follows:
first, the topic name of a certain data to which a remote participant needs to subscribe is also mapped into a query vector, which is also a one-dimensional vector with m bits, through the same k hash operations and modulo operations.
Then, the query vector and all data vectors DV sent by the other local participants are respectively subjected to dot product, if the obtained dot product value is equal to k and the corresponding non-zero value in the flag vector FV is not more than 1, the corresponding local participant is considered to be able to provide the data and subscribe the data, corresponding to the third and fourth steps in the first embodiment.
As shown in fig. 3, assuming that the elements W, Y, H need to be queried, if they belong to a data element set T contained in a local participant, mapping the three elements into a Query Vector (QV) through three identical hash functions, where the symbols in fig. 3 are W, Y, H, respectively, for the element W, the dot product between the query Vector W and the data Vector DV satisfies d=3, and simultaneously satisfies that the corresponding bit non-zero value in the flag Vector FV is only 1, then it is determined that W belongs to the set T; for the element Y, the dot product d=2 between the query vector Y and the data vector DV, and judging that Y does not belong to the set T; for element H, the dot product between the query vector H and the data vector DV satisfies d=3, but the number of corresponding bits non-zero in the label vector FV is 2, and it is determined that H does not belong to the set T.
The foregoing has shown and described the basic principles, principal features and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that the above embodiments and descriptions are merely illustrative of the principles of the present invention, and various changes and modifications may be made without departing from the scope of the invention as defined in the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (8)
1. The low false alarm rate data automatic matching method for the distributed simulation DDS bus is characterized by comprising the following steps of: the method comprises the following steps:
step one: when data communication is carried out among a plurality of groups of simulation nodes, each simulation node respectively passes through an FBF (fiber-film) mark bloom filter, the description information of all participant endpoints contained in the simulation nodes is compressed and mapped into two vectors which are respectively a data vector DV and a mark vector FV, and the initial value of each bit of the data vector DV and the mark vector FV is set to be 0;
step two: the information compression mapping is to describe information of each participant endpoint, so that the description information carries out hash operation for k times through k hash functions respectively, then the k hash operation results are limited between 0 and (m-1) through modulo operation, the modulo operation results are mapped to corresponding bits in a data vector DV, and then the corresponding bits in the obtained data vector DV are in one-to-one correspondence with bits in a flag vector FV;
step three: each simulation node mutually transmits a data vector DV and a marking vector FV, any simulation node searches the two vectors in the first step through a query algorithm, and if the two vectors contain the needed participant endpoint description information, the simulation nodes corresponding to the simulation nodes are subjected to data subscription, so that data communication is realized;
step four: the simulation node searches the data vector DV and the marking vector FV through a query algorithm, when the simulation node contains the needed participant endpoint description information, the corresponding simulation node is subjected to data subscription, the topic name of certain data to be subscribed by the simulation node is mapped into the query vector through the same k hash functions and modulo operation, then the query vector and all the data vectors DV sent by other simulation nodes are respectively subjected to dot product, and subsequent subscription operation is carried out according to the obtained dot product value.
2. The method for automatically matching low false alarm rate data for a distributed emulated DDS bus of claim 1, wherein: in the first step, the data vector DV is a one-dimensional vector having m bits, and the flag vector FV is a multidimensional vector having m bits, and the dimension of each bit is determined by the flag value corresponding to the bit.
3. The method for automatically matching low false alarm rate data for a distributed emulated DDS bus of claim 1, wherein: in the second step, a certain bit in the data vector DV is mapped by the i-th element in the data set, and the corresponding bit in the flag vector FV is set to i.
4. The method for automatically matching low false alarm rate data for a distributed emulated DDS bus of claim 1, wherein: in the second step, when a certain bit in the data vector DV is mapped multiple times by multiple elements, the corresponding bit in the flag vector FV is restored to 0.
5. The method for automatically matching low false alarm rate data for a distributed emulated DDS bus of claim 1, wherein: in the second step, the hash operation is specifically that the data element sets to be compressed are mapped into machine words through hash functions respectively.
6. The method for automatically matching low false positive rate data for a distributed emulated DDS bus of claim 5, wherein: in the second step, the modulo operation is to modulo the machine word to the size of the data vector DV.
7. The method for automatically matching low false alarm rate data for a distributed emulated DDS bus of claim 1, wherein: in the fourth step, if the obtained dot product value is equal to k and the value of the corresponding bit in the flag vector FV is not more than one, the corresponding simulation node is considered to be able to provide the data, so as to subscribe.
8. The method for automatically matching low false alarm rate data for a distributed emulated DDS bus of claim 1, wherein: in the fourth step, the query vector is also a one-dimensional vector having m bits.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311561319.7A CN117762556A (en) | 2023-11-22 | 2023-11-22 | Low false alarm rate data automatic matching method for distributed simulation DDS bus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311561319.7A CN117762556A (en) | 2023-11-22 | 2023-11-22 | Low false alarm rate data automatic matching method for distributed simulation DDS bus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117762556A true CN117762556A (en) | 2024-03-26 |
Family
ID=90317171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311561319.7A Pending CN117762556A (en) | 2023-11-22 | 2023-11-22 | Low false alarm rate data automatic matching method for distributed simulation DDS bus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117762556A (en) |
-
2023
- 2023-11-22 CN CN202311561319.7A patent/CN117762556A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107241221B (en) | Support the network management protocol data unified processing system and method for different network management protocols | |
| CN107832343B (en) | Bitmap-based method for quickly retrieving data by MBF data index structure | |
| WO2016106591A1 (en) | Method and device for data packet extraction | |
| US10057344B2 (en) | Efficient transport of data encoded using multiple templates | |
| CN103838770A (en) | Logic data partition method and system | |
| US8719345B2 (en) | Database replication using collaborative data transfers | |
| Li et al. | GBC3: A versatile cube-based server-centric network for data centers | |
| Luo et al. | Efficient multiset synchronization | |
| Zhang et al. | Partialsync: Efficient synchronization of a partial namespace in ndn | |
| CN117762556A (en) | Low false alarm rate data automatic matching method for distributed simulation DDS bus | |
| WO2021073405A1 (en) | Data distribution method, electronic device, and storage medium | |
| CN110750897B (en) | DDS automatic discovery method based on threshold bloom filter | |
| Putra et al. | Node discovery scheme of DDS for combat management system | |
| CN112269726A (en) | Data processing method and device | |
| CN113626369B (en) | Method, device, equipment and readable medium for multi-node cluster ring communication | |
| CN112257285B (en) | Lightweight DDS automatic discovery method for large-scale distributed simulation | |
| CN111917901B (en) | IP address addressing method in data center network BCDC | |
| CN114936097A (en) | Processing node determination method, processing node determination device, computer equipment and storage medium | |
| Pu et al. | Practical implementation of an OPC UA multi-server aggregation and management architecture for IIoT | |
| CN114385673A (en) | Three-element query method based on privacy protection set intersection | |
| CN113709012A (en) | Extended communication method based on Modbus communication protocol | |
| CN112202677B (en) | Hardware acceleration query method, system, electronic equipment and storage medium | |
| CN113132233B (en) | Data processing method, software defined network controller and data processing system | |
| CN115242702B (en) | Internet of things node optimal path planning method and system | |
| CN111030856B (en) | Cloud-based data access method, electronic device and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |