CN116776275A - A multi-thread-based multi-link data reception and fusion method - Google Patents

Abstract

The invention relates to a multi-thread-based multi-link data reception and fusion method, belonging to the field of high-availability data fusion. The invention is applied to high-reliability data remote transmission and storage, node service cluster construction, high reliability, and can be applied to application environments such as GPS or Beidou positioning objects and object tracking display services, so that data providers do not need to worry about server downtime. Data transmission is suspended due to problems such as power outage or node application process stopping. When a power outage occurs in one of the service clusters of multiple computer rooms, users do not have to worry about data transmission being interrupted, thereby improving the stability of data transmission, reducing the operation and maintenance costs of supervisors, and allowing users to only care about the data display business. At the same time, the application has configurable data source connections, configurable data protocol parsing, and configurable fusion filtering algorithms. Users can customize data source connections, customize data protocols, and data filtering methods for parsing, filtering, and forwarding data.

Description

A multi-thread-based multi-link data reception and fusion method

Technical field

The invention belongs to the field of high-availability data fusion, and specifically relates to a multi-thread-based multi-link data receiving and fusion method.

Background technique

As Beidou positioning technology evolves more and more maturely and is used more and more widely, more and more mobile phones, computer terminals, and web pages provide users with convenient and fast user positioning experience services. Beidou technology has become a political service Conventional technology in many fields such as , military and civilian applications. The content of the Beidou data protocol is more diverse, including digital forms of longitude, latitude, time and date, and string forms such as daily text. However, traditional positioning technology cannot solve the problem of data positioning accuracy, long data transmission cycle and lack of sustainable data transmission solutions. Therefore, it is no longer suitable for positioning services that are used more and more frequently and require high reliability.

In order to solve many problems such as server shutdown, server application process disappearance, data loss, data timing confusion, and Beidou positioning data offset during the application process of Beidou positioning technology, in order to make data transmission more stable during the combination of Beidou positioning technology and application, Highly efficient, the present invention aims to apply multi-threading technology and fusion filtering algorithm technology to Beidou positioning data source connection, data protocol analysis and error data filtering to solve various problems that arise in the process of receiving and forwarding data.

The invention is a data transmission service applied to Beidou/GPS positioning data, and mainly solves the sustainable transmission of basic data such as multi-link data filtering and fusion, multi-protocol parsing generalization, and multi-connection generalization. Specific issues addressed include:

To solve the problem of data not being lost when downtime occurs, application processes disappear, etc., it is necessary to build a node cluster, thereby increasing stability and reducing operation and maintenance costs.

To solve the problems of processing too many types of protocols and processing new protocols in uncertain formats, it is necessary to design a set of general configuration files for parsing multiple data protocols. Through the configuration file configuration instructions, users can customize the data protocol format and reduce user operations. steps and reduce R&D costs.

It solves the problem of data redundancy and the problem of sending erroneous data. According to the characteristics of user-defined data, users can customize the data filtering algorithm to filter out erroneous data.

Solve the problem of complex and diverse data sources.

Solve the problem of status synchronization in node cluster construction.

Node recovery startup status recovery problem.

Solve the problem of single or multiple server shutdowns, process loss, and request failure during the status synchronization process of all nodes.

Contents of the invention

(1) Technical problems to be solved

The technical problem to be solved by this invention is how to provide a multi-thread-based multi-link data receiving and fusion method to solve the problems of server shutdown, server application process disappearance, data loss, data timing confusion, and Beidou positioning technology problems during the application process. Positioning data offset and many other issues.

(2) Technical solutions

In order to solve the above technical problems, the present invention proposes a multi-thread-based multi-link data reception and fusion method. The method is based on a fusion system. The fusion system is composed of multiple nodes to form a cluster, and each node is connected to three targets. The first target is the data source, which is used to receive data. The second target is the database. The database is a relational database and is used to record the cluster status. The third target is the cache middleware Redis, which is used for the data fusion script protocol. Execution; the method includes:

In each node, the receiver form, data protocol form and data filtering algorithm are defined through a custom configuration file, and multiple threads are established through thread pool technology. Each thread establishes a receiver and runs data reception and data processing services at the same time. And the data processing services of each receiver will not affect each other. After parsing, the data from multiple data sources will be summarized to perform data deduplication and data filtering operations;

Each node is an independently running node, and each node uses multi-threading technology to connect multiple data sources and run in parallel. Each node deduplicates the data received by the data source, and then calculates and receives the data distance between the two nearest points. Calculate the time difference to determine whether it is wrong data. If so, filter it. Finally, send the processed data to the cache middleware Redis. Rely on the script technology and SET collection of the cache middleware Redis to perform data deduplication and data sending, and cache the data. Outdated data can be destroyed regularly;

The cache middleware Redis is set outside the application cluster. Each node in the cluster is connected to the cache middleware Redis through the TCP protocol. Data is sent to the SET set of the cache middleware Redis. When the set SET does not exist, the data is sent to the blocking queue. And the cache middleware Redis supports multiple language connections and can consume blocking queue data;

Each node is deployed on a different server. Each node is a process in the server. The cluster is built on multiple servers. Multiple servers can be distributed in multiple computer rooms. If one server goes down or the server restarts, If the process disappears, the cluster will lose a node. However, other nodes are still running and the data will not be lost; because the data of all nodes receive the same data, the data sent is also the same. Even if the server of one node fails, the data will not be lost. It does not affect the reception and sending of data by other nodes, and due to the caching middleware Redis script technology and the non-duplicate characteristics of SET collections, the data of all nodes are sent to this SET collection to determine whether it exists. If it exists, it means it has been received. If not, it means that it has been received. exists, it means that the data has not been received, so when a node hangs up, the data can be resumed within 1 second, and the data is unique;

The data received by each node has a data time. First, each node will filter out time-ordered data. However, the time each node sends to the cache middleware Redis is different. Therefore, there are two ways to customize the script protocol of the cache middleware Redis. There are two solutions. The first solution is that because the data of each node is ordered, you only need to set the SET set in the cache middleware Redis, send the data of all nodes to the SET set, and take advantage of the non-repeatability of the SET set. , determine whether the data is received. If there is no data in the SET set, cache the data to the SET set and send the data to the blocking queue. Because the queue has a first-in, first-out characteristic, the final consumption of data will be in order; The second option is to set the permission Key in the cache middleware Redis, and it has a validity of 5 seconds. This 5-second validity can be customized to any positive integer value. The value of the Key is the IP address of the node. The allocation rules of this permission The node that sends data first has this permission. That is to say, the value of this Key is the IP address of the node that sends data first. Then this node will have 5 seconds to send data to the SET collection permission, and then determine whether the collection is Receive the same data. If not, cache the data to the SET collection and send the data to the blocking queue; after 5 seconds, an empty Key will be created again. If any node sends data first, then the IP address of this node will be assigned a value. Given this Key, since the data of each node is sequential, the data finally collected into the queue is sequential;

The user sends a request to Node 1, and Node 1 obtains the cluster configuration through the configuration file, asynchronously forwards the request to other nodes, and synchronizes the status so that all nodes in the cluster have the same receiving and forwarding status at the same time. When the synchronization is successful, the current status information is saved in the database. ; When a node is down, or requests are blocked, the node can be removed without affecting the request operations of other nodes; if it is not down or requests are blocked, when a node service request fails, in order to make all node states consistent, Use the previous status of the node that has been stored in the database to read the previous status of the database for the node to restore the previous status; use the database to store the node status, connection and forwarding information. When the node hangs up, restart it to restore the connection status. and other states.

(3) Beneficial effects

The present invention proposes a multi-thread-based multi-link data reception and fusion method. The present invention discloses a multi-thread, multi-link data data fusion filtering method. The main advantages are reflected in the following aspects:

1. Designed a multi-link data fusion solution. Even if problems such as server shutdown and application process disappearance occur, data can be resumed within one second, so that customers do not have to worry about power outages and other problems. There is also a startup script to start the server. The app can be run without much action required.

2. Designed a cluster building solution that uses multi-threading technology to send a request to a node. This node will asynchronously send the same request to all other nodes through multi-threading, so that the cluster's configuration file is uploaded, the configuration file is synchronized, and the status is sent. Problems such as synchronization can be easily solved. However, when a server goes down, the application node on this server will disappear, and other nodes will remove this node, without affecting the normal operation of other nodes. However, when a request problem occurs on one node, you can also set up a rollback operation for the status of all nodes to make the status of all nodes consistent, thereby reducing operation and maintenance operations.

3. Designed a method to support custom data filtering algorithms. Users can customize what kind of data to obtain according to their own data needs, and customize their own filtering methods by uploading binary algorithm files and configuration files, which is simple and effective.

4. Designed a simple method to build services that supports connecting multiple data sources. Using multi-threading technology, one node can connect to multiple data sources. By configuring basic information of multiple data sources, forwarding destinations, routing tables and other writing operations , and upload the configuration file, that is, the application node required by the user can be customized and saved in the database. When the node is restarted, the last configuration connection information will still be used, and there is no need to upload the configuration file again.

5. Designed a universal solution for data protocol parsing. By customizing the configuration file protocol, the application node's function of processing data protocols is universal, reducing developers' research and development of new protocols, and only needing to upload the configuration file once. , you can build an environment for custom data protocol parsing algorithms, thereby reducing the operations of user-defined data protocols.

6. Designed an overall architecture, including cache middleware Redis, database and node applications, and supported clients written in multiple languages to access and obtain data.

Description of drawings

Figure 1 is a system architecture diagram of the present invention;

Figure 2 is a business flow chart;

Figure 3 is a multi-link deployment diagram;

Figure 4 is a node cluster status synchronization diagram;

Figure 5 shows the difference data processing flow;

Figure 6 is a flow chart of the difference algorithm.

Detailed ways

In order to make the purpose, content and advantages of the present invention clearer, specific implementation modes of the present invention will be further described in detail below in conjunction with the accompanying drawings and examples.

The invention belongs to the field of high-availability data fusion, a multi-thread-based multi-link data reception and fusion method, and specifically involves multi-thread technology, filtering algorithm technology, and data protocol analysis technology.

The invention provides a solution for data filtering fusion, node status synchronization and node status recovery, and combines Beidou technology with multi-thread technology. The invention provides a multi-thread-based multi-link data reception fusion filtering method.

The method of the present invention includes application in high-reliability data remote transmission and storage, node service cluster construction, and can be applied to application environments such as GPS or Beidou positioning objects and object tracking display services, so that data providers do not need to worry about server shutdown and power outage. Or the node application process stops running, causing problems such as data transmission interruption. When a power outage occurs in one of the service clusters of multiple computer rooms, users do not have to worry about data transmission being interrupted, thereby improving the stability of data transmission, reducing the operation and maintenance costs of supervisors, and allowing users to only care about the data display business. At the same time, the application has configuration functions such as configurable data source connection, configurable data protocol parsing, and configurable fusion filtering algorithms. Users can customize data source connections, customize data protocols, and data filtering methods for parsing, filtering, and Forward data.

A multi-thread-based multi-link data receiving and fusion method. The present invention can start multiple nodes as a cluster. Each node has data sending and receiving and data fusion functions, so there will be no repeated data transmission or data loss. The specific architecture is shown in Figure 1.

The method of the present invention is based on a fusion system. The fusion system is composed of multiple nodes, forming a cluster. Each node is connected to three targets. The first target is a data source. There can be multiple data sources for receiving data. The third target is a data source. The second target is the database. The database is a relational database, such as Oracle, MySQL or the domestic database Dameng and Shentong. It is used to record the cluster status. The third target is the cache middleware Redis, which is used to execute the data fusion script protocol. , the program includes the following solutions to several problems.

1.1 Multi-threaded processing of data from multiple data sources

In each node, the receiver form, data protocol form and data filtering algorithm are defined through a custom configuration file, and multiple threads are established through thread pool technology. Each thread establishes a receiver and runs data reception and data processing services at the same time. And the data processing services of each receiver will not affect each other. After parsing, the data from multiple data sources will be summarized to perform data deduplication and data filtering operations.

1.2 Multi-link data fusion

As shown in Figure 1, the cluster consists of multiple nodes. Each node is an independently running node, and each node uses multi-threading technology to connect multiple data sources and run in parallel. However, since the data received by each node has Duplicate data and erroneous data. In order to ensure that the data sent by the cluster is not redundant and does not produce errors, each node first deduplicates the data received by the data source, and then calculates the data distance and reception time difference between the two most recent points to determine whether it is an error. data, if so, it is filtered, and finally the processed data is sent to the cache middleware Redis. It relies on the scripting technology of the cache middleware Redis and the SET collection to perform data deduplication and data sending, and destroys outdated data regularly, relying on the advanced scripting technology. The efficiency is such that data can be restored and sent within 1 second. The specific script protocol is shown in Figure 2.

(1) This invention provides data to the outside world and can support different programming languages.

The cache middleware Redis is set up outside the application cluster. As long as the network is connected, it can be in different computer rooms or even different regions. The cache middleware Redis is built in a cluster. Each node in the cluster is connected to the cache middleware Redis through the TCP protocol, and writes data to the SET collection and blocking queue of the cache middleware Redis. The cache middleware Redis supports multiple language connections, so it can provide Systems developed in different development languages consume data.

(2) The present invention has high availability, and the fused data is unique

Each node is deployed on a different server. Each node is a process in the server. The cluster is built on multiple servers. Multiple servers can be distributed in multiple computer rooms. If one server goes down or the server restarts, If the process disappears, the cluster will lose one node. However, other nodes are still running and the data will not be lost, as shown in Figure 3. The data of all nodes receive the same data, so the data sent is also the same. Even if the server of one node fails, it will not affect the reception and sending of data by other nodes. Moreover, due to the non-duplicate characteristics of the caching middleware Redis script technology and SET collection , send the data of all nodes to this SET set to determine whether it exists. If it exists, it means that it has been received. If it does not exist, it means that the data has not been received. Therefore, when a node hangs up, the data can also be received in 1 second. The transmission is resumed within the system, and the data is unique. Then the system that needs to display the motion status of the object receives the data and displays the operation of the object on the map.

(3) The fused data is orderly

The data received by each node has a data time, as shown in Figure 2. First, each node will filter out time-ordered data. However, the time each node sends to the cache middleware Redis is different, so the cache middleware Redis There are two solutions for customizing the script protocol. The first solution is that because the data of each node is ordered, you only need to set the SET set in the cache middleware Redis, and use the non-repetitive nature of the SET set to send the data. Go to the SET collection to determine whether the data is received. If there is no data in the SET collection, cache the data to the SET collection, and then send the data to the blocking queue. Because the queue has the first-in-first-out characteristic, the final consumer of data has sequence. The second option is to set the permission Key in the cache middleware Redis, and it has a validity of 5 seconds. This 5-second validity can be customized to any positive integer value. The value of the Key is the IP address of the node. The allocation rules of this permission The node that sends data first has this permission. That is to say, the value of this Key is the IP address of the node that sends data first. Then this node will have 5 seconds to send data to the SET collection permission, and then determine whether the collection is Receive the same data, if not, cache the data to the SET collection, and then send the data to the blocking queue. After 5 seconds, an empty Key is created again, or any node sends data first, then the IP address of this node will be assigned to this Key. Since the data of each node has timing, the data finally collected into the queue has timing. sex.

1.3 Cluster settings

As shown in Figure 3, the cluster is composed of one or more nodes. Each node has the same data receiving and data sending services. Each node is connected to the same data source. There may be multiple data sources, so there are multiple data sources. Send the same data to each node at the same time. Just configure the IP addresses and ports of all nodes in the application configuration file, use server sockets to write rules and connect with commas, and write node cluster configuration, such as 127.0.0.1:8080,127.0 .0.1:8081,127.0.0.1:8082

1.4 Data protocol universal analysis solution

Configure the data protocol format through the configuration file, configure the protocol name, protocol ID, and the byte composition of the protocol. For example, the first byte is the word connector, the second byte is the data type, the third byte is the data quantity, and the fourth to fourth byte is the data type. The seven bytes are the longitude value of the data, etc., and the programming language reflection technology is used to configure the protocol data in the configuration file for each byte or each byte array to parse the method name, and the reflection mechanism and mapping mechanism of the programming language are used to convert the name Executable methods mapped to byte data through which the data is parsed. Users also upload their own method files to the database and load them into memory, and then use reflection technology to obtain executable methods. Finally, the method names and methods are cached in the application through the mapping relationship. When the data needs to be parsed, only It is necessary to obtain the parsing method through the configuration file protocol parsing data method name, and the application can parse the data through these methods.

1.5 Node connection status setting

The database is a relational database, such as Oracle, MySQL or the domestic databases Dameng and Shentong. If you need to use these databases, you only need to add the database driver and configuration file connection URL to connect, and set the connection target and forwarding target in the database. , after the node starts, it will automatically read the database and establish receiving and forwarding connections.

1.6 Node status setting

(1)State synchronization mechanism

The user sends a request to node 1, and node 1 obtains the cluster configuration through the configuration file, and asynchronously forwards the request to node 2, node 3, node 4... and other nodes. The synchronization state ensures that all nodes in the cluster have the same receiving and forwarding state at the same time. When synchronization is successful, the current status information is saved in the database.

(2)Remove node mechanism

When a node goes down or requests fail, the node can be removed without affecting the request operations of other nodes.

(2)State rollback mechanism

Use the database to store the previous state of the node. When a node service request fails, read the previous state of the database and roll back to restore the previous state.

(3)State recovery mechanism

Use a database to store node status, connection and forwarding information. When a node hangs up and is restarted, the connection status and other statuses can be restored.

1.7 Customizable data filtering protocol

(1) Customized data filtering algorithm

Users can customize the data filtering algorithm by uploading the data filtering algorithm binary file. The application loads the binary file into the memory, and then obtains the filtering algorithm through the reflection technology of the programming language, so that the application can filter the data that the user wants to filter through the algorithm.

1.8 Custom data connection scheme

(1) Customized data source connection

The invention supports multiple data sources. The data source connection protocol TCP/UDP and data connection mode are configured through the configuration file. The data source is configured as a client or a server, etc.

(2) Customized TCP data sticky half-packet data processing algorithm

Configure TCP data sticky half-packet data processing algorithm through configuration file

(3) Customized data protocol parsing rules

Configure data protocol construction and parsing rules through configuration files to parse data.

1.9 Startup script

Create an application startup script and place the script in the folder where the server system starts up. When the server starts, it will automatically run the application, cache middleware Redis, database and message queue middleware and other applications, and have an execution sequence.

2.0 monitoring script

Write monitoring scripts to monitor application process names, cache middleware Redis and database processes. They are used to start database, cache middleware Redis, message queue middleware and other applications, and have an execution sequence.

Example 1:

The invention is an application developed based on the Java open source technology framework and supports multiple databases, multiple data sources, multiple data protocols, and multiple stream data filtering algorithms. The construction process is to first build the database and cache middleware Redis, then start the data source, start node applications on multiple servers, upload configuration files, and finally start the data source to send data to perform data fusion and filtering processing on the data.

(1) Connector configuration: Establish multiple data source connections through multi-threading. The connector can be configured with multiple connection protocols, TCP or UDP connections. UDP is used to support multicast and unicast modes, and TCP connections are used to support processing of data stickiness. Package and half package issues. The invention provides two data sticky half-packet processing algorithms. One of them is that each frame in the data has a data header and data length field. Based on the data header and data length, it is judged whether the current data is a complete frame of data. If not, Discard directly. The second one supports frame header, frame tail and verification bytes, which is used to detect whether the data is a complete frame of data. If the data is stuck, it will be unpacked. If it is a half packet, it will be added to the cache. The arrival of the next frame of data is being processed.

(2) Protocol configuration: configure the protocol name, protocol ID and protocol sticky half-packet parsing algorithm, and configure the data type and parsing algorithm represented by each byte of the protocol data;

(3) Filter configuration: Configure a custom algorithm. The present invention provides two algorithms. One is an algorithm that calculates the speed of Beidou positioning objects based on time-series data, and determines whether the current data or objects are abnormal based on the speed, and handles exceptions. The calculation formula is as follows:

distence＝R*arccos(sin(latitude1)*sin(latitude 2)+cos(latitude1)*cos(latitude 2)*cos(longitude1-longitude 2))

v＝distence/(time2–time1)

Where distance is the distance, R is the radius, latitude1 is the latitude of the first point, latitude2 is the latitude of the second point, longitude1 is the longitude of the first point, longitude2 is the longitude of the second point, v is the speed, time2 is the time of the second point, time1 is The first bit of time.

You can also customize the algorithm. If the abnormal data is Gaussian distribution, you can use the Gaussian filtering algorithm to establish a confidence interval, retain the confidence interval data, and filter out erroneous data.

(4) Configure difference algorithm: If it is GPS data or Beidou data, configuration options can be added. Due to the use of multi-threading technology, it can support the difference calculation of multiple objects.

(5) Method configuration in field configuration, to parse protocol data, you can also use the protocol data parsing method that comes with the node application.

The invention discloses a data fusion and filtering method for multi-threaded and multi-link data. The main advantages are reflected in the following aspects:

1. Designed a multi-link data fusion solution. Even if problems such as server shutdown and application process disappearance occur, data can be resumed within one second, so that customers do not have to worry about power outages and other problems. There is also a startup script to start the server. The app can be run without much action required.

2. Designed a solution for cluster construction, using multi-threading technology to send a request to a node, and asynchronously sending the same request to other nodes through multi-threading, so that the cluster's configuration file upload, configuration file synchronization, sending status synchronization and other issues can be solved. It can be solved easily. However, when a server goes down, the application node on this server will disappear, and other nodes will remove this node, without affecting the normal operation of other nodes. However, when a request problem occurs on one node, you can also set up a rollback operation for the status of all nodes to make the status of all nodes consistent, thereby reducing operation and maintenance operations.

3. Designed a method to support custom data filtering algorithms. Users can customize what kind of data to obtain according to their own data needs, and customize their own filtering methods by uploading binary files and configuration files, which is simple and effective.

4. Designed a simple method to build services that supports connecting multiple data sources. Using multi-threading technology, one node can connect to multiple data sources. By configuring basic information of multiple data sources, forwarding destinations, routing tables and other writing operations , and upload the configuration file, that is, the application node required by the user can be customized and saved in the database. When the node is restarted, the last configuration connection information will still be used, and there is no need to upload the configuration file again.

5. Designed a universal solution for data protocol parsing. By customizing the configuration file protocol, the application node's function of processing data protocols is universal, reducing developers' research and development of new protocols, and only needing to upload the configuration file once. , you can build an environment for custom data protocol parsing algorithms, thereby reducing the operations of user-defined data protocols.

6. Designed an overall architecture, including cache middleware Redis, database and node applications, and supported clients written in multiple languages to access and obtain data.

The above are only preferred embodiments of the present invention. It should be noted that those of ordinary skill in the art can also make several improvements and modifications without departing from the technical principles of the present invention. These improvements and modifications It should also be regarded as the protection scope of the present invention.

Claims (10)

1. A multi-link data receiving and fusion method based on multi-threading, characterized in that the method is based on a fusion system, which consists of multiple nodes forming a cluster. Each node is connected to three targets. The first The target is the data source, which is used to receive data. The second target is the database. The database is a relational database used to record the cluster status. The third target is the cache middleware Redis, used for the execution of the data fusion script protocol; The method includes: In each node, the receiver form, data protocol form and data filtering algorithm are defined through a custom configuration file, and multiple threads are established through thread pool technology. Each thread establishes a receiver and runs data reception and data processing services at the same time. And the data processing services of each receiver will not affect each other. After parsing, the data from multiple data sources will be summarized to perform data deduplication and data filtering operations; Each node is an independently running node, and each node uses multi-threading technology to connect multiple data sources and run in parallel. Each node deduplicates the data received by the data source, and then calculates and receives the data distance between the two nearest points. Calculate the time difference to determine whether it is wrong data. If so, filter it. Finally, send the processed data to the cache middleware Redis. Rely on the script technology and SET collection of the cache middleware Redis to perform data deduplication and data sending, and cache the data. Outdated data is destroyed regularly; The cache middleware Redis is set outside the application cluster. Each node in the cluster is connected to the cache middleware Redis through the TCP protocol. Data is sent to the SET set of the cache middleware Redis. When the set SET does not exist, the data is sent to the blocking queue. Moreover, the cache middleware Redis supports multiple language connections and can consume blocking queue data; Each node is deployed on a different server. Each node is a process in the server. The cluster is built on multiple servers. Multiple servers are distributed in multiple computer rooms. If a server goes down or the server restarts, the process disappears, the cluster will lose a node, but other nodes are still running and the data will not be lost; the data of all nodes receive the same data, so the data sent is also the same. Even if the server of a node fails, it will not be affected. Other nodes receive and send data, and due to the caching middleware Redis script technology and the non-duplicate characteristics of SET collections, the data of all nodes are sent to this SET collection to determine whether it exists. If it exists, it means it has been received. If it does not exist, It means that the data has not been received, so when a node hangs up, the data can be resumed within 1 second, and the data is unique; The data received by each node has a data time. First, each node will filter out time-ordered data. However, the time each node sends to the cache middleware Redis is different. Therefore, there are two ways to customize the script protocol of the cache middleware Redis. There are two solutions. The first solution is that because the data of each node is ordered, you only need to set the SET set in the cache middleware Redis, send the data of all nodes to the SET set, and take advantage of the non-repeatability of the SET set. , determine whether the data is received. If there is no data in the SET set, the data will be cached in the SET set and sent to the blocking queue. Because the queue has the first-in, first-out characteristic, the final consumption of data will have orderliness; The second option is to set the permission Key in the cache middleware Redis and have a 5-second aging. This 5-second aging can be customized as any positive integer value. The value of the Key is the IP address of the node. The allocation rules for this permission are: The node that sends data first has this permission. That is to say, the value of this Key is the IP address of the node that sends data first. Then this node will have 5 seconds to send data to the SET collection permission, and then determine whether the collection receives it. The same data, if not, caches the data to the SET collection, and then sends the data to the blocking queue; after 5 seconds, an empty Key is created again, or any node sends data first, then the IP address of this node will be assigned a value Given this Key, since the data of each node is sequential, the data finally collected into the queue is sequential; The user sends a request to Node 1, and Node 1 obtains the cluster configuration through the configuration file, asynchronously forwards the request to other nodes, and synchronizes the status so that all nodes in the cluster have the same receiving and forwarding status at the same time. When the synchronization is successful, the current status information is saved in the database. ; When a node is down, or requests are blocked, removing this node will not affect the request operations of other nodes; if it is not down or requests are blocked, when a node service request fails, in order to make the status of all nodes consistent, use Read the previous status of the node that has been stored in the database, and use it to restore the previous status of the node; use the database to store the node status, connection and forwarding information. When the node hangs up and restarts, the connection status and Other status. 2. The multi-thread-based multi-link data receiving and fusion method as claimed in claim 1, wherein the relational database is Oracle, MySQL, Dameng or Shentong. 3. The multi-thread-based multi-link data receiving and fusion method as claimed in claim 1, characterized in that multiple data sources send the same data to each node at the same time, as long as the IPs of all nodes are configured in the application configuration file. Address and port, write rules using server sockets and connect with commas, write node cluster configuration. 4. The multi-thread-based multi-link data receiving and fusion method as claimed in claim 1, characterized in that the protocol name, protocol ID, and byte array of the protocol are configured through the configuration file data protocol format, and a programming language is used. Reflection technology configures the protocol data in the configuration file to parse the method name for each byte or byte array. It uses the reflection mechanism and mapping mechanism of the programming language to map the name to an executable method of byte data, and parses the data through these methods. . 5. The multi-thread-based multi-link data receiving and fusion method as claimed in claim 1, characterized in that the user customizes the filtering data algorithm by uploading a data filtering algorithm binary file, and then loads the binary file into the memory through programming. The reflection technology of the language obtains the filtering algorithm, so that the application can filter the data that the user wants to filter through the algorithm. 6. The multi-thread-based multi-link data receiving and fusion method according to claim 1, characterized in that the user customizes the data source connection and configures the data source connection protocol TCP/UDP, the data connection mode, and the data source through the configuration file. Configure for the client or server, configure the TCP data sticky half-packet data processing algorithm through the configuration file, configure the data protocol construction and parsing rules through the configuration file, and parse the data. 7. The multi-thread-based multi-link data receiving and fusion method as claimed in claim 1, characterized in that an application startup script is created and the script is placed in a folder where the server system is started and run. When the server is started, the application will be automatically run. , cache middleware Redis, database and message queue middleware applications, and have execution order. 8. The multi-thread-based multi-link data receiving and fusion method as claimed in claim 1, characterized by writing monitoring scripts, monitoring application process names, cache middleware Redis and database processes for starting the database and cache middleware. Redis, message queue middleware applications, and execution sequence. 9. The multi-thread-based multi-link data receiving and fusion method according to any one of claims 1-8, characterized in that multiple data source connections are established through multi-threads, and the connector is configured with multiple connection protocols, such as TCP or UDP connection, which uses UDP to support multicast and unicast modes, uses TCP connection to support processing of data sticky and half-packet problems; data sticky and half-packet processing algorithms include two types, one of which is that each frame in the data contains The data header and data length fields are used to determine whether the current data is a complete frame of data based on the data header and data length. If it is not discarded directly, the second method supports frame header, frame tail and verification bytes for detecting data. Is it a complete frame of data? If it is sticky, it will be unpacked and processed. If it is a half packet, it will be added to the cache and will be processed when the next frame of data arrives. 10. The multi-thread-based multi-link data receiving and fusion method according to claim 9, wherein the filter configuration includes: configuring a custom algorithm, one is to calculate the speed of Beidou positioning objects based on time-series data algorithm, and determine whether the current data or object is abnormal based on the speed, and perform exception handling. The calculation formula is as follows: distence＝R*arccos(sin(latitude1)*sin(latitude2)+cos(latitude1)*cos(latitude2)*cos(longitude1-longitude2)) v＝distence/(time2–time1) Where distance is the distance, R is the radius, latitude1 is the latitude of the first point, latitude2 is the latitude of the second point, longitude1 is the longitude of the first point, longitude2 is the longitude of the second point, v is the speed, time2 is the time of the second point, time1 is The first bit of time; The second is a custom algorithm. If the abnormal data is Gaussian distributed, use the Gaussian filtering algorithm to establish a confidence interval, retain the confidence interval data, and filter out erroneous data.