CN114124929B - Cross-network data processing method and device - Google Patents

Cross-network data processing method and device Download PDF

Info

Publication number
CN114124929B
CN114124929B CN202111148217.3A CN202111148217A CN114124929B CN 114124929 B CN114124929 B CN 114124929B CN 202111148217 A CN202111148217 A CN 202111148217A CN 114124929 B CN114124929 B CN 114124929B
Authority
CN
China
Prior art keywords
network
transmission protocol
end processor
communication transmission
file
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111148217.3A
Other languages
Chinese (zh)
Other versions
CN114124929A (en
Inventor
王庆振
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qianxin Technology Group Co Ltd
Secworld Information Technology Beijing Co Ltd
Original Assignee
Qianxin Technology Group Co Ltd
Secworld Information Technology Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qianxin Technology Group Co Ltd, Secworld Information Technology Beijing Co Ltd filed Critical Qianxin Technology Group Co Ltd
Priority to CN202111148217.3A priority Critical patent/CN114124929B/en
Publication of CN114124929A publication Critical patent/CN114124929A/en
Application granted granted Critical
Publication of CN114124929B publication Critical patent/CN114124929B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/06Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/08Protocols for interworking; Protocol conversion

Abstract

The embodiment of the invention provides a data processing method across networks, which comprises the steps of receiving a first communication transmission protocol request message sent by a client through a first front-end processor, converting the first communication transmission protocol request message into a request file, and forwarding the request file; the first front-end processor and the client are positioned in a first network; receiving the request file through the gateway and forwarding the request file; receiving the request file through the second front-end processor, converting the request file into a second communication transmission protocol request message, and forwarding the second communication transmission protocol request message to the server so that the server generates and returns a target response message; the second front-end processor and the server are located in a second network, and the second network is different from the first network. In the embodiment of the invention, the first front-end processor and the second front-end processor are introduced to perform the mutual conversion between the message and the request file, thereby effectively improving the data transmission efficiency across networks.

Description

Cross-network data processing method and device
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for processing data across networks, a computer device, and a computer readable storage medium.
Background
Conventional service invocation is limited to the same network. Cross-network service invocation then needs to be implemented based on the communication transport protocol. Based on the consideration of data security, the service systems of different networks are usually isolated by a gateway. The gatekeeper can block the physical connection, the logical connection and the communication transmission protocol connection of the communication between the service systems of different networks, so that the service systems of different networks cannot exchange information based on the communication transmission protocol. Therefore, the service systems of different networks can only exchange information in the form of data files through the gatekeeper.
Therefore, each time of service data exchange across the network, the exchange mode of the gatekeeper needs to be adapted, resulting in low data transmission efficiency.
Disclosure of Invention
The invention aims to provide a cross-network data processing method, a device, computer equipment and a computer readable storage medium, which are used for solving the problem of low cross-network data transmission efficiency.
An aspect of an embodiment of the present invention provides a method for processing data across a network, including:
receiving a first communication transmission protocol request message sent by a client through a first front-end processor, converting the first communication transmission protocol request message into a request file, and forwarding the request file; wherein the first front-end processor and the client are located in a first network;
Receiving a request file forwarded by the first front-end processor through a gateway, and forwarding the request file; and
Receiving a request file forwarded by the gateway through a second front-end processor, converting the request file into a second communication transmission protocol request message, and forwarding the second communication transmission protocol request message to a server, so that the server generates and returns a target response message based on the second communication transmission protocol request message; the second front-end processor and the server are located in a second network, and the second network is different from the first network.
Optionally, the converting the first communication transmission protocol request message into a request file includes:
creating a first file in a disk of the first front-end processor through the first front-end processor;
analyzing the first communication transmission protocol request message to obtain the content in the first communication transmission protocol request message; and
And writing the content in the first communication transmission protocol request message into the first file to obtain the request file.
Optionally, the method further comprises:
and checking the request file through the gatekeeper, and forwarding or intercepting the request file according to a checking result.
Optionally, the second front-end processor is configured with a thread pool, and the thread pool comprises IO threads and business threads; the IO thread is used for sending the second communication transmission protocol request message to the server; the method further comprises the steps of:
if the second communication transmission protocol request message is sent to the server through the IO thread, calling the service thread to wait for the server to return the target response message and blocking the service thread; and
If the target response message is received within the preset time, waking up the service thread, converting the target response message into a target response file through the service thread, and forwarding the target response file to the gatekeeper through the IO thread.
Optionally, the method further comprises:
and if the target response message is not received within the preset time, returning timeout response information representing calling timeout.
Optionally, the method further comprises:
when the target response message is received and the service thread is awakened, a target identifier in the target response message is obtained, and whether the target identifier is the same as a preset identifier in the second communication transmission protocol request message is judged; and
And if the target identifier is the same as the preset identifier, determining that the target response message corresponds to the second communication transmission protocol request message, and further determining to provide the target response message to the client.
Optionally, the method further comprises:
configuring a connection channel through the second front-end processor, wherein the connection channel serves each request of the client; the connection channel comprises a connection established between the second front-end processor and the network gate, and a connection established between the second front-end processor and the server; the connecting channel is in an open state when a preset condition is met.
An aspect of the embodiment of the present invention further provides a cross-network data processing method, which is used in a second front-end processor, where the method includes:
receiving a request file forwarded by a gateway; the request file in the network gate is from a first front-end processor, the request file is obtained by converting a first communication transmission protocol request message by the first front-end processor, the first communication transmission protocol request message is from a client, and the client and the first front-end processor are positioned in a first network;
converting the request file into a second communication transmission protocol request message; and
Calling a server through the second communication transmission protocol request message so that the server generates and returns a target response message based on the second communication transmission protocol request message; the second front-end processor and the server are located in a second network, and the second network is different from the first network.
Optionally, the second front-end processor is configured with a thread pool, and the thread pool comprises IO threads and business threads; the calling the server through the second communication transmission protocol request message comprises the following steps:
sending the second communication transmission protocol request message to the server through the IO thread;
calling the service thread to wait for the server to return the target response message and blocking the service thread; and
If the target response message is received within the preset time, waking up the service thread, converting the target response message into a target response file through the service thread, and forwarding the target response file to the gatekeeper through the IO thread.
Optionally, the method further comprises:
and if the target response message is not received within the preset time, returning timeout response information representing calling timeout.
Optionally, the method further comprises:
when the target response message is received and the service thread is awakened, a target identification of the target response message is obtained, and whether the target identification is the same as a preset identification in the second communication transmission protocol request message or not is judged; and
And if the target identifier is the same as the preset identifier, determining that the target response message corresponds to the second communication transmission protocol request message, and further determining to provide the target response message to the client.
Optionally, the method further comprises:
configuring a connection channel through a second front-end processor, wherein the connection channel serves each request of the client; the connection channel comprises a connection established between the second front-end processor and the network gate, and a connection established between the second front-end processor and the server; the connecting channel is in an open state when a preset condition is met.
An aspect of an embodiment of the present invention further provides a cross-network data processing apparatus, for use in a second front-end processor, where the apparatus includes:
the receiving module is used for receiving the request file forwarded by the gateway; the request file in the network gate is from a first front-end processor, the request file is obtained by converting a first communication transmission protocol request message by the first front-end processor, the first communication transmission protocol request message is from a client, and the client and the first front-end processor are positioned in a first network;
The conversion module is used for converting the request file into a second communication transmission protocol request message;
the calling module is used for calling a server through the second communication transmission protocol request message so that the server generates and returns a target response message based on the second communication transmission protocol request message; the second front-end processor and the server are located in a second network, and the second network is different from the first network.
An aspect of an embodiment of the present invention further provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the data processing method as described above across a network when the computer program is executed.
An aspect of an embodiment of the present invention further provides a computer-readable storage medium comprising a memory, a processor and a computer program stored on the memory and executable on at least one processor, the at least one processor implementing the steps of a data processing method as described above across a network when the computer program is executed.
According to the cross-network data processing method, device, computer equipment and computer readable storage medium, the first front-end processor and the second front-end processor are introduced to perform the mutual conversion between the messages (such as the first communication transmission protocol request message and the second communication transmission protocol request message) and the request file, so that the cross-network data transmission efficiency is effectively improved.
Drawings
FIG. 1 schematically illustrates an environmental application schematic of a data processing method across a network according to an embodiment of the invention;
FIG. 2 schematically illustrates an example flow chart of the present invention implementing data processing across a network;
fig. 3 schematically illustrates a flowchart of converting a first communication transmission protocol request message into a request file in a cross-network data processing method according to a first embodiment of the present invention;
fig. 4 schematically illustrates a flowchart of processing a second communication transmission protocol request message by a second front-end processor in a cross-network data processing method according to a first embodiment of the present invention;
FIG. 5 schematically illustrates a flow chart of one-to-one correspondence of requests and responses in a method of data processing across a network according to a first embodiment of the present invention;
fig. 6 schematically illustrates a flowchart of processing a request file and a target response message in a second front-end processor in a cross-network data processing method according to a first embodiment of the present invention;
FIG. 7 schematically illustrates a flowchart of a service call performed in a server in a cross-network data processing method according to a first embodiment of the present invention;
FIG. 8 schematically illustrates a flow chart of a method of data processing across a network according to a second embodiment of the invention;
FIG. 9 schematically shows a block diagram of a data processing apparatus across a network according to a third embodiment of the invention;
fig. 10 schematically shows a hardware architecture diagram of a computer device adapted to implement a data processing method across a network according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that the descriptions of "first," "second," etc. in the embodiments of the present invention are for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be considered to be absent and not within the scope of protection claimed in the present invention.
In the description of the present invention, it should be understood that the numerical references before the steps do not identify the order in which the steps are performed, but are merely used to facilitate description of the present invention and to distinguish between each step, and thus should not be construed as limiting the present invention.
The inventors have appreciated that: based on data security considerations, a gatekeeper is typically used between two different networks to block direct TCP (Transmission Control Protocol ))/IP (Internet Protocol Address, internetworking protocol) connections from communicating directly based on the communication transmission protocol. Therefore, the data transmission between different networks is complicated and the communication efficiency is low every time the network gate is crossed.
To solve the above-mentioned problems, various embodiments are provided below, and each of the embodiments provided below can be used to achieve efficient data transmission between different networks, so as to achieve efficient call of clients located in the different networks to a server.
Fig. 1 schematically shows an environmental application schematic of a data processing method according to an embodiment of the invention.
As shown, it comprises a first network 1 and a second network 2, said second network 2 being different from said first network 1. Wherein the client 3 and the first front end processor 4 are located in the first network 1, and the second front end processor 6 and the server 7 are located in the second network 2. The client 3 may be connected to the first front-end processor 4 through the first network 1, and the first front-end processor 4 may be connected to the gatekeeper 5 through the first network 1; the gatekeeper 5 may be connected to the second front-end processor 6 through the second network 2, and the second front-end processor 6 may be connected to the server 7 through the second network 2. It should be noted that, the geographic location where the first network 1 is deployed is different from the geographic location where the second network 2 is deployed, or the security of the first network 1 is different from the security of the second network 2.
The first network 1 and the second network 2 may each comprise various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and/or the like. By way of example, the first network 1 and the second network 2 may comprise physical links, such as coaxial cable links, twisted pair cable links, fiber optic links, combinations thereof, and/or the like. The first network 1 and the second network 2 may also comprise wireless links, such as cellular links, satellite links, wi-Fi links and/or the like.
The client 3 may initiate a call request to the server 7 to obtain corresponding response data from the server 7. The client 3 may be a service system in a low density network. The service system in the low-density network may be understood as a service system in a network with a low server density and a low security level. The client 3 may comprise any type of device, such as: mobile devices, tablet devices, laptop computers, smart devices (e.g., smartwatches, smart glasses), virtual reality headphones, gaming devices, set-top boxes, digital streaming devices, robots, vehicle terminals, smart televisions, electronic book readers, MP4 (moving picture experts group audio layer IV) players, and the like.
The first front end processor 4 is used for converting between the message and the file. The first front end processor 4 may be any type of server.
And the gatekeeper 5 is used for forwarding files and checking files among different networks. The gatekeeper 5 may be an information security device that connects two independent networks using a solid state switching read-write medium with multiple control functions. Because the two independent networks are isolated through the network gate, no physical connection, logical connection and information transmission protocol exist between the networks, no information exchange according to the protocol exists, and no protocol ferry is performed only in the form of data files. Therefore, the network gate logically isolates and blocks all network connections with potential attacks on the internal network, so that external attackers cannot directly invade, attack or destroy the internal network, and the safety of each network is ensured.
And the second front end processor 6 is used for converting the messages and the files. The second head-end 6 may be any type of server.
And the server 7 is used for generating and returning corresponding response data according to the call request. The server 7 may comprise any type of server, such as: rack servers, blade servers, tower servers, or rack servers (including stand-alone servers, or a server cluster made up of multiple servers), and the like. In this embodiment, the service end 7 may be a service system in a high-density network. The service system in the high-density network may be understood as a service system in a network with a higher server density and a higher security level.
The application provides a data processing scheme crossing networks, and aims to realize the mutual conversion between a message and a file based on a communication transmission protocol through a first front-end processor and a second front-end processor, thereby realizing efficient data transmission between different networks.
In the invention, a thread pool is also configured in the second front-end processor, a thread model is optimized, and high-performance service call is provided.
In summary, the key points of the data processing method across networks in the present invention include, but are not limited to:
(1) And configuring a message-file conversion function in the first front-end processor and the second front-end processor, so as to realize the mutual conversion between data and files based on a communication transmission protocol, and further realize the mutual access intercommunication between services of different networks.
(2) And optimizing a thread model to realize high-performance service call.
(3) And the first front-end processor, the network gate and the second front-end processor are used for carrying out data verification on the respectively received request/response messages or files based on the communication transmission protocol, so that the management and control of the data transmission security among different networks are realized.
Example 1
In an embodiment, the cross-network data processing method can be applied to the military or public security industry.
Referring to fig. 2, a flowchart illustrating overall steps of a data processing method across networks according to a first embodiment of the present invention is shown. As shown in fig. 2, the data processing method across the network may include steps S200 to S204, wherein,
Step S200, a first communication transmission protocol request message sent by a client is received through a first front-end processor, the first communication transmission protocol request message is converted into a request file, and the request file is forwarded; the first front-end processor and the client are located in a first network.
The communication transmission protocol related to this embodiment includes, but is not limited to, HTTP protocol (Hyper Text Transfer Protocol ), HTTPs protocol (Hyper Text Transfer Protocol over SecureSocket Layer, hypertext transfer security protocol), qic protocol (low latency internet transport layer protocol based on UDP; UDP is User Datagram Protocol, user datagram protocol), SSL protocol (Secure Sockets Layer, secure socket protocol), and the like. In this embodiment, the first communication transmission protocol request packet may be an HTTP protocol request packet. In other embodiments, the first communication transmission protocol request message may also be an HTTPS protocol request message, a QUIC protocol request message, or an SSL protocol request message.
The file format of the request file includes, but is not limited to, JSON (JavaScript Object Notation, JS object profile) or XML (Extensible Markup Language ), etc.
In order to realize high-performance service, the first front-end processor bottom layer adopts Netty as a network communication framework, wherein the Netty framework is the framework of a high-level network using Java (computer programming language) for rapidly developing high-performance and high-reliability network servers and client programs. The first front end processor may process multiple requests simultaneously.
In order to achieve the mutual conversion between the first communication transmission protocol request message and the request file, referring to fig. 3, the converting the first communication transmission protocol request message into the request file may further include steps S300 to S304, where: step S300, a first file is created in a disk of the first front-end processor through the first front-end processor; step S302, analyzing the first communication transmission protocol request message to obtain the content in the first communication transmission protocol request message; and step S304, writing the content in the first communication transmission protocol request message into the first file to obtain the request file. In this embodiment, the first front end processor 4 is configured with a conversion function, and the conversion function of the first front end processor 4 implements the mutual conversion between the first communication transmission protocol request message and the request file, and the request file obtained by conversion is stored in the first front end processor 4. In order to improve the security of data transmission, the method further comprises: the first front end processor 4 checks the request file.
Step S202, receiving the request file forwarded by the first front end processor through a gateway, and forwarding the request file.
In order to improve the security of data transmission between different networks, the method further comprises: and checking the request file through the gatekeeper, and forwarding or intercepting the request file according to a checking result. In this embodiment, the file format of the request file, whether the request file includes an illegal keyword, whether the request file includes a sensitive word, whether the request file includes a virus, whether the request file includes a malicious insert program, and the like are checked by a security policy preset in the gatekeeper 5. If the verification result shows that the verification of the request file is passed, forwarding the request file to the second front-end processor 6; if the verification result shows that the verification of the request file fails, the request file is intercepted, and response information showing that the connection fails is returned to the client 3. And the data verification is carried out on the request file from multiple dimensions through the network gate, so that the safety of data transmission among different networks is effectively improved.
Step S204, receiving a request file forwarded by the gatekeeper through a second front-end processor, converting the request file into a second communication transmission protocol request message, and forwarding the second communication transmission protocol request message to a server, so that the server generates and returns a target response message based on the second communication transmission protocol request message; the second front-end processor and the server are located in a second network, and the second network is different from the first network.
Wherein, netty is adopted as a network communication framework at the bottom layer of the second front-end processor 6.
In order to enable efficient communication between different networks, as shown in fig. 4, the second front-end processor 6 is configured with a thread pool, where the thread pool includes an IO thread (In and Out thread, an input/output thread) and a service thread; the IO thread is configured to send the second communication transmission protocol request packet to the server 7. The method may further comprise: step S400, if the second communication transmission protocol request message is sent to the server through the IO thread, calling the service thread to wait for the server to return the target response message and blocking the service thread; and step S402, if the target response message is received within the preset time, waking up the service thread, converting the target response message into a target response file through the service thread, and forwarding the target response file to the gatekeeper through the IO thread. In this embodiment, the thread pool further includes other service threads, where the other service threads are used for time-consuming service processing, such as sensitive word verification, keyword filtering, virus killing, and the like. The IO threads are different from the business threads and other business threads. And through the cooperation of the IO thread and the service thread, the asynchronous communication is converted into synchronous communication, and a synchronous communication mechanism corresponding to the request and the response one by one is realized, so that high-concurrency service call is realized. And the service thread waits for the service end 7 to return the target response message and processes other services through other service threads, so that asynchronous communication of the IO thread is not blocked, and the second front-end processor 7 is facilitated to rapidly process the high-concurrency request file.
In an exemplary embodiment, the method may further include: and if the target response message is not received within the preset time, returning timeout response information representing calling timeout. Illustratively, if the service thread does not receive the target response message within 3 seconds, the service thread provides timeout response information to the client 3, so that the client 3 can do subsequent operations.
In order to guarantee a one-to-one correspondence of requests and responses under highly concurrent requests, referring to fig. 5, the method may further include: step S500, when the target response message is received and the service thread is awakened, a target identifier in the target response message is obtained, and whether the target identifier is the same as a preset identifier in the second communication transmission protocol request message is judged; and step S502, if the target identifier is the same as the preset identifier, determining that the target response message corresponds to the second communication transmission protocol request message, and further determining to provide the target response message to the client. In this embodiment, the first communication transmission protocol request packet carries a unique preset identifier; the request file can take the preset identification as the file name; the second communication transmission protocol request message also carries the preset identifier. According to the comparison of the target identifier and the preset identifier, the one-to-one correspondence between the request and the response can be ensured under the high concurrency environment without network connection.
In an exemplary embodiment, the target response message is a first communication transmission protocol response message. The method may further comprise: receiving a first communication transmission protocol response message provided by the server through a second front-end processor, converting the first communication transmission protocol response message into a response file, and forwarding the response file; receiving a response file forwarded by the second front-end processor through a gateway, and forwarding the response file; and receiving the response file forwarded by the gateway through the first front-end processor, converting the response file into a second communication transmission protocol response message, and forwarding the second communication transmission protocol response message to the client. In this embodiment, the conversion function of the "message-file" configured by the first front-end processor and the second front-end processor is used to implement the mutual conversion between the response message and the response file based on the communication transmission protocol, so as to implement the mutual access intercommunication between the services of different networks.
For better clarity, the processing of the request file and the target response message by the second front-end processor 6 in the present invention is illustrated with reference to fig. 6, and the processing of the request message by the service end 7 in the present invention is illustrated with reference to fig. 7, where the second front-end processor includes an IO thread and a service thread; the method comprises the following steps:
In step S600, the second front end processor 6 receives the request file through the IO thread.
In step S602, the second front end processor 6 analyzes the request file through the service thread to obtain the request data.
In step S604, the second front end processor 6 encapsulates the request data through its service thread to obtain a second communication transmission protocol request message.
In step S606, the second front end processor 6 sends the second communication transmission protocol request message to the server 7 through the IO thread.
Step S608 and step S700 are advanced.
The sending is to send the second communication transmission protocol request message to the Socket of the server 7 through the Socket of the second front-end processor based on the connection Channel (Netty Channel) in the open state.
In step S608, the second front end processor 6 invokes the service thread and waits for the target response message within a preset time.
In step S700, the server 7 receives the second communication transmission protocol request message and returns a target response message.
Referring to fig. 7, in the server 7, step S700 may include steps S702 to S708, wherein:
in step S702, the server 7 receives the second communication transmission protocol request message through its connection channel. In step S704, the server 7 calls the service interface (API, application Programming Interface) according to the second communication transmission protocol request message to obtain the target response message.
In step S706, the server 7 writes the target response message into its connection channel.
In step S708, the server 7 sends the target response message to the second front end processor 6 through its connection channel.
Referring to fig. 6, after step S608, the process advances to step S612.
Step S612, judging whether the target response message is received within the preset time.
If the target response message is received within the preset time, the process proceeds to step S614, otherwise, the process proceeds to step S616.
Step S614 wakes up the service thread, converts the target response message into a response file through the service thread, sends the response file to the IO thread, and sends the response file to the second front-end processor 6 through the IO thread.
In step S616, timeout response information is returned to the client 3.
In the server 7, the connection channel may also be encapsulated with the following functions: connected functions, unconnected functions, send functions, received functions, and cast functions.
In order to effectively reduce the loss of the system resources of the second front-end processor, the method further comprises: pooling the Netty connection channels in the second front-end processor to avoid continuously establishing the connection channels, specifically comprising: configuring a connection channel through the second front-end processor, wherein the connection channel serves each request of the client; the connection channel comprises a connection established between the second front-end processor and the network gate, and a connection established between the second front-end processor and the server; the connecting channel is in an open state when a preset condition is met. In this embodiment, the second front-end processor is configured with a connection pool, where the connection pool includes a plurality of connection channels corresponding to different clients. When the second front-end processor receives a first request sent by the client, the second front-end processor applies for a corresponding connection channel from the connection pool according to the first request, and keeps the connection channel in an open state when the connection channel meets the preset condition; when the second front-end processor receives a subsequent request sent by the client, if the connection channel still meets the preset condition, the connection channel is ensured to be still in an open state, the connection channel is reused, repeated establishment and closing of the connection channel are not needed, and energy consumption is saved. For example, if the preset condition indicates that the corresponding connection channel is closed after processing 100 requests. And multiplexing the connection channel when the second front-end processor receives the 100 th request sent by the client, and closing the connection channel after receiving a response corresponding to the 100 th request through the connection channel.
The invention provides a data processing method across networks, which is characterized in that a first communication transmission protocol request message sent by a client is received through a first front-end processor, and the first communication transmission protocol request message is converted into a request file; receiving the request file through a gateway and forwarding the request file; receiving a request file forwarded by the gateway through a second front-end processor, converting the request file into a second communication transmission protocol request message, and forwarding the second communication transmission protocol request message to a server so that the server responds to the second communication transmission protocol request message; the first front-end processor and the second front-end processor are introduced to perform the mutual conversion between the messages (such as the first communication transmission protocol request message and the second communication transmission protocol request message) and the request file, so that the data transmission efficiency across the network is effectively improved.
The embodiment of the invention has at least the following beneficial effects:
(1) On the premise of not developing the client and the server secondarily, a first front-end processor is introduced into a first network, and a second front-end processor is introduced into a second network to perform mutual conversion between messages and files, so that high-efficiency data transmission across networks is realized, high-efficiency call of the client positioned in different networks to the server is realized, and the problem of service call among different networks is effectively solved.
(2) A first front-end processor with high performance processing capability is introduced that is capable of simultaneously processing multiple requests of one or more clients at high concurrency.
(3) The synchronous communication mechanism is realized through the cooperation of the IO thread and the business thread, so that the high concurrency service call is realized.
Example two
In this embodiment, the second front end processor 6 is used as an execution body, and reference may be made to the first embodiment for technical details and effects.
Referring to fig. 8, a flowchart illustrating overall steps of a data processing method across networks according to a second embodiment of the present invention is shown. As shown in fig. 8, the method for processing data across networks may include steps S800 to S804, where:
step S800, receiving a request file forwarded by a gateway; the request file in the network gate is from a first front-end processor, the request file is obtained by converting a first communication transmission protocol request message by the first front-end processor, the first communication transmission protocol request message is from a client, and the client and the first front-end processor are positioned in the first network;
step S802, converting the request file into a second communication transmission protocol request message; and
Step S804, calling a server through the second communication transmission protocol request message, so that the server generates and returns a target response message based on the second communication transmission protocol request message; the second front-end processor and the server are located in a second network, and the second network is different from the first network.
As an example, the second front-end processor is configured with a thread pool, the thread pool including IO threads and business threads; the calling the server through the second communication transmission protocol request message comprises the following steps:
sending the second communication transmission protocol request message to the server through the IO thread;
calling the service thread to wait for the server to return the target response message and blocking the service thread; and
If the target response message is received within the preset time, waking up the service thread, converting the target response message into a target response file through the service thread, and forwarding the target response file to the gatekeeper through the IO thread.
As an example, the method further comprises:
and if the target response message is not received within the preset time, returning timeout response information representing calling timeout.
As an example, the method further comprises:
when the target response message is received and the service thread is awakened, a target identification of the target response message is obtained, and whether the target identification is the same as a preset identification in the second communication transmission protocol request message or not is judged; and
And if the target identifier is the same as the preset identifier, determining that the target response message corresponds to the second communication transmission protocol request message, and further determining to provide the target response message to the client.
As an example, the method further comprises:
configuring a connection channel through a second front-end processor, wherein the connection channel serves each request of the client; the connection channel comprises a connection established between the second front-end processor and the network gate, and a connection established between the second front-end processor and the server; the connecting channel is in an open state when a preset condition is met.
Example III
Fig. 9 schematically shows a block diagram of a cross-network data processing apparatus according to a fourth embodiment of the present invention, which may be divided into program modules, one or more of which are stored in a storage medium and executed by a processor to complete the embodiment of the present invention. Program modules in accordance with the embodiments of the present invention are directed to a series of computer program instruction segments capable of performing the specified functions, and the following description describes each program module in detail. The cross-network data processing apparatus 900 is used in a second front end processor.
As shown in fig. 9, the cross-network data processing apparatus 900 may include a receiving module 902, a converting module 904, and a calling module 906, wherein:
a receiving module 902, configured to receive a request file forwarded by a gatekeeper; the request file in the network gate is from a first front-end processor, the request file is obtained by converting a first communication transmission protocol request message by the first front-end processor, the first communication transmission protocol request message is from a client, and the client and the first front-end processor are positioned in a first network;
a conversion module 904, configured to convert the request file into a second communication transmission protocol request packet;
a calling module 906, configured to call a server through the second communication transmission protocol request packet, so that the server generates and returns a target response packet based on the second communication transmission protocol request packet; the second front-end processor and the server are located in a second network, and the second network is different from the first network.
Optionally, the second front-end processor is configured with a thread pool, and the thread pool includes an IO thread and a service thread. The calling module 906 is further configured to: sending the second communication transmission protocol request message to the server through the IO thread; calling the service thread to wait for the server to return the target response message and blocking the service thread; and if the target response message is received within the preset time, waking up the service thread, converting the target response message into a target response file through the service thread, and forwarding the target response file to the gatekeeper through the IO thread.
Optionally, the data processing apparatus 900 across a network may further include a timeout response module (not identified) configured to: and if the target response message is not received within the preset time, returning timeout response information representing calling timeout.
Optionally, the cross-network data processing apparatus 900 may further include a judging module (not identified) and a determining module (not identified), where:
the judging module is used for acquiring a target identifier of the target response message and judging whether the target identifier is the same as a preset identifier in the second communication transmission protocol request message when the target response message is received and the service thread is awakened; and
The determining module is configured to determine that the target response message corresponds to the second communication transmission protocol request message if the target identifier is the same as the preset identifier, and further determine to provide the target response message to the client.
Optionally, the cross-network data processing apparatus 900 may further include a configuration module (not identified) configured to configure, through a second front end processor, a connection channel, where the connection channel serves respective requests of the client; the connection channel comprises a connection established between the second front-end processor and the network gate, and a connection established between the second front-end processor and the server; the connecting channel is in an open state when a preset condition is met.
Example IV
Fig. 10 schematically shows a hardware architecture diagram of a computer device 10000 adapted to implement a data processing method across networks according to a fourth embodiment of the present invention. In this embodiment, the computer device 10000 is a device capable of automatically performing score calculation and/or information processing in accordance with an instruction set or stored in advance. For example, it may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server, or a rack server (including a stand-alone server, or a server cluster composed of a plurality of servers), a gateway, or the like. As shown in fig. 10, the computer device 10000 includes at least, but is not limited to: the memory 10010, processor 10020, network interface 10030 may be communicatively linked to each other via a system bus. Wherein:
memory 10010 includes at least one type of computer-readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, memory 10010 may be an internal storage module of computer device 10000, such as a hard disk or memory of computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like. Of course, the memory 10010 may also include both an internal memory module of the computer device 10000 and an external memory device thereof. In this embodiment, the memory 10010 is typically used to store an operating system and various types of application software installed on the computer device 10000, such as program codes of the data processing method across networks in the above embodiment. In addition, the memory 10010 may be used to temporarily store various types of data that have been output or are to be output.
The processor 10020 may be a central processing unit (Central Processing Unit, simply CPU), controller, microcontroller, microprocessor, or other data processing chip across a network in some embodiments. The processor 10020 is typically configured to control overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is configured to execute program codes or process data stored in the memory 10010.
The network interface 10030 may comprise a wireless network interface or a wired network interface, which network interface 10030 is typically used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, establish a data transmission channel and a communication link between the computer device 10000 and the external terminal, and the like. The network may be an Intranet (Intranet), the Internet (Internet), a global system for mobile communications (Global System of Mobile communicat input/output n, abbreviated as GSM), wideband code division multiple access (Wideband Code Divis input/output n Multiple Access, abbreviated as WCDMA), a 4G network, a 5G network, bluetooth (Bluetooth), wi-Fi, or other wireless or wired network.
It should be noted that fig. 10 only shows a computer device having components 10010-10030, but it should be understood that not all of the illustrated components are required to be implemented, and more or fewer components may be implemented instead.
In this embodiment, the data processing method across the network stored in the memory 10010 may also be divided into one or more program modules and executed by a processor (the processor 10020 in this embodiment) to complete the embodiment of the present invention.
Example five
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by at least one processor, performs the steps of the method for processing data across a network in the above embodiments.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of a computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card), etc. that are provided on the computer device. Of course, the computer-readable storage medium may also include both internal storage units of a computer device and external storage devices. In this embodiment, the computer readable storage medium is typically used to store an operating system and various types of application software installed on a computer device, such as program codes of the data processing method across the network in the above embodiment. Furthermore, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
It will be apparent to those skilled in the art that the modules or steps of the embodiments of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than what is shown or described, or they may be separately fabricated into individual integrated circuit modules, or a plurality of modules or steps in them may be fabricated into a single integrated circuit module. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims (12)

1. A method for processing data across a network, comprising:
Receiving a first communication transmission protocol request message sent by a client through a first front-end processor, converting the first communication transmission protocol request message into a request file, and forwarding the request file; wherein the first front-end processor and the client are located in a first network;
receiving a request file forwarded by the first front-end processor through a gateway, and forwarding the request file; and
Receiving a request file forwarded by the gateway through a second front-end processor, converting the request file into a second communication transmission protocol request message, and forwarding the second communication transmission protocol request message to a server, so that the server generates and returns a target response message based on the second communication transmission protocol request message; the second front-end processor and the server are located in a second network, and the second network is different from the first network;
the request file is checked through the gatekeeper, and the request file is forwarded or intercepted according to a check result;
the first front-end processor and the second front-end processor perform data verification on the received request/response messages or files based on the communication transmission protocol;
the second front-end processor is configured with a thread pool, and the thread pool comprises IO threads and business threads; the IO thread is used for sending the second communication transmission protocol request message to the server; the method further comprises the steps of:
If the second communication transmission protocol request message is sent to the server through the IO thread, calling the service thread to wait for the server to return the target response message and blocking the service thread; and
If the target response message is received within the preset time, waking up the service thread, converting the target response message into a target response file through the service thread, and forwarding the target response file to the gatekeeper through the IO thread.
2. The method for processing data across a network according to claim 1, wherein converting the first communication transmission protocol request message into a request file comprises:
creating a first file in a disk of the first front-end processor through the first front-end processor;
analyzing the first communication transmission protocol request message to obtain the content in the first communication transmission protocol request message; and
And writing the content in the first communication transmission protocol request message into the first file to obtain the request file.
3. The method of cross-network data processing according to claim 1, wherein the method further comprises:
and if the target response message is not received within the preset time, returning timeout response information representing calling timeout.
4. A method of cross-network data processing according to claim 3, wherein the method further comprises:
when the target response message is received and the service thread is awakened, a target identifier in the target response message is obtained, and whether the target identifier is the same as a preset identifier in the second communication transmission protocol request message is judged; and
And if the target identifier is the same as the preset identifier, determining that the target response message corresponds to the second communication transmission protocol request message, and further determining to provide the target response message to the client.
5. The method of cross-network data processing according to claim 1, wherein the method further comprises:
configuring a connection channel through the second front-end processor, wherein the connection channel serves each request of the client; the connection channel comprises a connection established between the second front-end processor and the network gate, and a connection established between the second front-end processor and the server; the connecting channel is in an open state when a preset condition is met.
6. A method for processing data across a network, for use in a second front-end processor, the method comprising:
Receiving a request file forwarded by a gateway; the request file in the network gate is from a first front-end processor, the request file is obtained by converting a first communication transmission protocol request message by the first front-end processor, the first communication transmission protocol request message is from a client, and the client and the first front-end processor are positioned in a first network;
converting the request file into a second communication transmission protocol request message; and
Calling a server through the second communication transmission protocol request message so that the server generates and returns a target response message based on the second communication transmission protocol request message; the second front-end processor and the server are located in a second network, and the second network is different from the first network;
the second front-end processor is configured with a thread pool, and the thread pool comprises IO threads and business threads; the calling the server through the second communication transmission protocol request message comprises the following steps:
sending the second communication transmission protocol request message to the server through the IO thread;
calling the service thread to wait for the server to return the target response message and blocking the service thread; and
If the target response message is received within the preset time, waking up the service thread, converting the target response message into a target response file through the service thread, and forwarding the target response file to the gatekeeper through the IO thread.
7. The method of cross-network data processing according to claim 6, further comprising:
and if the target response message is not received within the preset time, returning timeout response information representing calling timeout.
8. The method of cross-network data processing according to claim 6, further comprising:
when the target response message is received and the service thread is awakened, a target identification of the target response message is obtained, and whether the target identification is the same as a preset identification in the second communication transmission protocol request message or not is judged; and
And if the target identifier is the same as the preset identifier, determining that the target response message corresponds to the second communication transmission protocol request message, and further determining to provide the target response message to the client.
9. The method of cross-network data processing according to claim 6, further comprising:
Configuring a connection channel through a second front-end processor, wherein the connection channel serves each request of the client; the connection channel comprises a connection established between the second front-end processor and the network gate, and a connection established between the second front-end processor and the server; the connecting channel is in an open state when a preset condition is met.
10. A cross-network data processing apparatus for use in a second head-end, the apparatus comprising:
the receiving module is used for receiving the request file forwarded by the gateway; the request file in the network gate is from a first front-end processor, the request file is obtained by converting a first communication transmission protocol request message by the first front-end processor, the first communication transmission protocol request message is from a client, and the client and the first front-end processor are positioned in a first network;
the conversion module is used for converting the request file into a second communication transmission protocol request message;
the calling module is used for calling a server through the second communication transmission protocol request message so that the server generates and returns a target response message based on the second communication transmission protocol request message; the second front-end processor and the server are located in a second network, and the second network is different from the first network;
The second front-end processor is configured with a thread pool, and the thread pool comprises IO threads and business threads; the calling module is further configured to: sending the second communication transmission protocol request message to the server through the IO thread; calling the service thread to wait for the server to return the target response message and blocking the service thread; and if the target response message is received within the preset time, waking up the service thread, converting the target response message into a target response file through the service thread, and forwarding the target response file to the gatekeeper through the IO thread.
11. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor is adapted to implement the steps of the cross-network data processing method of any one of claims 1-5, or 6-9 when the computer program is executed.
12. A computer readable storage medium having stored therein a computer program executable by at least one processor to cause the at least one processor to perform the steps of the cross-network data processing method of any one of claims 1 to 5, or 6 to 9.
CN202111148217.3A 2021-09-29 2021-09-29 Cross-network data processing method and device Active CN114124929B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111148217.3A CN114124929B (en) 2021-09-29 2021-09-29 Cross-network data processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111148217.3A CN114124929B (en) 2021-09-29 2021-09-29 Cross-network data processing method and device

Publications (2)

Publication Number Publication Date
CN114124929A CN114124929A (en) 2022-03-01
CN114124929B true CN114124929B (en) 2024-03-29

Family

ID=80441645

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111148217.3A Active CN114124929B (en) 2021-09-29 2021-09-29 Cross-network data processing method and device

Country Status (1)

Country Link
CN (1) CN114124929B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115242422B (en) * 2022-05-17 2024-01-02 阿里巴巴(中国)有限公司 Data intercommunication processing method and device and informationized management system
CN115001846A (en) * 2022-06-28 2022-09-02 湖北天融信网络安全技术有限公司 Method, isolation device, device and medium for cross-network data transmission
CN116319733A (en) * 2022-09-09 2023-06-23 中央军委政治工作部军事人力资源保障中心 Cross-network service switching system and method

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106973050A (en) * 2017-03-23 2017-07-21 山东中创软件商用中间件股份有限公司 A kind of method and device of inter-network lock information sharing
CN108040060A (en) * 2017-12-18 2018-05-15 杭州优云软件有限公司 The method and device of inter-network lock communication
CN108933774A (en) * 2018-05-04 2018-12-04 北京明朝万达科技股份有限公司 Data interaction system and method
CN109309631A (en) * 2018-08-15 2019-02-05 新华三技术有限公司成都分公司 A kind of method and device based on universal network file system write-in data
CN109450948A (en) * 2018-12-27 2019-03-08 北京明朝万达科技股份有限公司 Data transmission method and device
CN110381093A (en) * 2019-09-03 2019-10-25 北京旷视科技有限公司 Method, apparatus, the system and electronic equipment of data transmission of data protocol conversion
CN111444029A (en) * 2020-03-05 2020-07-24 中国司法大数据研究院有限公司 Cross-domain micro-service calling method and system based on file exchange
CN113037875A (en) * 2021-05-24 2021-06-25 武汉众邦银行股份有限公司 Method for realizing asynchronous gateway in distributed real-time service system

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106973050A (en) * 2017-03-23 2017-07-21 山东中创软件商用中间件股份有限公司 A kind of method and device of inter-network lock information sharing
CN108040060A (en) * 2017-12-18 2018-05-15 杭州优云软件有限公司 The method and device of inter-network lock communication
CN108933774A (en) * 2018-05-04 2018-12-04 北京明朝万达科技股份有限公司 Data interaction system and method
CN109309631A (en) * 2018-08-15 2019-02-05 新华三技术有限公司成都分公司 A kind of method and device based on universal network file system write-in data
CN109450948A (en) * 2018-12-27 2019-03-08 北京明朝万达科技股份有限公司 Data transmission method and device
CN110381093A (en) * 2019-09-03 2019-10-25 北京旷视科技有限公司 Method, apparatus, the system and electronic equipment of data transmission of data protocol conversion
CN111444029A (en) * 2020-03-05 2020-07-24 中国司法大数据研究院有限公司 Cross-domain micro-service calling method and system based on file exchange
CN113037875A (en) * 2021-05-24 2021-06-25 武汉众邦银行股份有限公司 Method for realizing asynchronous gateway in distributed real-time service system

Also Published As

Publication number Publication date
CN114124929A (en) 2022-03-01

Similar Documents

Publication Publication Date Title
CN114124929B (en) Cross-network data processing method and device
CN110417766B (en) Protocol analysis method and device
CN112039824B (en) Communication method, system, device and computer readable storage medium
CN108965259B (en) Method and device for discovering and isolating malicious nodes of block chain
CN112104754B (en) Network proxy method, system, device, equipment and storage medium
US9313225B2 (en) Method for packet processing, electronic device and storage medium
US11907700B2 (en) Upgrading method and system, server, and terminal device
CN112468518B (en) Access data processing method and device, storage medium and computer equipment
CN113645234B (en) Honeypot-based network defense method, system, medium and device
CN110177128B (en) Data transmission system and method for establishing VPN connection, terminal and VPN proxy thereof
CN112187903B (en) Message pushing method and device and message service system
CN111885172A (en) Cross-network platform cascading method, computer storage medium and electronic device
CN114157607A (en) Media stream transmission method and system
CN110413354B (en) Method, equipment and storage medium for realizing interaction with third-party platform
CN113873057B (en) Data processing method and device
CN104202432B (en) Remote web management system and management method
CN112492055A (en) Method, device and equipment for redirecting transmission protocol and readable storage medium
CN113746851B (en) Proxy system and method supporting real-time analysis of GRPC request
CN115022178A (en) Network card switching method and device, electronic equipment and readable storage medium
CN114567678A (en) Resource calling method and device of cloud security service and electronic equipment
CN108984614B (en) Visual image rapid identification method based on big data environment
CN112565188B (en) Data access method, device, computer equipment and storage medium
CN111988319B (en) Access control method and device
CN114978643B (en) Communication method, network equipment and storage medium
CN114978786B (en) Method and device for converting third party interface into system standard interface

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Country or region after: China

Address after: Room 332, 3 / F, Building 102, 28 xinjiekouwei street, Xicheng District, Beijing 100088

Applicant after: Qianxin Technology Group Co.,Ltd.

Applicant after: Qianxin Wangshen information technology (Beijing) Co.,Ltd.

Address before: Room 332, 3 / F, Building 102, 28 xinjiekouwei street, Xicheng District, Beijing 100088

Applicant before: Qianxin Technology Group Co.,Ltd.

Country or region before: China

Applicant before: LEGENDSEC INFORMATION TECHNOLOGY (BEIJING) Inc.

CB02 Change of applicant information
GR01 Patent grant
GR01 Patent grant