CN115865886A - Cross-network data interaction method and device based on HTTP - Google Patents
Cross-network data interaction method and device based on HTTP Download PDFInfo
- Publication number
- CN115865886A CN115865886A CN202211498656.1A CN202211498656A CN115865886A CN 115865886 A CN115865886 A CN 115865886A CN 202211498656 A CN202211498656 A CN 202211498656A CN 115865886 A CN115865886 A CN 115865886A
- Authority
- CN
- China
- Prior art keywords
- data
- proxy
- task
- request
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Computer And Data Communications (AREA)
Abstract
The invention discloses a cross-network data interaction method and a device based on HTTP, wherein the method comprises the following steps: analyzing the data request, adding an agent mode mark for the data request and determining a data interface agent mode; acquiring data interface declaration information of a data processing platform, and packaging the data request added with the agent mode mark according to the data interface declaration information; sending the packaged data request to an agent server through an agent client according to the determined data interface agent mode, processing through a data processing platform, and returning a processing result; the data interface proxy mode comprises a data interface synchronous proxy mode and a data interface asynchronous proxy mode, and the data interface comprises a synchronous proxy data interface and an asynchronous proxy data interface, so that the parallel processing of a plurality of remote data requests of different types is realized, and the processing capacity and the processing speed are improved.
Description
Technical Field
The invention relates to the field of database technology and information technology Internet, in particular to a cross-network data interaction method and device based on HTTP.
Background
The application scene of the remote data request comprises two conditions, namely, a data request is initiated, and a server side can immediately return a data result; and after the request is initiated, the data result can not be returned immediately, and the processing time of the server is uncertain.
Currently, the prevalent way of remote data requests is by initiating an HTTP request to a remote server and responding by the remote server. The HTTP request is mainly divided into two processes of request and response.
In this way, the process of the client requesting data from the server by initiating an HTTP request must be continuous and uninterrupted, otherwise the HTTP request will fail. Meanwhile, in the process of establishing connection with the server and waiting for data return, the connection between the client and the server is in a blocking state, and other data request tasks can be processed only after the current data request task is processed, otherwise, other data request tasks cannot be processed and only can be waited. This approach is only suitable for application scenarios (e.g., data query scenarios) where a data request is sent and the processing result can be returned immediately; application scenarios (e.g., data analysis scenarios) where sending a data request may not be able to return processing results immediately are simply not applicable. Firstly, the process of requesting data from a server by initiating an HTTP request in the scene is not continuous, and there is a break in the middle, which may cause a data request based on the HTTP protocol to fail; secondly, the duration of processing the data request by the server cannot be determined, so that the connection between the client and the server is always in a blocked state, and other data request tasks are always queued and cannot be processed, so that the waiting time of the remote data request is greatly prolonged, and the processing efficiency of the data request task is reduced. Therefore, the application scene requirements cannot be completely met only by the existing HTTP request mode.
Therefore, a new method and a new system are needed to be introduced, which can be applied to an application scenario in which a processing result cannot necessarily be immediately returned when a data request is sent, so as to solve the technical problems in the prior art that the data request processing process cannot be interrupted, so that the data request task processing fails, the data request task cannot be processed in parallel, so that the processing efficiency is low, thereby realizing the parallel processing of a plurality of remote data request tasks, and the mixed processing of different types of remote data requests in different application scenarios, and improving the scenario adaptability, the processing capability and the response speed of the HTTP-based cross-network data interaction system.
Disclosure of Invention
In order to solve the technical problems mentioned above, the invention provides a cross-network data interaction method and device based on HTTP, based on HTTP protocol, parallel processing of different types of remote data requests, supervision of unfinished data request tasks and processing of completed data request tasks are realized by providing a data interface synchronous proxy mode and a data interface asynchronous proxy mode for different types of remote data requests and calling corresponding data interfaces according to the data interface proxy mode, so that the technical problems that in the prior art, the data request processing process cannot be interrupted to cause data request task processing failure and the data request tasks cannot be processed in parallel to cause low processing efficiency are solved, and further the scene adaptability, processing capacity and response speed of the cross-network data interaction system based on HTTP are improved.
The invention provides a cross-network data interaction method based on HTTP, which comprises the following steps:
s1, determining a data interface proxy mode: analyzing the data request, adding an agent mode mark for the data request and determining a data interface agent mode;
s2, data request preprocessing: acquiring data interface declaration information of a data processing platform, and packaging the data request added with the agent mode mark according to the data interface declaration information;
s3, processing the data request, sending the packaged data request to an agent server through an agent client according to the determined data interface agent mode, processing through a data processing platform, and returning a processing result;
the data interface proxy mode comprises a data interface synchronous proxy mode and a data interface asynchronous proxy mode;
the data interface comprises a synchronous proxy data interface and an asynchronous proxy data interface;
the agent mode mark comprises a synchronous agent mode mark and an asynchronous agent mode mark;
the data interface declaration information comprises a data interface path, a data interface request path and a data interface request parameter;
the data interface request path is generated by the data interface path and the proxy mode flag.
As described above, the steps of analyzing the data request, adding the proxy mode flag to the data request, and determining the proxy mode of the data interface include:
determining a request type, receiving a data request by the proxy client, and obtaining the data request type according to data request information;
adding an agent mode mark, adding the agent mode mark for the data request according to the data request type, and determining the data interface path;
determining a proxy mode, generating the data interface request path according to the proxy mode mark and the data interface path, and determining the data interface proxy mode;
when the data request type is a data query request, the added proxy mode is marked as the synchronous proxy mode, and the data interface proxy mode is the data interface synchronous proxy mode; when the data request type is a data analysis request, the added proxy mode is marked as the asynchronous proxy mode, and the data interface proxy mode is the data interface asynchronous proxy mode.
As described above, the data interface synchronization proxy method includes the steps of:
packaging the data request, wherein the proxy client packages the synchronous data request according to the synchronous proxy mode mark and the data interface declaration information;
sending a synchronous data request, wherein the proxy client sends the encapsulated synchronous data request to the proxy server and the synchronous data request is forwarded to the synchronous proxy data interface by the proxy server for processing;
and returning a processing result, wherein the proxy server side returns the processing result of the synchronous data request to the proxy client side after acquiring the processing result of the synchronous data request through the synchronous proxy data interface.
As described above, the data interface asynchronous proxy mode includes the following steps:
data request preprocessing: packaging data request information, adding a task ID and a task processing state mark for an asynchronous data request by the proxy client, and packaging the task ID, the task processing state mark, the data interface request path, the data interface request parameter and the asynchronous proxy mode mark according to the asynchronous proxy mode mark and the data interface declaration information to obtain packaging information and an asynchronous result callback path;
sending a request, wherein the proxy client sends the encapsulation information to the proxy server and the proxy server forwards the encapsulation information to the asynchronous proxy data interface for processing;
the proxy client side monitors the task processing state of the asynchronous data request at regular time according to the task processing state mark after sending the encapsulation information to the proxy server side;
returning a result and releasing data processing resources;
wherein the task processing state flags include complete and incomplete.
As described above, after the proxy client sends the encapsulation information to the proxy server, the step of regularly supervising the task processing state of the asynchronous data request according to the task processing state flag includes:
according to the task processing state mark, the proxy client side sends a task supervision request to the proxy server side at regular time, and the task processing state of the uncompleted asynchronous data request is inquired;
the proxy server receives the task supervision request and returns a task supervision processing result to the proxy client according to the processing result of the asynchronous data request, and if the proxy server has the processing result, the returned task supervision processing result is the processing result; if the processing result does not exist in the agent server, the returned task supervision processing result is task incompletion information;
and the proxy client receives the task supervision processing result, updates the task processing state mark according to the task supervision processing result, and updates the task processing state mark to be finished if the task supervision processing result is the processing result.
As described above, the step of returning the result and releasing the data processing resource is:
the proxy server side obtains the processing result through the asynchronous proxy data interface;
when the task supervision processing result obtained after the proxy client sends the task supervision request to the proxy server is the processing result, the proxy server returns the processing result to the proxy client through the asynchronous result callback path;
the proxy client receives and caches the processing result, and sends the task ID of the asynchronous data request to the proxy server;
and the proxy client and the proxy server delete the packaging information and the asynchronous result callback path according to the task ID respectively, and release data processing resources.
Correspondingly, the invention also provides a device for implementing the cross-network data interaction method based on the HTTP, and the device comprises a data proxy client unit and a data proxy server unit;
the data proxy client unit is deployed at a data caller and used for receiving and processing a data request of the data caller, forwarding the processed data request to the data proxy server unit, and forwarding a processing result returned by the data proxy server unit to the data caller;
the data agent server unit is deployed on a data platform side and used for receiving the data request forwarded by the data agent client unit, calling a data interface provided by the data platform side to process the data request and returning a processing result to the data agent client unit.
As described above, the data proxy client unit includes a task forwarding module, an unfinished task supervision module, and a finished task processing module; the data agent server unit comprises a task receiving module, a task checking module, a task state updating module and a completed task clearing module;
wherein the content of the first and second substances,
the data proxy client unit is used for adding a task ID and a task processing state mark to an asynchronous data request, packaging the task ID, the task processing state mark, the data interface request path, the data interface request parameter and the proxy mode mark according to the asynchronous proxy mode mark and the data interface declaration information to obtain packaging information and an asynchronous result callback path, and sending the packaging information to the data proxy server unit;
the uncompleted task supervision module is used for storing the submitted data request task in a local cache after the packaging information is sent to the data proxy server unit, regularly checking the local uncompleted task according to the task processing state mark, acquiring the uncompleted data request task processing state from the data proxy server unit, returning the result of the data request task and storing the result in the local cache if the data request task has a processing result in the data proxy server unit so as to prepare for the completed task processing module to continue processing the processing result;
the completed task processing module is used for regularly checking whether a result of an uncompleted task is updated in a local cache, returning a processing result to the data proxy client unit through a processing result callback interface provided by the data proxy client unit if the result exists, and releasing task processing resources of the data request according to the task ID;
the task receiving module is used for receiving and storing the encapsulation information and the task ID forwarded by the data agent client unit and forwarding a data request to a data interface of a data platform;
the inspection task module is used for inquiring the task processing state of a data request task when receiving the task supervision request sent by the data agent client unit; if the task processing state is marked as incomplete, returning incomplete information, and if the data request task is inquired to have a processing result, acquiring and returning the processing result;
the update task state module is used for sending the processing result to the data proxy server unit through an update result interface of the data proxy server unit after the data interface processes the data request, and updating the processing result to a local cache for query;
and the task clearing completed module is used for receiving a task processing completed notice sent by the data agent client unit and releasing data processing resources according to the task ID of the completed task.
By applying the technical scheme, the invention realizes the parallel processing of different types of remote data requests, the supervision of unfinished data request tasks and the processing of finished data request tasks by a data interface synchronous proxy mode and a data interface asynchronous proxy mode which are provided for different types of remote data requests and corresponding data interfaces based on the HTTP protocol, solves the technical problems that the data request processing process can not be interrupted to cause the processing failure of the data request tasks, the data request tasks can not be processed in parallel to cause the low processing efficiency and the like in the prior art, and improves the scene adaptability, the processing capability and the response speed of the cross-network data interaction system based on the HTTP.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart illustrating a cross-network data interaction method based on HTTP according to an embodiment of the present invention;
fig. 2 is a schematic flowchart illustrating a method for HTTP-based cross-network data interaction according to an embodiment of the present invention, for adding a proxy mode flag to a data request and determining a data interface proxy mode;
fig. 3 is a flowchart illustrating a data interface synchronization proxy manner of the HTTP-based cross-network data interaction method according to an embodiment of the present invention;
FIG. 4 is a flow chart illustrating a data interface asynchronous proxy mode of a cross-network data interaction method based on HTTP according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an HTTP-based cross-network data interaction apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a cross-network data interaction method based on HTTP (hyper text transport protocol), as shown in figure 1, the method comprises the following steps:
s101, determining a data interface proxy mode: and analyzing the data request, adding an agent mode mark for the data request and determining a data interface agent mode.
In this embodiment, the data interface proxy mode includes a data interface synchronous proxy mode and a data interface asynchronous proxy mode, and the proxy mode flag includes a synchronous proxy mode flag and an asynchronous proxy mode flag.
S102, data request preprocessing: and acquiring data interface declaration information of a data processing platform, and packaging the data request added with the agent mode mark according to the data interface declaration information.
In this embodiment, the data interface includes a synchronous proxy data interface and an asynchronous proxy data interface, and the data interface declaration information includes a data interface path, a data interface request parameter, and a proxy mode flag; the data interface request path is generated by the data interface path and the proxy mode flag.
For example, the synchronous proxy mode is labeled "/sync/", the asynchronous proxy mode is labeled "/async/", a data interface path of the synchronous proxy data interface provided by the data platform side is "/abc", a data interface path of the asynchronous proxy data interface is "/def", a data interface request path calling the synchronous proxy data interface through the synchronous proxy mode is "/sync/abc", and a data interface request path calling the asynchronous proxy data interface through the asynchronous proxy mode is "/async/def".
And S103, processing the data request, sending the packaged data request to an agent server through an agent client according to the determined data interface agent mode, processing through a data processing platform, and returning a processing result.
As shown in fig. 2, the steps of analyzing the data request, adding a proxy mode flag to the data request, and determining a data interface proxy mode are as follows:
s201, determining a request type, receiving a data request by the proxy client, and obtaining the data interface request parameter and the data request type according to data request information.
S202, adding an agent mode mark, adding the agent mode mark for the data request according to the data request type, and determining the data interface path.
S203, determining a proxy mode, generating the data interface request path according to the proxy mode mark and the data interface path, and determining the data interface proxy mode.
In this embodiment, when the data request type is a data query request, the added proxy mode is marked as the synchronous proxy mode, and the data interface proxy mode is the data interface synchronous proxy mode; when the data request type is a data analysis request, the added proxy mode is marked as the asynchronous proxy mode, and the data interface proxy mode is the data interface asynchronous proxy mode.
For example, the synchronous proxy mode is labeled as "/sync/", the asynchronous proxy mode is labeled as "/async/", the data interface path of the synchronous proxy data interface provided by the data platform side is "/abc", and the data interface path of the asynchronous proxy data interface is "/def";
when the data request type is a data query request, the added proxy mode is marked as "/sync/", through "/sync/", the proxy client and the proxy server can judge that the proxy mode of the data request is a synchronous proxy mode, and the called data interface request path of the synchronous proxy data interface is "/sync/abc";
when the data request type is a data analysis request, the added proxy mode is marked as "/async/", through "/async/", the proxy client and the proxy server can judge that the proxy mode of the data request is an asynchronous proxy mode, and the called data interface request path of the asynchronous proxy data interface is "/async/def".
As shown in fig. 3, the data interface synchronization proxy mode includes the following steps:
and S301, encapsulating the data request, wherein the proxy client encapsulates the synchronous data request according to the synchronous proxy mode mark and the data interface statement information.
S302, a synchronous data request is sent, the proxy client sends the encapsulated synchronous data request to the proxy server, and the synchronous data request is forwarded to the synchronous proxy data interface by the proxy server for processing.
And S303, returning a processing result, wherein the proxy server side returns the processing result of the synchronous data request to the proxy client side after acquiring the processing result of the synchronous data request through the synchronous proxy data interface.
As shown in fig. 4, the data interface asynchronous proxy mode includes the following steps:
s401, data request preprocessing: packaging data request information, adding a task ID and a task processing state mark for an asynchronous data request by the proxy client, and packaging the task ID, the task processing state mark, the data interface request path, the data interface request parameter and the asynchronous proxy mode mark according to the asynchronous proxy mode mark and the data interface declaration information to obtain packaging information and an asynchronous result callback path;
wherein the task processing status flag includes complete and incomplete.
S402, sending a request, wherein the proxy client sends the encapsulation information to the proxy server and the proxy server forwards the encapsulation information to the asynchronous proxy data interface for processing.
And S403, task supervision, wherein the proxy client side supervises the task processing state of the asynchronous data request at regular time according to the task processing state mark after sending the encapsulation information to the proxy server side.
Task supervision for efficient task of outstanding data requestsIn some embodiments of the present invention, after the proxy client sends the encapsulation information to the proxy server, the step of regularly supervising the task processing state of the asynchronous data request according to the task processing state flag specifically includes:
according to the task processing state mark, the proxy client side sends a task supervision request to the proxy server side at regular time, and the task processing state of the uncompleted asynchronous data request is inquired;
the proxy server receives the task supervision request and returns a task supervision processing result to the proxy client according to the processing result of the asynchronous data request, and if the proxy server has the processing result, the returned task supervision processing result is the processing result; if the proxy server does not have the processing result, the returned task supervision processing result is task incompletion information;
and the proxy client receives the task supervision processing result, updates the task processing state mark according to the task supervision processing result, and updates the task processing state mark to be finished if the task supervision processing result is the processing result.
S404, returning the result and releasing the data processing resource.
To return results accurately and to make efficient use of processing resourcesIn some embodiments of the present invention, the step of returning the result and releasing the data processing resource specifically includes:
the proxy server side obtains the processing result through the asynchronous proxy data interface;
when the task supervision processing result obtained after the proxy client sends the task supervision request to the proxy server is the processing result, the proxy server returns the processing result to the proxy client through the asynchronous result callback path;
the proxy client receives and caches the processing result, and sends the task ID of the asynchronous data request to the proxy server;
and the proxy client and the proxy server delete the packaging information and the asynchronous result callback path according to the task ID respectively, and release data processing resources.
By applying the technical scheme, the data request is analyzed, the agent mode mark is added to the data request, and the data interface agent mode is determined; acquiring data interface declaration information of a data processing platform, and packaging the data request added with the agent mode mark according to the data interface declaration information; according to the determined data interface proxy mode, the packaged data request is sent to the proxy server side through the proxy client side, processing is carried out through the data processing platform, and a processing result is returned, so that the data request processing method based on the HTTP protocol provides different data interface synchronous proxy modes, data interface asynchronous proxy modes and data interfaces for different types of remote data requests, parallel processing of the different types of remote data requests, supervision of uncompleted data request tasks and processing of completed data request tasks are achieved, the uncompleted data request tasks do not influence processing of other data request tasks, data request processing failure cannot be caused due to interruption of the data request processing process in the prior art, the technical problems that data request task processing failure is caused due to incapability of interrupting the data request processing process, processing efficiency is low due to incapability of parallelly processing the data request tasks and the like are solved, and the scene adaptability, processing capacity and response speed of the HTTP-based cross-network data interaction system are improved.
Corresponding to the cross-network data interaction method based on HTTP in the embodiment of the present invention, the present invention further discloses a cross-network data interaction device based on HTTP, as shown in fig. 5, the device includes a data proxy client unit 501 and a data proxy server unit 502;
the data proxy client unit 501 is disposed at the data caller, and is configured to receive and process a data request of the data caller, forward the processed data request to the data proxy server unit, and forward a processing result returned by the data proxy server unit to the data caller.
The data agent server unit 502 is deployed on the data platform side, and is configured to receive the data request forwarded by the data agent client unit, call a data interface provided by the data platform side to process the data request, and return a processing result to the data agent client unit.
In this embodiment, the data proxy client unit 501 includes a task forwarding module 501-1, an unfinished task monitoring module 501-2, and a finished task processing module 501-3; the data agent server unit comprises a task receiving module 502-1, a task checking module 502-2, a task state updating module 502-3 and a task clearing completed module 502-4;
the task forwarding module 501-1 is configured to encapsulate data request information, add a task ID and a task processing status flag to an asynchronous data request by the data proxy client unit, encapsulate the task ID, the task processing status flag, the data interface request path, the data interface request parameter, and the proxy mode flag according to the asynchronous proxy mode flag and the data interface declaration information, obtain encapsulation information and an asynchronous result callback path, and send the encapsulation information to the data proxy server unit;
the unfinished task supervision module 501-2 is configured to store the submitted data request task in a local cache after sending the encapsulation information to the data proxy server unit, periodically check a local unfinished task according to a task processing state flag, and obtain an unfinished data request task processing state from the data proxy server unit, and if the data request task has a processing result in the data proxy server unit, return the result of the data request task and store the result in the local cache for the finished task processing module to continue to process the processing result;
the completed task processing module 501-3 is configured to periodically check whether a result of an uncompleted task is updated in a local cache, and if the result exists, return a processing result to the data proxy client unit through a processing result callback interface provided by the data proxy client unit, and release a task processing resource of the data request according to the task ID;
the task receiving module 502-1 is configured to receive and store the encapsulation information and the task ID forwarded by the data agent client unit, and forward a data request to a data interface of a data platform;
the inspection task module 502-2 is configured to query a task processing state of a data request task when receiving the task supervision request sent by the data proxy client unit; if the task processing state is marked as incomplete, returning incomplete information, and if the data request task is inquired to have a processing result, acquiring and returning the processing result;
the update task state module 502-3 is configured to, after the data interface finishes processing the data request, send the processing result to the data proxy server unit through the update result interface of the data proxy server unit, and update the processing result to a local cache for query;
the task clearing completed module 502-4 is configured to receive a task processing completion notification sent by the data proxy client unit, and release data processing resources according to the task ID of the completed task.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (8)
1. An HTTP-based cross-network data interaction method, the method comprising:
s1, determining a data interface agent mode: analyzing the data request, adding an agent mode mark for the data request and determining a data interface agent mode;
s2, preprocessing a data request: acquiring data interface declaration information of a data processing platform, and packaging the data request added with the agent mode mark according to the data interface declaration information;
s3, processing the data request, sending the packaged data request to an agent server through an agent client according to the determined data interface agent mode, processing through a data processing platform, and returning a processing result;
the data interface proxy mode comprises a data interface synchronous proxy mode and a data interface asynchronous proxy mode;
the data interface comprises a synchronous proxy data interface and an asynchronous proxy data interface;
the agent mode mark comprises a synchronous agent mode mark and an asynchronous agent mode mark;
the data interface declaration information comprises a data interface path, a data interface request path and a data interface request parameter;
wherein the data interface request path is generated by the data interface path and the proxy mode flag.
2. The method of claim 1, wherein analyzing the data request, adding a proxy mode token to the data request, and determining a data interface proxy mode comprises:
determining a request type, receiving a data request by the proxy client, and obtaining the data request type according to data request information;
adding an agent mode mark, adding the agent mode mark for the data request according to the data request type, and determining the data interface path;
determining a proxy mode, generating the data interface request path according to the proxy mode mark and the data interface path, and determining the data interface proxy mode;
when the data request type is a data query request, the added proxy mode is marked as the synchronous proxy mode, and the data interface proxy mode is the data interface synchronous proxy mode; when the data request type is a data analysis request, the added proxy mode is marked as the asynchronous proxy mode, and the data interface proxy mode is the data interface asynchronous proxy mode.
3. The data interface synchronization proxy of claim 2, wherein the data interface synchronization proxy comprises the steps of:
packaging the data request, wherein the proxy client packages the synchronous data request according to the synchronous proxy mode mark and the data interface declaration information;
sending a synchronous data request, wherein the proxy client sends the encapsulated synchronous data request to the proxy server and the synchronous data request is forwarded to the synchronous proxy data interface by the proxy server for processing;
and returning a processing result, wherein the proxy server side returns the processing result of the synchronous data request to the proxy client side after acquiring the processing result of the synchronous data request through the synchronous proxy data interface.
4. The data interface asynchronous proxy mode of claim 2, wherein the data interface asynchronous proxy mode comprises the steps of:
data request preprocessing: packaging data request information, adding a task ID and a task processing state mark for an asynchronous data request by the proxy client, and packaging the task ID, the task processing state mark, the data interface request path, the data interface request parameter and the asynchronous proxy mode mark according to the asynchronous proxy mode mark and the data interface declaration information to obtain packaging information and an asynchronous result callback path;
sending a request, wherein the proxy client sends the packaging information to the proxy server and the proxy server forwards the packaging information to the asynchronous proxy data interface for processing;
the proxy client side monitors the task processing state of the asynchronous data request at regular time according to the task processing state mark after sending the encapsulation information to the proxy server side;
returning a result and releasing data processing resources;
wherein the task processing status flag includes complete and incomplete.
5. The method of claim 4, wherein after the proxy client sends the encapsulation information to the proxy server, the step of regularly supervising the task processing status of the asynchronous data request according to the task processing status flag comprises:
according to the task processing state mark, the proxy client side sends a task supervision request to the proxy server side at regular time, and the task processing state of the uncompleted asynchronous data request is inquired;
the proxy server receives the task supervision request and returns a task supervision processing result to the proxy client according to the processing result of the asynchronous data request, and if the proxy server has the processing result, the returned task supervision processing result is the processing result; if the proxy server does not have the processing result, the returned task supervision processing result is task incompletion information;
and the proxy client receives the task supervision processing result, updates the task processing state mark according to the task supervision processing result, and updates the task processing state mark to be finished if the task supervision processing result is the processing result.
6. The method of claim 4, wherein the step of returning the result and releasing the data processing resources comprises:
the proxy server side obtains the processing result through the asynchronous proxy data interface;
when the task supervision processing result obtained after the proxy client sends the task supervision request to the proxy server is the processing result, the proxy server returns the processing result to the proxy client through the asynchronous result callback path;
the proxy client receives and caches the processing result, and sends the task ID of the asynchronous data request to the proxy server;
and the proxy client and the proxy server delete the packaging information and the asynchronous result callback path according to the task ID respectively, and release data processing resources.
7. An apparatus for implementing the HTTP-based cross-network data interaction method of claim 1, the apparatus comprising a data proxy client unit and a data proxy server unit;
the data proxy client unit is deployed at a data caller and used for receiving and processing a data request of the data caller, forwarding the processed data request to the data proxy server unit, and forwarding a processing result returned by the data proxy server unit to the data caller;
the data agent server unit is deployed on a data platform side and used for receiving the data request forwarded by the data agent client unit, calling a data interface provided by the data platform side to process the data request and returning a processing result to the data agent client unit.
8. The apparatus of claim 7, wherein the data proxy client unit comprises a task forwarding module, an incomplete task supervision module, a completed task processing module; the data agent server unit comprises a task receiving module, a task checking module, a task state updating module and a completed task clearing module;
wherein the content of the first and second substances,
the data proxy client unit is used for adding a task ID and a task processing state mark to an asynchronous data request, packaging the task ID, the task processing state mark, the data interface request path, the data interface request parameter and the proxy mode mark according to the asynchronous proxy mode mark and the data interface declaration information to obtain packaging information and an asynchronous result callback path, and sending the packaging information to the data proxy server unit;
the unfinished task supervision module is used for storing the submitted data request task in a local cache after sending the packaging information to the data proxy server unit, regularly checking the local unfinished task according to the task processing state mark, acquiring the unfinished data request task processing state from the data proxy server unit, returning the result of the data request task and storing the result into the local cache if the data request task has a processing result in the data proxy server unit so as to prepare for the finished task processing module to continue to process the processing result;
the completed task processing module is used for regularly checking whether a result of an uncompleted task is updated in a local cache, returning a processing result to the data proxy client unit through a processing result callback interface provided by the data proxy client unit if the result exists, and releasing task processing resources of the data request according to the task ID;
the task receiving module is used for receiving and storing the encapsulation information and the task ID forwarded by the data agent client unit and forwarding a data request to a data interface of a data platform;
the inspection task module is used for inquiring the task processing state of a data request task when receiving the task supervision request sent by the data agent client unit; if the task processing state is marked as incomplete, returning incomplete information, and if the data request task is inquired to have a processing result, acquiring and returning the processing result;
the update task state module is used for sending the processing result to the data proxy server unit through an update result interface of the data proxy server unit after the data interface processes the data request, and updating the processing result to a local cache for query;
and the task clearing completed module is used for receiving a task processing completed notification sent by the data agent client unit and releasing data processing resources according to the task ID of the completed task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211498656.1A CN115865886B (en) | 2022-11-28 | 2022-11-28 | HTTP-based cross-network data interaction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211498656.1A CN115865886B (en) | 2022-11-28 | 2022-11-28 | HTTP-based cross-network data interaction method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115865886A true CN115865886A (en) | 2023-03-28 |
| CN115865886B CN115865886B (en) | 2023-07-18 |
Family
ID=85666988
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211498656.1A Active CN115865886B (en) | 2022-11-28 | 2022-11-28 | HTTP-based cross-network data interaction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115865886B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116431316A (en) * | 2023-06-06 | 2023-07-14 | 阿里巴巴（中国）有限公司 | Task processing method, system, platform and automatic question-answering method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1722663A (en) * | 2004-07-13 | 2006-01-18 | 联想(北京)有限公司 | A proxy server system and method for realizing proxy communication thereof |
| WO2016192556A1 (en) * | 2015-05-29 | 2016-12-08 | 北京金山安全软件有限公司 | Interface invoking method, device and terminal |
| CN107517254A (en) * | 2017-08-22 | 2017-12-26 | 北京梅泰诺通信技术股份有限公司 | A kind of dynamic data demand processing system and method |
| CN111490997A (en) * | 2019-01-25 | 2020-08-04 | 北京京东尚科信息技术有限公司 | Task processing method, agent system, service system and electronic equipment |
| CN113835911A (en) * | 2021-11-23 | 2021-12-24 | 深圳市明源云科技有限公司 | Intranet penetration agent method, system, host and computer readable storage medium |
-
2022
- 2022-11-28 CN CN202211498656.1A patent/CN115865886B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1722663A (en) * | 2004-07-13 | 2006-01-18 | 联想(北京)有限公司 | A proxy server system and method for realizing proxy communication thereof |
| WO2016192556A1 (en) * | 2015-05-29 | 2016-12-08 | 北京金山安全软件有限公司 | Interface invoking method, device and terminal |
| CN107517254A (en) * | 2017-08-22 | 2017-12-26 | 北京梅泰诺通信技术股份有限公司 | A kind of dynamic data demand processing system and method |
| CN111490997A (en) * | 2019-01-25 | 2020-08-04 | 北京京东尚科信息技术有限公司 | Task processing method, agent system, service system and electronic equipment |
| CN113835911A (en) * | 2021-11-23 | 2021-12-24 | 深圳市明源云科技有限公司 | Intranet penetration agent method, system, host and computer readable storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116431316A (en) * | 2023-06-06 | 2023-07-14 | 阿里巴巴（中国）有限公司 | Task processing method, system, platform and automatic question-answering method |
| CN116431316B (en) * | 2023-06-06 | 2023-11-14 | 阿里巴巴（中国）有限公司 | Task processing method, system, platform and automatic question-answering method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115865886B (en) | 2023-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6164747B2 (en) | Method for flow control in a collaborative environment and for reliable communication | |
| US20140244721A1 (en) | Real-time communications using a restlike api | |
| US20040001514A1 (en) | Remote services system communication module | |
| CN104219316A (en) | Method and device for processing call request in distributed system | |
| US20030212738A1 (en) | Remote services system message system to support redundancy of data flow | |
| CN112671697B (en) | Data processing method, device and system of comprehensive monitoring system | |
| CN111343237A (en) | Server cluster communication method, communication device and computer storage medium | |
| CN113162848B (en) | Method, device, gateway and medium for realizing block chain gateway | |
| CN113517985B (en) | File data processing method and device, electronic equipment and computer readable medium | |
| CN115865886A (en) | Cross-network data interaction method and device based on HTTP | |
| WO2023221452A1 (en) | Packet processing system and method, device, and storage medium | |
| CN113986501A (en) | Real-time database API (application program interface) uninterrupted calling method, system, storage medium and server | |
| US20090172122A1 (en) | Message Transmission Method, Message Transmission Device, and Storage Medium Recorded with Message Transmission Program | |
| CN110582757A (en) | Intermediate device for network routing of data messages | |
| CN111901308A (en) | Information interaction method | |
| JPH10320362A (en) | Method for transmitting communication in network including communication service and network for using the same | |
| CN115065686B (en) | Configuration method, device and system of distributed load balancing system | |
| CN116366644A (en) | Message sending method and device, electronic equipment and storage medium | |
| CN111301789A (en) | Application software packaging method and device | |
| CN114422100A (en) | Up-and-down connection processing system, computer equipment and medium of national standard signaling service terminal | |
| CN112187916B (en) | Cross-system data synchronization method and device | |
| CN113783826B (en) | Micro-service exception handling method and device | |
| CN113722126A (en) | Request processing method and device | |
| CN113542324A (en) | Message pushing method and device | |
| CN111262773A (en) | Message synchronization method and data center |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |