CN113132458A - Abnormal handling method and system based on flow replication - Google Patents
Abnormal handling method and system based on flow replication Download PDFInfo
- Publication number
- CN113132458A CN113132458A CN202110251159.0A CN202110251159A CN113132458A CN 113132458 A CN113132458 A CN 113132458A CN 202110251159 A CN202110251159 A CN 202110251159A CN 113132458 A CN113132458 A CN 113132458A
- Authority
- CN
- China
- Prior art keywords
- interaction
- replication
- order
- data
- party system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000010076 replication Effects 0.000 title claims abstract description 59
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 29
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000003993 interaction Effects 0.000 claims abstract description 71
- 238000012544 monitoring process Methods 0.000 claims abstract description 9
- 230000002452 interceptive effect Effects 0.000 claims description 13
- 230000005856 abnormality Effects 0.000 claims description 3
- 235000014510 cooky Nutrition 0.000 claims description 3
- 238000003672 processing method Methods 0.000 claims 1
- 238000012937 correction Methods 0.000 abstract description 3
- 238000012545 processing Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/083—Shipping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0633—Lists, e.g. purchase orders, compilation or processing
- G06Q30/0635—Processing of requisition or of purchase orders
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
Abstract
The invention discloses an exception handling method and system based on flow replication, wherein the method comprises the following steps: interacting with a third party system based on the order; performing flow replication on the interaction to obtain replication information; monitoring the order; judging whether abnormal interaction data is monitored; and if so, extracting a corresponding interaction request from the replication information, and retransmitting the interaction request to the third-party system. By carrying out flow replication on interaction with a third-party system, when abnormal data occurs, extracting an interaction request from replication information, and retransmitting the interaction request to the third-party system to obtain accurate third-party data so as to correct the abnormal data, so that the steps of manual examination and abnormal data correction are reduced, and the abnormal data processing efficiency is improved; meanwhile, the problem of data accuracy is solved, the order is monitored in real time, and the problem of data instantaneity can be solved, so that the user experience is improved.
Description
Technical Field
The invention relates to the technical field of computers, in particular to an exception handling method and system based on flow replication.
Background
In the cross-platform interaction, interaction with a third-party system is usually required, for example, an order of an e-commerce is interacted with a payment platform of a third party to pay and obtain payment information, and then, for example, the order is interacted with a logistics platform to exchange logistics information. However, system errors inevitably occur between different platforms, which causes situations such as repeated information, lost orders, system error notification and the like, and causes data errors of orders. Manual verification is often required, resulting in low work efficiency and poor user experience.
For example, a user initiates a flow of an order, pays and the like, submits payment of a payment platform of a third party, and obtains a returned payment result, but the background does not receive an asynchronous payment result notification, or cannot process the received asynchronous payment result notification in time due to system abnormality. The accuracy of the order data is high, but the accuracy of the order data depends on the accuracy of a third-party system on one hand, and on the other hand, some program exceptions in a background can also cause that the data of the third-party system cannot be received.
Disclosure of Invention
In order to solve the technical problems in the prior art, the invention provides an exception handling method and system based on traffic replication, which performs traffic replication on interaction with a third-party system, extracts an interaction request from replication information when an exception occurs, and retransmits the interaction request to the third-party system to obtain accurate third-party data.
The invention discloses an exception handling method based on flow replication, which comprises the following steps: interacting with a third party system based on the order; performing flow replication on the interaction to obtain replication information; monitoring the order; judging whether abnormal interaction data is monitored; and if so, extracting a corresponding interaction request from the replication information, and retransmitting the interaction request to the third-party system.
Preferably, the method of the present invention further comprises a method of interacting with a third party system via a Nginx server: interacting with a third-party system through an Nginx server based on the order; traffic replication of the interaction is performed by the Nginx server.
Preferably, the means for flow replication comprises: GoReplacy or TCPcopy.
Preferably, the copy information is stored in a memory, and whether abnormal data is monitored is judged; if yes, extracting the interaction request from the copy information of the memory, and retransmitting the interaction request to the third-party system.
Preferably, the storage includes an ES storage, and the storage format of the copy information includes one or a combination of the following: data table, parameter table and configuration table.
Preferably, the replication information includes Cookie data corresponding to the http request, a reference request, and a request time.
The invention also provides a system for realizing the exception handling method, which comprises an order module, an interaction module, a flow replication module and a scheduling module;
the interaction module is used for interacting with a third-party system based on an order;
the flow replication module is used for performing flow replication on the interaction to obtain replication information;
the order module is used for storing and managing orders and order data;
the scheduling module is used for monitoring the order and judging whether abnormal interaction data is monitored; and if so, extracting a corresponding interaction request from the replication information, and retransmitting the interaction request to the third-party system.
Preferably, the interaction module interacts with a third-party system through a nginnx server, and the traffic replication module performs traffic replication on the interaction through the nginnx server.
Preferably, the interaction module further comprises a service application, wherein the service application is used for interacting with the user and sending an interaction request to the third-party system through the Nginx server based on the order;
and when receiving the interactive data of the third-party system, storing the interactive data in order information of the order module.
Preferably, the scheduling module monitors the state of the order in real time based on a distributed method.
Compared with the prior art, the invention has the beneficial effects that: by carrying out flow replication on interaction with a third-party system, when abnormal data occurs, extracting an interaction request from replication information, and retransmitting the interaction request to the third-party system to obtain accurate third-party data so as to correct the abnormal data, so that the steps of manual examination and abnormal data correction are reduced, and the abnormal data processing efficiency is improved; meanwhile, the problem of data accuracy is solved, the order is monitored in real time, and the problem of data instantaneity can be solved, so that the user experience is improved.
Drawings
FIG. 1 is a flow chart of a traffic replication based exception handling method of the present invention;
FIG. 2 is a logical block diagram of the system of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
The invention is described in further detail below with reference to the attached drawing figures:
an exception handling method based on traffic replication, as shown in fig. 1, includes:
step 101: interacting with a third party system based on the order. For example, but not limited to, a payment interaction is performed to a payment platform of the third-party system based on the order, or logistics information is requested from a logistics platform of the third-party system.
Step 102: and carrying out flow replication on the interaction to obtain replication information. Traffic replication can be done using the following tools: GoReplacy or TCPcopy.
In a specific embodiment, the traffic replication is performed on the request of the third-party system, and the request may include parameters such as Cookie data, an entry request, and a request time, but is not limited thereto.
Step 103: and monitoring the order.
Step 104: and judging whether abnormal interaction data is monitored. The abnormal interaction data comprises error interaction data or interaction data which is not received for a long time, wherein the interaction data refers to data returned by a third-party system.
If yes, go to step 105: and extracting a corresponding interaction request from the copy information, and retransmitting the interaction request to the third-party system. The request traffic corresponding to the abnormal data may be retransmitted once to retransmit the interactive request, or the parameters of the interactive request may be extracted from the request traffic to generate the interactive request and then transmit the interactive request.
By carrying out flow replication on interaction with a third-party system, when abnormal data occurs, extracting an interaction request from replication information, and retransmitting the interaction request to the third-party system to obtain accurate third-party data so as to correct the abnormal data, so that the steps of manual examination and abnormal data correction are reduced, and the abnormal data processing efficiency is improved; meanwhile, the problem of data accuracy is solved, the order is monitored in real time, and the problem of data instantaneity can be solved, so that the user experience is improved.
The GoRelay is an open source network monitoring tool, can capture and replay real-time HTTP traffic to a testing environment, provides a simple method for a user to reuse existing traffic for testing, and can analyze and record the traffic under the conditions that a product infrastructure is not changed and the existing traffic is not influenced, so that the confidence of the user in code deployment, configuration and infrastructure change is enhanced. TCPCopy is a request replication tool, which can lead online traffic into a test system, and is applied to systems such as an advertisement delivery system, a Nginx hmux protocol and the like, and TCPCoporcent of TCPCopy runs on an online server and is used for capturing online request data packets.
Wherein, can interact with the third party system through the Nginx server: interacting with a third-party system through an Nginx server based on the order; traffic replication of the interaction is performed by the Nginx server. And GoReplacay or TCPcopy may be deployed on a Nginx server. The Nginx is a high-performance HTTP and reverse proxy WEB server and is generally applied to a website deployment architecture.
Wherein, the copy information can be stored in a memory, such as an ES memory, and whether abnormal data is monitored is judged; if yes, extracting the interaction request from the copy information of the memory, and retransmitting the interaction request to the third-party system. The storage format of the copy information comprises one or a combination of the following: data table, parameter table and configuration table.
Example 1
Successfully creating an order in the mall, completing payment and carrying out logistics distribution;
the third-party logistics system informs the order system, because the order system is in fault, the request cannot be received, the logistics system retransmits the request for several times at intervals, or the request cannot be received, so the request is not retransmitted any more, the order is signed by the logistics, but the system is still in an unfinished state, the user can be queried by the third-party logistics system, the user experience is poor, and even the user complaint is generated;
carrying out flow replication through GoRelay and storing;
after the abnormal order is monitored, the request flow is extracted from the copied information and reissued once again, so that the accuracy and timeliness of the order state are guaranteed.
Example 2
A user initiates an order and a payment process, and a background informs a third-party system to carry out payment;
the third-party system returns the payment result, but the asynchronous payment result notification of the background is not received, or the asynchronous payment result notification cannot be processed in time due to system abnormality after being received;
the abnormal flow is copied and stored through the TCPcopy, and when a payment result is not received within a certain time, the request is retransmitted once, so that the accuracy and timeliness of the order state are guaranteed.
As shown in fig. 2, the present invention further provides a system for implementing the above method: the system comprises an order module 1, an interaction module 2, a flow replication module 3 and a scheduling module 4;
the interaction module 2 is used for interacting with a third-party system based on the order;
the flow replication module 3 is used for performing flow replication on the interaction to obtain replication information;
the order module 1 is used for storing and managing orders and order data;
and the scheduling module 4 is used for monitoring the order and judging whether abnormal interactive data is monitored, if so, extracting a corresponding interactive request from the copied information and retransmitting the interactive request to the third-party system.
The interaction module 2 interacts with the third-party system 9 through the Nginx server 21, and the traffic replication module 3 performs traffic replication on the interaction through the Nginx server 21. The interaction module 4 further comprises a service application 22, wherein the service application 22 is configured to interact with the user and send an interaction request to the third-party system 9 through the Nginx server 21 based on the order; when receiving the interactive data of the third party system 9, the interactive data is stored in the order information of the order module 1.
The system of the invention can also comprise an ES memory 5, the ES memory 5 is used for storing the copy information, and the scheduling module 4 can monitor the state of the order in real time based on distributed application. The scheduling modules are distributed on different computers, and the tasks are completed together through a network, so that the uninterrupted operation of the service for 24 hours is ensured, the data with abnormal order states are monitored in real time, the original request flow of a third-party system is obtained from the ES memory 5, the data states are updated in time, and the like, and the accuracy and timeliness of the order data are ensured.
In one embodiment, a distributed scheduling module is employed: and uniformly managing the configuration data and providing a uniform input interface of the configuration data. And load balancing is realized, automatic expansion of distributed application is realized, namely hosts are dynamically added or deleted, and failover is realized. The method ensures the stable operation of the distributed application, provides a background monitoring interface, and can see the operation condition of the distributed application in real time through a built-in command, and comprises the following steps: how much data the process has processed; how many exceptions are present, details of the exception; starting time; the allocation of processes on the host, etc.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. An exception handling method based on traffic replication, the method comprising:
interacting with a third party system based on the order;
performing flow replication on the interaction to obtain replication information;
monitoring the order;
judging whether abnormal interaction data is monitored;
and if so, extracting a corresponding interaction request from the replication information, and retransmitting the interaction request to the third-party system.
2. The exception handling method of claim 1 further comprising a method of interacting with a third party system through a Nginx server:
interacting with a third-party system through an Nginx server based on the order;
traffic replication of the interaction is performed by the Nginx server.
3. The exception handling method of claim 1 wherein the means for traffic replication comprises: GoReplacy or TCPcopy.
4. The abnormality processing method according to claim 1, characterized in that the duplicated information is stored in a memory, and it is judged whether abnormal data is monitored;
if yes, extracting the interaction request from the copy information of the memory, and retransmitting the interaction request to the third-party system.
5. The exception handling method of claim 4, wherein said memory comprises an ES storage, and wherein a storage format of said copy information comprises one or a combination of: data table, parameter table and configuration table.
6. The exception handling method according to claim 1, wherein the replication information includes Cookie data corresponding to an http request, an entry request, and a request time.
7. A system for implementing the exception handling method of any one of claims 1 to 6, comprising an order module, an interaction module, a traffic replication module and a scheduling module;
the interaction module is used for interacting with a third-party system based on an order;
the flow replication module is used for performing flow replication on the interaction to obtain replication information;
the order module is used for storing and managing orders and order data;
the scheduling module is used for monitoring the order and judging whether abnormal interaction data is monitored; and if so, extracting a corresponding interaction request from the replication information, and retransmitting the interaction request to the third-party system.
8. The system of claim 7, wherein the interaction module interacts with a third party system through a Nginx server, and wherein the traffic replication module performs traffic replication of the interaction through the Nginx server.
9. The system of claim 8, wherein the interaction module further comprises a business application for interacting with the user and sending an interaction request to the third party system through the Nginx server based on the order;
and when receiving the interactive data of the third-party system, storing the interactive data in order information of the order module.
10. The system of claim 7, wherein the scheduling module monitors the status of the order in real time based on a distributed approach.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110251159.0A CN113132458A (en) | 2021-03-08 | 2021-03-08 | Abnormal handling method and system based on flow replication |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110251159.0A CN113132458A (en) | 2021-03-08 | 2021-03-08 | Abnormal handling method and system based on flow replication |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113132458A true CN113132458A (en) | 2021-07-16 |
Family
ID=76772823
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110251159.0A Pending CN113132458A (en) | 2021-03-08 | 2021-03-08 | Abnormal handling method and system based on flow replication |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113132458A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2390947A1 (en) * | 2001-07-06 | 2003-01-06 | General Electric Company | System for electronically submitting and monitoring copy reproduction jobs |
| AU2006331932A1 (en) * | 2005-12-19 | 2007-07-05 | Commvault Systems, Inc. | Systems and methods for performing data replication |
| CN106453299A (en) * | 2016-09-30 | 2017-02-22 | 北京奇虎科技有限公司 | Network security monitoring method and device, and cloud WEB application firewall |
| US20170279817A1 (en) * | 2016-03-25 | 2017-09-28 | Arbor Networks, Inc. | System and method for retrospective network traffic analysis |
| CN109711843A (en) * | 2018-12-29 | 2019-05-03 | 创发科技有限责任公司 | Payment process monitoring method and system |
| CN109785042A (en) * | 2018-12-13 | 2019-05-21 | 深圳平安财富宝投资咨询有限公司 | Abnormality eliminating method, server, storage medium and the device of distributed deployment |
-
2021
- 2021-03-08 CN CN202110251159.0A patent/CN113132458A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2390947A1 (en) * | 2001-07-06 | 2003-01-06 | General Electric Company | System for electronically submitting and monitoring copy reproduction jobs |
| AU2006331932A1 (en) * | 2005-12-19 | 2007-07-05 | Commvault Systems, Inc. | Systems and methods for performing data replication |
| US20170279817A1 (en) * | 2016-03-25 | 2017-09-28 | Arbor Networks, Inc. | System and method for retrospective network traffic analysis |
| CN106453299A (en) * | 2016-09-30 | 2017-02-22 | 北京奇虎科技有限公司 | Network security monitoring method and device, and cloud WEB application firewall |
| CN109785042A (en) * | 2018-12-13 | 2019-05-21 | 深圳平安财富宝投资咨询有限公司 | Abnormality eliminating method, server, storage medium and the device of distributed deployment |
| CN109711843A (en) * | 2018-12-29 | 2019-05-03 | 创发科技有限责任公司 | Payment process monitoring method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111726420A (en) | Communication method, device, equipment and storage medium based on RPA | |
| CN109818962B (en) | Service data processing method and device, electronic equipment and storage medium | |
| KR101416280B1 (en) | Event handling system and method | |
| CN111338893A (en) | Process log processing method and device, computer equipment and storage medium | |
| CN112202635B (en) | Link monitoring method and device, storage medium and electronic device | |
| CN104182683A (en) | Method, device and system for monitoring online transactions | |
| US9823999B2 (en) | Program lifecycle testing | |
| CN112416512A (en) | Mirror image construction method, server, client, storage medium and system | |
| US11531612B2 (en) | Methods for providing an enterprise synthetic monitoring framework | |
| CN111586022A (en) | Firewall opening verification method, electronic device, computer equipment and storage medium | |
| CN109522202B (en) | Software testing method and device | |
| CN116384993B (en) | Method and system for realizing high consistency of order payment state based on cloud payment center | |
| US20230037390A1 (en) | Techniques to provide streaming data resiliency utilizing a distributed message queue system | |
| CN110875832B (en) | Abnormal service monitoring method, device and system and computer readable storage medium | |
| CN111698301A (en) | Service management method, device and storage medium for ensuring service continuation | |
| CN113132458A (en) | Abnormal handling method and system based on flow replication | |
| CN111581002A (en) | Automatic fault reporting method, device and equipment for server fault | |
| CN113810379B (en) | Exception handling method and multi-service system | |
| CN113485902A (en) | Method, device, equipment and computer readable medium for testing service platform | |
| CN111552907A (en) | Message processing method, device, equipment and storage medium | |
| CN112540771A (en) | Automated operation and maintenance method, system, equipment and computer readable storage medium | |
| CN112734052B (en) | Fault repairing method and system | |
| CN111756748B (en) | Data interaction method and device, electronic equipment and storage medium | |
| CN116167699B (en) | Equipment guarantee resource management method and system | |
| KR100979521B1 (en) | System and method for electronic business document communication |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |