CN117579507A - Method for automatically extracting recording flow - Google Patents

Method for automatically extracting recording flow Download PDF

Info

Publication number
CN117579507A
CN117579507A CN202410066135.1A CN202410066135A CN117579507A CN 117579507 A CN117579507 A CN 117579507A CN 202410066135 A CN202410066135 A CN 202410066135A CN 117579507 A CN117579507 A CN 117579507A
Authority
CN
China
Prior art keywords
flow
recorded
extraction
traffic
recording
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410066135.1A
Other languages
Chinese (zh)
Inventor
李茹钰
李洋
张鑫
张自平
刘毅强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yunzhu Information Technology Chengdu Co ltd
Original Assignee
Yunzhu Information Technology Chengdu Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yunzhu Information Technology Chengdu Co ltd filed Critical Yunzhu Information Technology Chengdu Co ltd
Priority to CN202410066135.1A priority Critical patent/CN117579507A/en
Publication of CN117579507A publication Critical patent/CN117579507A/en
Pending legal-status Critical Current

Links

Abstract

The invention discloses a method for automatically extracting and recording flow, which comprises the steps of creating an automatic flow extraction task; configuring a flow extraction rule; acquiring recording flow; starting an automatic flow extraction task, and filtering each recorded flow by adopting a flow extraction rule; and sending the filtered recorded flow to a background management platform for rechecking to obtain the target sample flow. According to the invention, the target flow is obtained after repeated filtration for many times through the flow extraction rule, so that the manual screening workload is reduced, the extraction time is shortened, and the extraction efficiency is improved. The flow extraction rule provided by the invention can be set in a self-defined way, has good expandability and is suitable for flow extraction of different scenes.

Description

Method for automatically extracting recording flow
Technical Field
The invention relates to the technical field of recorded flow extraction, in particular to a method for automatically extracting recorded flow.
Background
In the context of software traffic recording and playback, a large number of requested traffic operations are often recorded continuously, however, we need to extract a stylized traffic record from these large number of traffic records as the stylized sample traffic data. When a subsequent tester plays back, the corresponding sample flow data can be selected according to the specific scene to play back the flow automatically, so that the workload of the tester is greatly reduced.
However, the difficulty of efficiently extracting sample flow data of a specified scene from a continuous mass flow record is great. Because the flow operated by the user enters at any moment, the same interface also has the flow operated by a large number of different personnel, and the difficulty of selecting the flow can be disturbed. The related interfaces of the scene are called one by one, and the related interfaces are called one by one, so that the workload is extremely high.
The flow extraction mainly comprises the following two technical schemes: 1. and aiming at each scene, manually searching the traffic of each interface one by one and adding. 2. After time period retrieval, all the additions are made in batches. The prior art mainly has the following technical problems: the flow records of the appointed scene can only be added after single-piece one-by-one screening; the staff to be screened knows which interfaces are included in each scene; each flow record cannot be associated with a specific operating user; the flow range to be selected and the recorded data volume are too much, so that the workload of manual screening is too large, and the continuous maintenance difficulty is high.
Therefore, the present invention provides a method for automatically extracting recording flow, so as to at least solve the above-mentioned part of technical problems.
Disclosure of Invention
The invention aims to solve the technical problems that: a method for automatically extracting recording flow is provided to solve at least some of the above technical problems.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
a method for automatically extracting recorded flow, comprising the following steps:
step 1, creating an automatic flow extraction task;
step 2, configuring a flow extraction rule;
step 3, obtaining recording flow;
step 4, starting an automatic flow extraction task, and filtering each recorded flow by adopting a flow extraction rule;
and step 5, sending the filtered recorded flow to a background management platform for rechecking to obtain the target flow.
Further, in step 2, the configuration information of the traffic extraction rule includes a start extraction time, an end extraction time, a user ID to be extracted, a switch for extracting only a service successfully, an Http extraction field, a Dubbo extraction field, an interface address, and a custom extraction rule expression.
Further, in step 3, each acquired recording flow is converted into a Map object as the calculated up-down parameter information.
Further, step 4 includes: step 41, extracting the recording time of each recording flow based on the key value pair of the Map object of the recording flow, and if the recording time is between the starting extraction time and the ending extraction time, collecting the current recording flow into the first flow set, otherwise, ignoring the current recording flow; step 42, extracting a login user ID of each recording flow in the first flow set based on the key value pair of the Map object of the recording flow, if the login user ID is the user ID to be extracted, collecting the current recording flow into the second flow set, otherwise ignoring the current recording flow; and 43, extracting the state of whether the service of each recorded flow in the second flow set is successful or not based on the key value pair of the Map object of the recorded flow, and if the configuration information is a switch for only extracting the service successfully, collecting the recorded flow in the state that the service is successful to the third flow set, otherwise, ignoring the current recorded flow.
Further, step 4 further includes: if the recorded traffic is HTTP traffic, extracting the URL address of the HTTP interface of each recorded traffic in the third traffic set based on the key value pair of the Map object of the recorded traffic, and if the URL address is the same as the HTTP extraction field, collecting the current recorded traffic until the current recorded traffic is ignored otherwise.
Further, step 4 further includes: and if the recorded flow is the Dubbo flow, extracting the service address of the Dubbo interface of each recorded flow in the third flow set based on the key value pair of the Map object of the recorded flow, and if the prefix of the service address is the same as the Dubbo extraction field, collecting the current recorded flow, otherwise, ignoring the current recorded flow.
Further, step 4 further includes: and taking the Map object of the recorded flow as the whole context parameter, executing the custom extraction rule expression, and if the execution result is true, collecting the current recorded flow, otherwise, ignoring the current recorded flow.
Further, the custom extraction rule expression is an MVEL expression.
Further, in step 5, review includes manually rejecting excess recording traffic and manually adding missing recording traffic.
Compared with the prior art, the invention has the following beneficial effects:
according to the invention, the target flow is obtained after repeated filtration for many times through the flow extraction rule, so that the manual screening workload is reduced, the extraction time is shortened, and the extraction efficiency is improved. The flow extraction rule can be set in a self-defined way, has good expandability and is suitable for flows in different scenes.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
Term interpretation:
a Map object is an object with corresponding key-value pairs.
The HTTP interface refers to a request message from a client to a server.
The Dubbo interface is an interface for communication between a service end providing a service and a client invoking the service.
The URL address is a uniform resource locator.
The MVEL expression is an expression language, and is mainly used for writing conditional expressions and logic expressions.
The present invention will be described in further detail with reference to the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the description of the present invention, it should be noted that the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
As shown in fig. 1, the method for automatically extracting recording flow provided by the invention comprises the following steps:
step 1, creating an automatic flow extraction task;
step 2, configuring a flow extraction rule;
step 3, obtaining recording flow;
step 4, starting an automatic flow extraction task, and filtering each recorded flow by adopting a flow extraction rule;
and step 5, sending the filtered recorded flow to a background management platform for rechecking to obtain the target sample flow.
In some embodiments, the configuration information of the flow extraction rule in step 2 includes a start extraction time, an end extraction time, a user ID to be extracted, a switch for extracting only a service successfully, an Http extraction field, a Dubbo extraction field, and a custom extraction rule expression.
In some embodiments, step 3 converts each acquired recording flow into a Map object.
In some embodiments, step 4 creates a flow extraction rule filter using JAVA, and adds the flow extraction rule filter to the process of obtaining the recorded flow, and performs filtering on the obtained recorded flow.
In some embodiments, step 4 includes: step 41, extracting the recording time of each recording flow based on the key value pair of the Map object of the recording flow, wherein the recording time is specifically as follows: obtaining recording time through map. Get ("createTime"), setting the starting extraction time as T1 and the ending extraction time as T2, if the recording time is between the starting extraction time T1 and the ending extraction time T2, collecting the current recording flow into the first flow set, otherwise ignoring the current recording flow; step 42, extracting a login user ID of each recording flow in the first flow set based on a key value pair of the Map object of the recording flow, which specifically includes: obtaining login user IDs through map.get ('userId'), wherein a plurality of user IDs to be extracted are contained in a user ID list to be extracted, and if the login user IDs are the user IDs to be extracted contained in the user ID list to be extracted, collecting the current recording flow into a second flow set, otherwise, ignoring the current recording flow; step 43, extracting a status of whether the service of each recording flow in the second flow set is successful based on the key value pair of the Map object of the recording flow, specifically: and extracting a service state field through map. Get ("success") to obtain the state of whether the service is successful, wherein only the scene of success of the service is concerned, namely, the configuration information is a switch which only extracts the service successfully, and recording traffic of which the service is in a successful state is collected into a third traffic set, otherwise, the current recording traffic is ignored. The names of the createTime, userId and success parameters can be customized according to actual situations.
Step 41 is the first filtering to filter out the flow in non-monitoring time; step 42 is a second filtering, filtering the traffic of the non-monitoring user; step 43 is the third filtering, and most of the traffic in the service failure scenario is filtered. The above three times of filtering can greatly reduce the size of the sample data to be extracted. If part of the traffic scenarios require interface errors, i.e. traffic failure scenarios, and thus traffic for such scenarios, step 43 is not required.
Typically, the interfaces related to login, menu and authorization are written in a control layer respectively, so that each type of traffic basically has the same prefix.
In some embodiments, step 4 further includes: if the recorded traffic is HTTP traffic, extracting a URL address of an HTTP interface of each recorded traffic in the third traffic set based on a key value pair of a Map object of the recorded traffic, specifically: and acquiring the URL address of the HTTP interface of the recorded flow through map. Get ('URL'), and if the URL address is the same as the HTTP extraction field, collecting the current recorded flow until the current recorded flow is ignored otherwise.
The Http extract field includes a URL address prefix and a postamble, where the postamble includes the following three means, the first: "is" means that any single character can be matched; second kind: "+" means that any number of characters (excluding path delimiters) can be matched; third kind: "x" means that any number of characters (including path splitters) can be matched. Setting the Http extraction field to "/user/" means that only all Http traffic beginning with "/user/" is extracted.
In some embodiments, step 4 further includes: if the recorded traffic is the Dubbo traffic, extracting a service address of the Dubbo interface of each recorded traffic in the third traffic set based on a key value pair of Map objects of the recorded traffic, specifically: and acquiring the service address of the Dubbo interface of the recorded traffic through map. Get ('serviceId'), and if the prefix of the service address is the same as the Dubbo extraction field, collecting the current recorded traffic, otherwise, ignoring the current recorded traffic.
Similarly, the Dubbo extraction field includes a service address prefix and a postamble, and the postamble also includes three modes ". Setting the Dubbo extraction field to "cn.demo.service.facade.userfacade/", indicates that only all Dubbo traffic beginning with "cn.demo.service.facade.userfacade/", is extracted.
The two filtering modes included in the above step 4 may be set after the step 43, and the filtering is performed for the fourth time. The flow after the four times of filtering is very close to the final target flow, but when a developer writes codes, more similar interfaces are usually written in the same control layer at the same time, or very individual related interfaces are written in other unrelated control layers, so that a scene of extraction omission occurs. Preferably, the flow rate after four times of filtering can be corrected.
The invention can also set the custom extraction rule expression to meet different extraction requirements, and adapt to various abnormal scenes.
In some embodiments, step 4 further includes: and taking the Map object of the recorded flow as the whole context parameter, executing the custom extraction rule expression, and if the execution result is true, collecting the current recorded flow, otherwise, ignoring the current recorded flow. Such as: script Arrays. AsList (200,401) contains (code) indicates that the service status code (code) of the login interface is 200 or 401, which needs to be collected. Preferably, the custom extraction rule expression is an Mvel expression, each custom extraction rule expression is executed through an Mvel (express, map), where express is a sentence string of the custom extraction rule expression, map is a context parameter including a recording flow, mvel (virtual) is an execution engine method of an open-source Mvel 2 toolkit, and an Mvel (virtual) execution result is true, the current recording flow is collected, otherwise false ignores the current recording flow.
After the multiple times of filtering, the filtered recorded flow is sent to a background management platform for rechecking, wherein rechecking comprises manually removing redundant recorded flow and manually adding missing recorded flow, and finally the target sample flow is obtained.
Finally, it should be noted that: the above embodiments are merely preferred embodiments of the present invention for illustrating the technical solution of the present invention, but not limiting the scope of the present invention; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions; that is, even though the main design concept and spirit of the present invention is modified or finished in an insubstantial manner, the technical problem solved by the present invention is still consistent with the present invention, and all the technical problems are included in the protection scope of the present invention; in addition, the technical scheme of the invention is directly or indirectly applied to other related technical fields, and the technical scheme is included in the scope of the invention.

Claims (9)

1. A method for automatically extracting recorded flow, comprising the steps of:
step 1, creating an automatic flow extraction task;
step 2, configuring a flow extraction rule;
step 3, obtaining recording flow;
step 4, starting an automatic flow extraction task, and filtering each recorded flow by adopting a flow extraction rule;
and step 5, sending the filtered recorded flow to a background management platform for rechecking to obtain the target sample flow.
2. The method according to claim 1, wherein in step 2, the configuration information of the traffic extraction rule includes a start extraction time, an end extraction time, a user ID to be extracted, a switch for judging whether only the extracted service is successful, an Http extraction field, a Dubbo extraction field, an interface address, and a custom extraction rule expression.
3. The method according to claim 2, wherein in step 3, each obtained recording flow is converted into a Map object as the calculated upper and lower parameter information.
4. A method for automatically extracting recorded traffic as in claim 3, wherein step 4 comprises: step 41, extracting the recording time of each recording flow based on the key value pair of the Map object of the recording flow, and if the recording time is between the starting extraction time and the ending extraction time, collecting the current recording flow into the first flow set, otherwise, ignoring the current recording flow; step 42, extracting a login user ID of each recording flow in the first flow set based on the key value pair of the Map object of the recording flow, if the login user ID is the user ID to be extracted, collecting the current recording flow into the second flow set, otherwise ignoring the current recording flow; and 43, extracting the state of whether the service of each recorded flow in the second flow set is successful or not based on the key value pair of the Map object of the recorded flow, and if the configuration information is a switch for only extracting the service successfully, collecting the recorded flow in the state that the service is successful to the third flow set, otherwise, ignoring the current recorded flow.
5. A method for automatically extracting recorded traffic as in claim 3, wherein step 4 further comprises: if the recorded traffic is HTTP traffic, extracting the URL address of the HTTP interface of each recorded traffic in the third traffic set based on the key value pair of the Map object of the recorded traffic, and if the URL address is the same as the HTTP extraction field, collecting the current recorded traffic until the current recorded traffic is ignored otherwise.
6. A method for automatically extracting recorded traffic as in claim 3, wherein step 4 further comprises: and if the recorded flow is the Dubbo flow, extracting the service address of the Dubbo interface of each recorded flow in the third flow set based on the key value pair of the Map object of the recorded flow, and if the prefix of the service address is the same as the Dubbo extraction field, collecting the current recorded flow, otherwise, ignoring the current recorded flow.
7. A method for automatically extracting recorded traffic as in claim 3, wherein step 4 further comprises: and taking the Map object of the recorded flow as the whole context parameter, executing the custom extraction rule expression, and if the execution result is true, collecting the current recorded flow, otherwise, ignoring the current recorded flow.
8. The method of claim 7, wherein the custom extraction rule expression is an MVEL expression.
9. The method of claim 1, wherein in step 5, the rechecking includes manually rejecting excess recorded traffic and manually adding missing recorded traffic.
CN202410066135.1A 2024-01-17 2024-01-17 Method for automatically extracting recording flow Pending CN117579507A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410066135.1A CN117579507A (en) 2024-01-17 2024-01-17 Method for automatically extracting recording flow

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410066135.1A CN117579507A (en) 2024-01-17 2024-01-17 Method for automatically extracting recording flow

Publications (1)

Publication Number Publication Date
CN117579507A true CN117579507A (en) 2024-02-20

Family

ID=89895968

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410066135.1A Pending CN117579507A (en) 2024-01-17 2024-01-17 Method for automatically extracting recording flow

Country Status (1)

Country Link
CN (1) CN117579507A (en)

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100293316A1 (en) * 2009-05-15 2010-11-18 Vivek Mehrotra Migration of Switch in a Storage Area Network
US20140095711A1 (en) * 2011-06-27 2014-04-03 Sk Telecom Co., Ltd. System, device and method for managing network traffic by using monitoring and filtering policies
KR101428721B1 (en) * 2013-06-24 2014-08-12 한국인터넷진흥원 Method and system for detecting malicious traffic by analyzing traffic
CN106878108A (en) * 2017-03-01 2017-06-20 广州唯品会信息科技有限公司 Network flow playback method of testing and device
CN107609020A (en) * 2017-08-07 2018-01-19 北京京东尚科信息技术有限公司 A kind of method and apparatus of the daily record classification based on mark
CN110784356A (en) * 2019-10-30 2020-02-11 北京蜜莱坞网络科技有限公司 Automatic flow playback method
WO2022034619A1 (en) * 2020-08-14 2022-02-17 Novi Digital Entertainment Private Limited System and method for delivering media content to users
CN115297041A (en) * 2022-07-29 2022-11-04 深圳前海微众银行股份有限公司 Data verification method and device for flow playback
CN115712569A (en) * 2022-11-22 2023-02-24 平安银行股份有限公司 Method and device for flow playback, electronic equipment and storage medium
CN115834443A (en) * 2022-09-07 2023-03-21 深圳依时货拉拉科技有限公司 Flow playback method and device, computer equipment and storage medium
CN116483707A (en) * 2023-04-18 2023-07-25 北京达佳互联信息技术有限公司 Test method, test device, test apparatus, test program, and test program
CN117130941A (en) * 2023-10-24 2023-11-28 易方信息科技股份有限公司 Interface automation method, system, equipment and medium based on browser plug-in
CN117278452A (en) * 2023-09-15 2023-12-22 蒲惠智造科技股份有限公司 Method and device for recording and playing back flow, electronic equipment and computer storage medium

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100293316A1 (en) * 2009-05-15 2010-11-18 Vivek Mehrotra Migration of Switch in a Storage Area Network
US20140095711A1 (en) * 2011-06-27 2014-04-03 Sk Telecom Co., Ltd. System, device and method for managing network traffic by using monitoring and filtering policies
KR101428721B1 (en) * 2013-06-24 2014-08-12 한국인터넷진흥원 Method and system for detecting malicious traffic by analyzing traffic
CN106878108A (en) * 2017-03-01 2017-06-20 广州唯品会信息科技有限公司 Network flow playback method of testing and device
CN107609020A (en) * 2017-08-07 2018-01-19 北京京东尚科信息技术有限公司 A kind of method and apparatus of the daily record classification based on mark
CN110784356A (en) * 2019-10-30 2020-02-11 北京蜜莱坞网络科技有限公司 Automatic flow playback method
WO2022034619A1 (en) * 2020-08-14 2022-02-17 Novi Digital Entertainment Private Limited System and method for delivering media content to users
CN115297041A (en) * 2022-07-29 2022-11-04 深圳前海微众银行股份有限公司 Data verification method and device for flow playback
CN115834443A (en) * 2022-09-07 2023-03-21 深圳依时货拉拉科技有限公司 Flow playback method and device, computer equipment and storage medium
CN115712569A (en) * 2022-11-22 2023-02-24 平安银行股份有限公司 Method and device for flow playback, electronic equipment and storage medium
CN116483707A (en) * 2023-04-18 2023-07-25 北京达佳互联信息技术有限公司 Test method, test device, test apparatus, test program, and test program
CN117278452A (en) * 2023-09-15 2023-12-22 蒲惠智造科技股份有限公司 Method and device for recording and playing back flow, electronic equipment and computer storage medium
CN117130941A (en) * 2023-10-24 2023-11-28 易方信息科技股份有限公司 Interface automation method, system, equipment and medium based on browser plug-in

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
XIAOJUN HEI ET AL: "A Measurement Study of a Large-Scale P2P IPTV System", 《IEEE TRANSACTIONS ON MULTIMEDIA》, 19 November 2007 (2007-11-19) *
刘国庆;汪兴轩;: "基于Charles录制会话的HTTP接口自动化测试框架设计与实现", 计算机应用与软件, no. 06, 12 June 2019 (2019-06-12) *

Similar Documents

Publication Publication Date Title
US20060104220A1 (en) Method and apparatus for managing configuration information, and configuration information managing system using the apparatus
CN111752799A (en) Service link tracking method, device, equipment and storage medium
JP4940791B2 (en) Test support program, test support apparatus, and test support method
CN107480036A (en) Automatic test approach, device, storage medium and the terminal of iOS Mobile solutions
JPWO2014049804A1 (en) System operation tracing method in distributed system
JP2008210308A (en) Log integrating managing device, log integrating managing method, and log integrating managing program
US8024706B1 (en) Techniques for embedding testing or debugging features within a service
JP4652090B2 (en) Event notification management program, event notification management apparatus, and event notification management method
JP3916232B2 (en) Knowledge-type operation management system, method and program
CN109587141A (en) A kind of system and method for remote server evidence obtaining
JP2010250583A (en) Program and method for generating application test, and application test device
CN117579507A (en) Method for automatically extracting recording flow
JP5012999B2 (en) Maintenance work support program, maintenance work support method, and maintenance work support apparatus
CN111538712A (en) Log recording method, processing node, electronic device and storage medium
CN116450885A (en) Data reconstruction method of Windows event log file
CN111563123B (en) Real-time synchronization method for hive warehouse metadata
Cisco Installing and Configuring Native Service Point
Cisco Installing and Configuring Native Service Point
Cisco Installing and Configuring Native Service Point
Cisco Installing and Configuring Native Service Point
Cisco Installing and Configuring Native Service Point
Cisco Installing and Configuring Native Service Point
Cisco Installing and Configuring Native Service Point
CN113407415A (en) Log management method and device of intelligent terminal
CN112217896A (en) JSON message conversion method and related device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination