CN112637049A - Data capture system and method - Google Patents
Data capture system and method Download PDFInfo
- Publication number
- CN112637049A CN112637049A CN202011485713.3A CN202011485713A CN112637049A CN 112637049 A CN112637049 A CN 112637049A CN 202011485713 A CN202011485713 A CN 202011485713A CN 112637049 A CN112637049 A CN 112637049A
- Authority
- CN
- China
- Prior art keywords
- data
- network
- virtual operating
- data source
- dialing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/64—Hybrid switching systems
- H04L12/6418—Hybrid transport
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/46—Interconnection of networks
- H04L12/4641—Virtual LANs, VLANs, e.g. virtual private networks [VPN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/64—Hybrid switching systems
- H04L12/6418—Hybrid transport
- H04L2012/6478—Digital subscriber line, e.g. DSL, ADSL, HDSL, XDSL, VDSL
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Telephonic Communication Services (AREA)
Abstract
The invention provides a data capture system and a method, wherein the data capture system comprises: the server is respectively connected with the switch and the client; the switch is connected with the dialing networks, and the corresponding VLAN numbers of each dialing network on the switch are different; the server is provided with at least two virtual operating systems, the server is connected with a dialing network through the virtual operating systems, the dialing network connected with each virtual operating system is different, and a data source is accessed through the dialing network; the client stores a dynamic IP pool, and the server controls the virtual operating system to access the data source according to the IP in the dynamic IP pool in series so as to capture data. The invention respectively connects different dialing networks through a plurality of virtual operating systems on the server, and serially accesses the data source by using the IP in the dynamic IP pool without using a plurality of servers to expand the dialing networks, thereby reducing the cost of service operation, accessing different links by using specific links and improving the efficiency of data capture.
Description
Technical Field
The invention relates to the field of big data acquisition, in particular to a data capture system and a data capture method.
Background
When the internet data is captured, a converged dialing network is used for capturing the data. At present, telecom operators are respectively China telecom, China Mobile, China Unicom and the like, but the network access modes of the telecom operators are generally two modes of fixed IP optical fiber access and dynamic dial access, and when network progress data in the fixed IP access mode is captured, the network progress data is easily shielded by a source website, so the network progress data is generally captured in the dial network mode. However, if only 1 single dial-up network is used for data capture, the short time is the same as the fixed IP network access method, because the dial-up network changes the IP each time the dial-up network dials up, but if the frequency of dial-up redial is dense, the redial is performed once in about 30 seconds, and if the frequency is long, the redial is easily detected by the telecommunication operator, and the process is also shielded or suspended. So data grabbing typically uses multiple dial-up networks for grabbing. However, the dialing operation is performed at a system level, one system cannot perform dialing on a plurality of different dialing networks, and if the capabilities of the dialing networks are expanded by using different servers, the cost for service operation is greatly increased, because the cost is greatly increased because a server needs to be bought due to the unilateral limitation of a network interface. And if the dial-up network is shielded by using the router mode, although the shielding can be relieved, the specific link cannot be used for accessing each link captured by the data on the router, so that the efficiency of data capture is reduced.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a data capturing system and a data capturing method, wherein a switch is respectively connected with a plurality of dialing networks, different dialing networks are respectively connected through a plurality of virtual operating systems on a server, and the IP in a dynamic IP pool is utilized to access a data source in series, so that the dialing networks do not need to be expanded by a plurality of servers, the cost of service operation is reduced, different links can be accessed by using specific links, and the data capturing efficiency is improved.
In order to solve the above problems, the present invention adopts a technical solution as follows: a data crawling system, the data crawling system comprising: the system comprises a switch, a server and a client, wherein the server is respectively connected with the switch and the client; the switch is connected with at least two dialing networks, and the corresponding VLAN numbers of each dialing network on the switch are different; the server is provided with at least two virtual operating systems, the server is connected with the dialing network through the virtual operating systems, the dialing network connected with each virtual operating system is different, and a data source is accessed through the dialing network; the client stores a dynamic IP pool, and the server controls the virtual operating system to access the data source in series according to the IP in the dynamic IP pool so as to capture data.
Further, the dial-up network is an ADSL dial-up network.
Further, the number of the virtual operating systems is the same as that of the dial-up networks, and the virtual operating systems correspond to the dial-up networks one by one.
Further, the step of the server controlling the virtual operating system to serially access the data source according to the IP in the dynamic IP pool to capture data specifically includes: and the server controls the virtual operating system to sequence the IPs in the dynamic IP pool, and sequentially selects the IPs to access the data source according to a sequencing result.
Further, the step of sequentially selecting the IPs to access the data source according to the sorting result specifically includes: judging whether the time for accessing the data source through the current IP reaches a preset value or not; if so, disconnecting the network, and accessing the data source through the next IP of the current IP according to the sequencing result; if not, continuing to access the data source.
Based on the same inventive concept, the invention also provides a data capturing method, which comprises the following steps: s101: creating virtual operating systems, and controlling at least two dialing networks connected between the at least two virtual operating systems and the switch, wherein the corresponding VLAN numbers of each dialing network on the switch are different, the dialing networks connected with each virtual operating system are different, and a data source is accessed through the dialing networks; s102: and acquiring a dynamic IP pool stored by a client, and controlling the virtual operating system to access the data source in series according to the IP in the dynamic IP pool so as to capture data.
Further, the dial-up network is an ADSL dial-up network.
Further, the number of the virtual operating systems is the same as that of the dial-up networks, and the virtual operating systems correspond to the dial-up networks one by one.
Further, the step of controlling the virtual operating system to serially access the data source according to the IP in the dynamic IP pool to capture data specifically includes: and controlling the virtual operating system to randomly sort the IPs in the dynamic IP pool, and sequentially selecting the IPs to access the data source according to a sorting result.
Further, the step of sequentially selecting the IPs to access the data source according to the sorting result specifically includes: judging whether the time for accessing the data source through the current IP reaches a preset value or not; if so, disconnecting the network, and accessing the data source through the next IP of the current IP according to the sequencing result; if not, continuing to access the data source.
Compared with the prior art, the invention has the beneficial effects that: the switch is connected with the plurality of dialing networks respectively, the virtual operating systems on the server are connected with different dialing networks respectively, and the IP in the dynamic IP pool is used for accessing the data source in series, so that the dialing networks do not need to be expanded by using the plurality of servers, the cost of service operation is reduced, different links can be accessed by using specific links, and the efficiency of data capture is improved.
Drawings
FIG. 1 is a block diagram of an embodiment of a data capture system according to the present invention;
FIG. 2 is a diagram of a data capture system according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a data capture method according to an embodiment of the present invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and the detailed description, and it should be noted that any combination of the embodiments or technical features described below can be used to form a new embodiment without conflict.
Referring to fig. 1 and 2, fig. 1 is a structural diagram of an embodiment of a data capture system according to the present invention; FIG. 2 is a diagram of a data capture system according to an embodiment of the present invention. The data capture system of the present invention will be described in detail with reference to fig. 1 and 2.
In this embodiment, the data capture system includes: the system comprises a switch, a server and a client, wherein the server is respectively connected with the switch and the client; the switch is connected with at least two dialing networks, and the number of a corresponding VLAN (Virtual Local Area Network) of each dialing Network on the switch is different; the server is provided with at least two virtual operating systems, the server is connected with a dialing network through the virtual operating systems, the dialing network connected with each virtual operating system is different, and a data source is accessed through the dialing network; the client stores a dynamic IP pool, and the server controls the virtual operating system to access the data source according to the IP in the dynamic IP pool in series so as to capture data.
In this embodiment, the dial-up network is an ADSL dial-up network, in which the switch is connected to the ADSL dial-up network through a network cable, and the ADSL dial-up network corresponding to each network is different.
In this embodiment, the switch is connected to the network cable through a connection port, where the connection port may be any one of a BNC interface, an RJ-45 interface, and an AUI interface, and the type of the network cable is set according to the type of the connection port, and the switch can be connected to the dial-up network through the connection port and the network cable.
In this embodiment, each dial-up network is isolated by an individual VLAN number on the switch, that is, each dial-up network is not connected to another, the server or the switch records the VLAN number corresponding to each dial-up network, and the corresponding dial-up network is determined by the VLAN number.
In the embodiment, the server and the switch are connected through a wired connection, wherein the server is connected with the switch through a network port (RJ 45) arranged on the server.
In this embodiment, the number of virtual operating systems is the same as the number of dial-up networks, and corresponds to one another.
In this embodiment, the operating system in the server may form a virtual operating system by installing a kvm (keyboard Video mouse) or hypervisor (virtual machine monitor) service. The virtual operating systems realize effective isolation of hardware and software resources of the network/operating system, so that each virtual operating system is not influenced by each other. When operating system network configuration is performed through KVM or hypervisor service, virtual LAN operation is started, and a VLAN number in the virtual LAN operation is a VLAN number of a designated dynamic dial-up network.
In this embodiment, dynamic dialing of a dial-up network is implemented by dial-up software installed in a virtual operating system, where the dial-up software is dial-up software in the prior art and is not described herein again.
In this embodiment, the step of the server controlling the virtual operating system to access the data source according to the IP concatenation in the dynamic IP pool to capture the data specifically includes: and the server controls the virtual operating system to sequence the IPs in the dynamic IP pool, and sequentially selects the IP access data sources according to a sequencing result.
In other embodiments, the dynamic IP pool may also be stored in the server, and the server sorts the 1IP in the dynamic IP pool and accesses the 1IP according to the sorting result.
In this embodiment, the sorting manner may be random sorting, bubble sorting, selection sorting, insertion sorting, hill sorting, merge sorting, quick sorting, radix sorting, heap sorting, count sorting, bucket sorting, or other sorting manners, and only the IPs in the dynamic IP pool need to be sorted through these sorting manners, which is not limited herein.
In this embodiment, the step of sequentially selecting the IP access data sources according to the sorting result specifically includes: judging whether the time for accessing the data source through the current IP reaches a preset value or not; if so, disconnecting the network, and accessing the data source through the next IP of the current IP according to the sequencing result; if not, continuing to access the data source. Data capture is performed by means of regular dialing reconnection of a network and serial access, so that network tracking and network limitation of a data source can be effectively prevented, and normal access of a human can be simulated. The invention uses lower cost, 1 server and 1 exchanger, which can manage a large amount of dynamic dialing network, effectively prevent data source network tracking.
The invention has the following advantages:
1. only 1 server is used to handle all data crawling related network accesses.
2. The number of dial-up networks can be greatly expanded, so that the source network for data capture cannot monitor the IP condition, and the problem of data capture failure is avoided.
Has the advantages that: the data capturing system of the invention connects the switch with the plurality of dialing networks respectively, connects different dialing networks through the plurality of virtual operating systems on the server respectively, and accesses the data source in series by using the IP in the dynamic IP pool without using a plurality of servers to expand the dialing networks, thereby reducing the cost of service operation, accessing different links by using specific links and improving the efficiency of data capturing.
Based on the same inventive concept, the present invention further provides a data capture method, please refer to fig. 3, fig. 3 is a flowchart of an embodiment of the data capture method of the present invention, and the data capture method of the present invention is specifically described with reference to fig. 3.
In this embodiment, the data capture method includes:
s101: and creating virtual operating systems, controlling at least two dialing networks connected with the switch by the at least two virtual operating systems, wherein the corresponding VLAN numbers of each dialing network on the switch are different, the dialing networks connected with each virtual operating system are different, and accessing the data source through the dialing networks.
In this embodiment, the device executing the data capture method is a server, and in other embodiments, the device may also be a computer, a cloud platform, a virtual server, or other entity or virtual device capable of setting a virtual operating system.
In this embodiment, the dial-up network is an ADSL dial-up network, in which the switch is connected to the ADSL dial-up network through a network cable, and the ADSL dial-up network corresponding to each network is different.
In this embodiment, the switch is connected to the network cable through a connection port, where the connection port may be any one of a BNC interface, an RJ-45 interface, and an AUI interface, and the type of the network cable is set according to the type of the connection port, and the switch can be connected to the dial-up network through the connection port and the network cable.
In this embodiment, each dial-up network is isolated by an individual VLAN number on the switch, that is, each dial-up network is not connected to another, the server or the switch records the VLAN number corresponding to each dial-up network, and the corresponding dial-up network is determined by the VLAN number.
In the embodiment, the server and the switch are connected through a wired connection, wherein the server is connected with the switch through a network port (RJ 45) arranged on the server.
In this embodiment, the number of virtual operating systems is the same as the number of dial-up networks, and corresponds to one another.
In this embodiment, the operating system in the server may form a virtual operating system by installing a kvm (keyboard Video mouse) or hypervisor (virtual machine monitor) service. The virtual operating systems realize effective isolation of hardware and software resources of the network/operating system, so that each virtual operating system is not influenced by each other. When operating system network configuration is performed through KVM or hypervisor service, virtual LAN operation is started, and a VLAN number in the virtual LAN operation is a VLAN number of a designated dynamic dial-up network.
In this embodiment, dynamic dialing of a dial-up network is implemented by dial-up software installed in a virtual operating system, where the dial-up software is dial-up software in the prior art and is not described herein again.
S102: and acquiring a dynamic IP pool stored by the client, and controlling the virtual operating system to access the data source according to the IP in the dynamic IP pool in series so as to capture data.
In this embodiment, the step of controlling the virtual operating system to serially access the data source according to the IP in the dynamic IP pool to capture the data specifically includes: and controlling the virtual operating system to sequence the IPs in the dynamic IP pool, and sequentially selecting the IP access data sources according to a sequencing result.
In other embodiments, the dynamic IP pool may also be stored in the server, and the server sorts the 1IP in the dynamic IP pool and accesses the 1IP according to the sorting result.
In this embodiment, the sorting manner may be random sorting, bubble sorting, selection sorting, insertion sorting, hill sorting, merge sorting, quick sorting, radix sorting, heap sorting, count sorting, bucket sorting, or other sorting manners, and only the IPs in the dynamic IP pool need to be sorted through these sorting manners, which is not limited herein.
In this embodiment, the step of sequentially selecting the IP access data sources according to the sorting result specifically includes: judging whether the time for accessing the data source through the current IP reaches a preset value or not; if so, disconnecting the network, and accessing the data source through the next IP of the current IP according to the sequencing result; if not, continuing to access the data source. Data capture is performed by means of regular dialing reconnection of a network and serial access, so that network tracking and network limitation of a data source can be effectively prevented, and normal access of a human can be simulated. The invention uses lower cost, 1 server and 1 exchanger, which can manage a large amount of dynamic dialing network, effectively prevent data source network tracking.
The invention has the following advantages:
1. only 1 server is used to handle all data crawling related network accesses.
2. The number of dial-up networks can be greatly expanded, so that the source network for data capture cannot monitor the IP condition, and the problem of data capture failure is avoided.
Has the advantages that: the data capturing method of the invention connects the switch with the plurality of dialing networks respectively, connects different dialing networks through the plurality of virtual operating systems on the server respectively, and accesses the data source in series by using the IP in the dynamic IP pool without using a plurality of servers to expand the dialing networks, thereby reducing the cost of service operation, accessing different links by using specific links and improving the efficiency of data capturing.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A data capture system, comprising; the system comprises a switch, a server and a client, wherein the server is respectively connected with the switch and the client;
the switch is connected with at least two dialing networks, and the corresponding VLAN numbers of each dialing network on the switch are different;
the server is provided with at least two virtual operating systems, the server is connected with the dialing network through the virtual operating systems, the dialing network connected with each virtual operating system is different, and a data source is accessed through the dialing network;
the client stores a dynamic IP pool, and the server controls the virtual operating system to access the data source in series according to the IP in the dynamic IP pool so as to capture data.
2. The data capture system of claim 1, wherein the dial-up network is an ADSL dial-up network.
3. The data crawling system of claim 1, wherein the number of virtual operating systems is the same as the number of dial-up networks and corresponds to one another.
4. The data crawling system of claim 1, wherein the step of the server controlling the virtual operating system to serially access the data source according to the IP in the dynamic IP pool for data crawling specifically comprises:
and the server controls the virtual operating system to sequence the IPs in the dynamic IP pool, and sequentially selects the IPs to access the data source according to a sequencing result.
5. The data crawling system of claim 4, wherein the step of sequentially selecting the IP to access the data source according to the sorting result specifically comprises:
judging whether the time for accessing the data source through the current IP reaches a preset value or not;
if so, disconnecting the network, and accessing the data source through the next IP of the current IP according to the sequencing result;
if not, continuing to access the data source.
6. A data capture system is characterized in that the data capture method comprises the following steps:
s101: creating virtual operating systems, and controlling at least two dialing networks connected between the at least two virtual operating systems and the switch, wherein the corresponding VLAN numbers of each dialing network on the switch are different, the dialing networks connected with each virtual operating system are different, and a data source is accessed through the dialing networks;
s102: and acquiring a dynamic IP pool stored by a client, and controlling the virtual operating system to access the data source in series according to the IP in the dynamic IP pool so as to capture data.
7. The data capture system of claim 6, wherein the dial-up network is an ADSL dial-up network.
8. The data crawling system of claim 6, wherein the number of virtual operating systems is the same as the number of dial-up networks and corresponds to one another.
9. The data fetching system of claim 6, wherein the step of controlling the virtual operating system to serially access the data source according to the IP in the dynamic IP pool for data fetching specifically comprises:
and controlling the virtual operating system to randomly sort the IPs in the dynamic IP pool, and sequentially selecting the IPs to access the data source according to a sorting result.
10. The data crawling system of claim 9, wherein the step of sequentially selecting the IPs to access the data sources according to the sorting result specifically comprises:
judging whether the time for accessing the data source through the current IP reaches a preset value or not;
if so, disconnecting the network, and accessing the data source through the next IP of the current IP according to the sequencing result;
if not, continuing to access the data source.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011485713.3A CN112637049A (en) | 2020-12-16 | 2020-12-16 | Data capture system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011485713.3A CN112637049A (en) | 2020-12-16 | 2020-12-16 | Data capture system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112637049A true CN112637049A (en) | 2021-04-09 |
Family
ID=75313750
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011485713.3A Pending CN112637049A (en) | 2020-12-16 | 2020-12-16 | Data capture system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112637049A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101199166A (en) * | 2005-04-29 | 2008-06-11 | 艾利森电话股份有限公司 | Operator shop selection in broadband access |
| US20130201978A1 (en) * | 2012-02-06 | 2013-08-08 | Pradeep Iyer | Method and System for Partitioning Wireless Local Area Network |
| CN103873338A (en) * | 2014-03-11 | 2014-06-18 | 上海爱数软件有限公司 | Method for realizing network virtualization by utilizing binding and VLANs (Virtual Local Area Networks) |
| CN104320488A (en) * | 2014-11-11 | 2015-01-28 | 携程计算机技术（上海）有限公司 | Proxy server system and proxy service method |
| CN108595543A (en) * | 2018-04-08 | 2018-09-28 | 北京知道创宇信息技术有限公司 | Data grab method, device and network crawler system |
| CN108595510A (en) * | 2018-03-22 | 2018-09-28 | 成都数聚城堡科技有限公司 | A kind of reptile based on browser end, distributed reptile system and method |
| CN111339388A (en) * | 2019-06-13 | 2020-06-26 | 海通证券股份有限公司 | Information crawling system |
| CN111371880A (en) * | 2020-02-28 | 2020-07-03 | 厦门中横科技有限公司 | Dialing cloud server capable of dynamically switching IP |
-
2020
- 2020-12-16 CN CN202011485713.3A patent/CN112637049A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101199166A (en) * | 2005-04-29 | 2008-06-11 | 艾利森电话股份有限公司 | Operator shop selection in broadband access |
| US20130201978A1 (en) * | 2012-02-06 | 2013-08-08 | Pradeep Iyer | Method and System for Partitioning Wireless Local Area Network |
| CN103873338A (en) * | 2014-03-11 | 2014-06-18 | 上海爱数软件有限公司 | Method for realizing network virtualization by utilizing binding and VLANs (Virtual Local Area Networks) |
| CN104320488A (en) * | 2014-11-11 | 2015-01-28 | 携程计算机技术（上海）有限公司 | Proxy server system and proxy service method |
| CN108595510A (en) * | 2018-03-22 | 2018-09-28 | 成都数聚城堡科技有限公司 | A kind of reptile based on browser end, distributed reptile system and method |
| CN108595543A (en) * | 2018-04-08 | 2018-09-28 | 北京知道创宇信息技术有限公司 | Data grab method, device and network crawler system |
| CN111339388A (en) * | 2019-06-13 | 2020-06-26 | 海通证券股份有限公司 | Information crawling system |
| CN111371880A (en) * | 2020-02-28 | 2020-07-03 | 厦门中横科技有限公司 | Dialing cloud server capable of dynamically switching IP |
Non-Patent Citations (2)
| Title |
|---|
| 胡俊潇;陈国伟;: "网络爬虫反爬策略研究", 科技创新与应用, no. 15 * |
| 运维-FRANK: "单linux服务器同时拨多条ADSL和挂多个固定公网ip通过squid实现代理池方案", HTTPS://BLOG.CSDN.NET/XUYAQUN/ARTICLE/DETAILS/19120239, pages 1 - 4 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1169323C (en) | Network fault isolation | |
| US20200287918A1 (en) | Threat mitigation system and method | |
| CN111092900B (en) | Method and device for monitoring abnormal connection and scanning behavior of server | |
| CN1083193C (en) | Data storage device | |
| CN111367693B (en) | Method, system, device and medium for scheduling plug-in tasks based on message queue | |
| EP4024765B1 (en) | Method and apparatus for extracting fault propagation condition, and storage medium | |
| CN111010405B (en) | SaaS-based website security monitoring system | |
| CN114070707A (en) | Internet performance monitoring method and system | |
| CN101043461A (en) | Method for testing DHCPv6 service and client | |
| CN109451020A (en) | Overtime management method, equipment and computer readable storage medium | |
| CN111934953A (en) | Batch testing method based on domestic processor computer platform | |
| CN101035025A (en) | Network device management method and system | |
| EP2521315A1 (en) | Systems and methods for modeling a multi-layer network topology | |
| CN110855784A (en) | Proxy server node selection method, electronic device, system and medium | |
| CN108347465B (en) | Method and device for selecting network data center | |
| CN106254337A (en) | A kind of network scanning method and network scanning system | |
| CN112637049A (en) | Data capture system and method | |
| US20060031445A1 (en) | Method and apparatus for determination of network topology | |
| CN116886286A (en) | Big data authentication service self-adaption method, device and equipment | |
| CN110380981B (en) | Flow distribution method and equipment | |
| CN105610991A (en) | Load balancing method of recursion and analysis DNS (Domain Name Server) under DHCP (Dynamic Host Configuration Protocol) environment | |
| CN106101378A (en) | A kind of apply the method for Anti-addiction, device with mobile terminal | |
| CN1815985A (en) | Wide-band network system | |
| JP3605469B2 (en) | Communication method, communication terminal device | |
| CN114039834B (en) | Processing method and device for realizing one-key diagnosis of end-to-end faults of optical network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |