CN110851753B - Website access method, device, equipment and storage medium - Google Patents

Website access method, device, equipment and storage medium Download PDF

Info

Publication number
CN110851753B
CN110851753B CN201911083339.1A CN201911083339A CN110851753B CN 110851753 B CN110851753 B CN 110851753B CN 201911083339 A CN201911083339 A CN 201911083339A CN 110851753 B CN110851753 B CN 110851753B
Authority
CN
China
Prior art keywords
target
address
website
pool
access request
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911083339.1A
Other languages
Chinese (zh)
Other versions
CN110851753A (en
Inventor
杨康
吴迪
鲁友炳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
17win Network Technology Co ltd
Original Assignee
17win Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 17win Network Technology Co ltd filed Critical 17win Network Technology Co ltd
Priority to CN201911083339.1A priority Critical patent/CN110851753B/en
Publication of CN110851753A publication Critical patent/CN110851753A/en
Application granted granted Critical
Publication of CN110851753B publication Critical patent/CN110851753B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
    • G06F16/972—Access to data in other repository systems, e.g. legacy data or dynamic Web page generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/957—Browsing optimisation, e.g. caching or content distillation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The application discloses a method, a device, equipment and a medium for accessing websites, wherein the method comprises the following steps: a plurality of IP addresses are obtained in advance and stored in an IP pool; when an access request to a target website initiated by a client is received, selecting a target IP address from an IP pool according to a preset allocation strategy, and forwarding the access request to a target proxy server corresponding to the target IP address; the control target proxy server initiates access to the target website. Therefore, when the same client initiates access requests to the target website for multiple times at high frequency, for the target website, the received access requests with different target IP addresses are received, so that multiple times of high-frequency access to the target website by the same IP address is avoided, and the risk that the IP address is sealed off by the target website is reduced; a large number of standby IP addresses exist in the IP pool, so that access interruption is relatively avoided; manual operation of a user is avoided, so that convenience and stability of accessing websites can be improved.

Description

Website access method, device, equipment and storage medium
Technical Field
The present application relates to the field of website management, and in particular, to a method, apparatus, device and computer readable storage medium for accessing a website.
Background
Currently, when a user accesses a target website through a browser or a crawler of a client, if the access amount is large or the access request is too frequent, the target website is easily judged as an attack, so that the target website can seal the IP address of the client. At this time, when the user initiates the request again through the IP address, the request will be directly rejected by the target website. In the case of IP sealed, the client will not be able to access the target website for a long time, which will result in an interruption of access.
At present, the risk of sealing the IP address is generally avoided by reducing the access frequency, which cannot solve the actual access requirement of the user. In the prior art, aiming at the situations of large access quantity and frequent access requests, generally, after the current IP address is sealed, a user manually sets a new IP address again, so that a client can continuously access a target website by using the new IP address; when the new IP address is sealed, the new IP address is set again, and so on. Therefore, the prior art cannot solve the problem that when the access amount is large or the access request is frequent, the new IP address is sealed again, so that the new IP address needs to be set continuously, the operation process is repeated and complicated, and the process of accessing the target website is complicated and unstable.
Therefore, how to improve the convenience and stability of accessing the website under the conditions of large access amount and high access frequency is a technical problem that needs to be solved by the person skilled in the art at present.
Disclosure of Invention
Accordingly, the application aims to provide a website access method which can improve the convenience and stability of accessing websites under the conditions of large access quantity and high access frequency; another object of the present application is to provide a website accessing apparatus, a website accessing device, and a computer readable storage medium, which have the above advantages.
In order to solve the technical problems, the application provides a website access method, which comprises the following steps:
a plurality of IP addresses are obtained in advance and stored in an IP pool;
when an access request to a target website initiated by a client is received, selecting a target IP address from the IP pool according to a preset allocation strategy, and forwarding the access request to a target proxy server corresponding to the target IP address;
and controlling the target proxy server to initiate access to the target website.
Preferably, the method further comprises:
setting identification information according to domain names of different websites in advance; the identification information is used for representing the performance requirement of the website on the IP address;
correspondingly, when an access request to a target website initiated by a client is received, selecting a target IP address from the IP pool according to a preset allocation policy, and forwarding the access request to a target proxy server corresponding to the target IP address, specifically:
when the access request to the target website initiated by the client is received, selecting a target IP address meeting the performance requirement of the target website from the IP pool according to target identification information corresponding to the target website;
and forwarding the access request to a target proxy server corresponding to the target IP address.
Preferably, the IP pool includes two performance IP pools, and the corresponding process of acquiring a plurality of IP addresses in advance and storing the IP addresses in the IP pool specifically includes:
acquiring a plurality of IP addresses in advance and storing each IP address in an IP pool of corresponding performance according to the performance of each IP address;
correspondingly, when receiving an access request to a target website initiated by a client, selecting a target IP address meeting the performance requirement of the target website from the IP pool according to the identification information, specifically:
when the access request to the target website initiated by the client is received, selecting the target IP address meeting the performance requirement of the target website from an IP pool with corresponding performance according to the identification information.
Preferably, the process of acquiring a plurality of IP addresses in advance and storing the IP addresses in an IP pool specifically includes:
and collecting the changed IP address in the current computer network, and storing the collected IP address in the IP pool.
Preferably, the process of acquiring a plurality of IP addresses in advance and storing the IP addresses in an IP pool specifically includes:
and collecting the IP addresses from the web pages through the Internet, and storing the collected IP addresses into the IP pool.
Preferably, the method further comprises:
detecting the performance of each IP address in each IP pool of the performance according to a preset time period;
updating each IP pool of the performance according to the performance condition detected by each IP address;
and deleting the detected unavailable IP address and the outdated IP address.
Preferably, when the access request is an http request, the process of selecting a target IP address from the IP pool according to a preset allocation policy and forwarding the access request to a target proxy server corresponding to the target IP address when the access request initiated by the client to the target website is received is specifically:
when the access request to the target website initiated by the client is received, analyzing the access request to obtain user information;
judging whether the user information is consistent with the user information of the last access request;
if yes, selecting the IP address corresponding to the last access request from the IP pool as a target IP address, and forwarding the access request to a target proxy server corresponding to the target IP address.
In order to solve the technical problem, the application also provides a website access device, which comprises:
the acquisition module is used for acquiring a plurality of IP addresses in advance and storing the IP addresses in the IP pool;
the setting module is used for selecting a target IP address from the IP pool according to a preset allocation strategy when an access request to a target website initiated by a client is received, and forwarding the access request to a target proxy server corresponding to the target IP address;
and the access module is used for controlling the target proxy server to initiate access to the target website.
In order to solve the technical problem, the present application further provides a website access device, including:
a memory for storing a computer program;
and the processor is used for realizing the steps of any website access method when executing the computer program.
In order to solve the above technical problem, the present application further provides a computer readable storage medium, where a computer program is stored, where the computer program when executed by a processor implements the steps of the method for accessing any website described above.
The application provides a website access method, which is characterized in that a plurality of IP addresses are obtained in advance and stored in an IP pool; when an access request to a target website initiated by a client is received, selecting a target IP address from an IP pool according to a preset allocation strategy, and forwarding the access request to a target proxy server corresponding to the target IP address; the target proxy server is then controlled to initiate access to the target website. Therefore, when the same client initiates access requests to the target website for multiple times at high frequency, for the target website, the received access requests with different target IP addresses are received, so that multiple times of high-frequency access to the target website by the same IP address is avoided, and the risk that the IP address is sealed off by the target website is reduced; in addition, even if a certain IP address is sealed, a plurality of IP addresses are prestored in the IP pool, namely a large number of standby IP addresses exist, and access interruption cannot be caused, so that the method can improve the convenience and stability of accessing websites; in addition, the method avoids manual operation of the user, and further improves convenience in accessing the target website.
In order to solve the technical problems, the application also provides a website access device, equipment and a computer readable storage medium, which have the beneficial effects.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a method for accessing a website according to an embodiment of the present application;
FIG. 2 is a schematic process diagram of a method for accessing a website according to an embodiment of the present application;
FIG. 3 is a block diagram of a website access device according to an embodiment of the present application;
fig. 4 is a block diagram of a website access device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The core of the embodiment of the application is to provide a website access method, which can improve the convenience and stability of accessing websites under the conditions of large access quantity and high access frequency; another core of the present application is to provide a website accessing apparatus, a website accessing device, and a computer readable storage medium, which all have the above advantages.
In order that those skilled in the art will better understand the present application, the following description of the present application will be taken with reference to the accompanying drawings and detailed description.
Fig. 1 is a flowchart of a method for accessing a website according to an embodiment of the present application. As shown in fig. 1, a method for accessing a website includes:
s10: a plurality of IP addresses are acquired in advance and stored in an IP pool.
It should be noted that the method is generally applied to a fixed proxy server of a client. There are multiple proxy servers of the client, i.e., a proxy server cluster. The present implementation selects one from a plurality of proxy servers as a fixed proxy server. Therefore, when one proxy server is not available, then one proxy server is selected from the proxy server cluster as the fixed proxy server.
In this embodiment, an IP pool for storing IP addresses is previously established, a plurality of IP addresses are acquired, and then the acquired IP addresses are stored in the IP pool. In actual operation, the specific number of the obtained IP addresses is not limited, and generally, the more IP addresses are obtained in advance, the more options as target IP addresses can be selected later, the less frequency of each IP address being selected, and thus the access amount and the access frequency corresponding to each IP address are reduced. And, in case that a certain amount of IP addresses have been collected in advance to perform a subsequent operation, the IP addresses may also be continuously collected during the subsequent operation to continuously increase the number of IP addresses in the IP pool.
It should be noted that, there are various ways of obtaining the IP address, for example, the IP address published in the web page may be searched through the internet, or the IP address may be purchased from a manufacturer that collects the IP address, or the IP address that changes in the current computer network may be collected, which is not limited in this embodiment. It should be noted that, generally, IP addresses with fast response and high stability are collected as much as possible, so as to improve the efficiency of accessing the target website.
S20: when an access request to a target website initiated by a client is received, selecting a target IP address from an IP pool according to a preset allocation strategy, and forwarding the access request to a target proxy server corresponding to the target IP address;
s30: the control target proxy server initiates access to the target website.
Specifically, the client includes a browser or a crawler. When an access request to a target website initiated by a client is received, the fixed proxy server firstly analyzes the access request and analyzes the access request to obtain user information; the fixed proxy server selects a target IP address from the IP pool according to the user information and a preset allocation strategy, and the target IP address corresponds to the target proxy server, so that the target proxy server is obtained; the fixed proxy server then forwards the access request to the target proxy server and controls the proxy server to initiate access to the target website based on the access request. Here, the manner in which the target proxy server initiates access to the target website is common knowledge of those skilled in the art, and thus will not be described in detail.
The access method of the website provided by the embodiment of the application comprises the steps of obtaining a plurality of IP addresses in advance and storing the IP addresses in an IP pool; when an access request to a target website initiated by a client is received, selecting a target IP address from an IP pool according to a preset allocation strategy, and forwarding the access request to a target proxy server corresponding to the target IP address; the target proxy server is then controlled to initiate access to the target website. Therefore, when the same client initiates access requests to the target website for multiple times at high frequency, for the target website, the received access requests with different target IP addresses are received, so that multiple times of high-frequency access to the target website by the same IP address is avoided, and the risk that the IP address is sealed off by the target website is reduced; in addition, even if a certain IP address is sealed, a plurality of IP addresses are prestored in the IP pool, namely a large number of standby IP addresses exist, and access interruption cannot be caused, so that the method can improve the convenience and stability of accessing websites; in addition, the method avoids manual operation of the user, and further improves convenience in accessing the target website.
On the basis of the above embodiment, the technical solution is further described and optimized in this embodiment, and specifically, this embodiment further includes:
setting identification information according to domain names of different websites in advance; the identification information is used for representing the performance requirement of the website on the IP address;
correspondingly, when an access request to a target website initiated by a client is received, selecting a target IP address from an IP pool according to a preset allocation policy, and forwarding the access request to a target proxy server corresponding to the target IP address, wherein the process specifically comprises:
when an access request to a target website initiated by a client is received, selecting a target IP address meeting the performance requirement of the target website from an IP pool according to target identification information corresponding to the target website;
and forwarding the access request to a target proxy server corresponding to the target IP address.
The present embodiment further considers that the performance requirements for the target IP address are different when accessing different websites, as the requirements for accessing different websites are different. In this embodiment, identification information is set in advance according to domain names of different websites, where each identification information respectively indicates performance requirements of the corresponding website on the IP address, for example, website a requires an IP address with better performance, and website B does not require performance of the IP address.
Correspondingly, when an access request to a target website initiated by a client is received, firstly acquiring target identification information of a domain name of the target website, and then selecting a target IP address meeting the performance requirement of the target website from an IP pool according to the target identification information; that is, an IP address corresponding to the target identification information is selected as the target IP address; and determining a corresponding target proxy server according to the target IP address, and forwarding the access request to the corresponding target proxy server according to the target IP address.
Therefore, according to the embodiment, the corresponding target IP address is selected according to the performance requirement of the target website on the IP address, so that each IP address can be utilized to the greatest extent.
On the basis of the above embodiment, the present embodiment further describes and optimizes the technical solution, and specifically, the IP pool includes two IP pools with two performances, and the corresponding process of acquiring a plurality of IP addresses in advance and storing the IP addresses in the IP pool specifically includes:
acquiring a plurality of IP addresses in advance and storing each IP address in an IP pool of corresponding performance according to the performance of each IP address;
correspondingly, when an access request to a target website initiated by a client is received, selecting a target IP address meeting the performance requirement of the target website from an IP pool according to the identification information, wherein the process specifically comprises the following steps:
when an access request to a target website initiated by a client is received, selecting a target IP address meeting the performance requirement of the target website from an IP pool of corresponding performance according to the identification information.
Specifically, in actual operation, a plurality of IP pools for storing IP addresses with different performances may be preset, and the number of types of the IP pools to be set is not limited in this embodiment.
As a preferred embodiment, the present embodiment preferably sets two IP pools for storing IP addresses for both capabilities. When the IP address is acquired, the IP address is stored in the first performance pool, each IP address in the first performance pool is detected through a preset detection rule, and the IP address meeting the preset condition in the first performance pool is transferred to the second performance pool, so that the IP addresses with different performances are stored in different IP pools.
When an access request to a target website initiated by a client is received, selecting a target IP address meeting the performance requirement of the target website from an IP pool of corresponding performance according to the identification information.
Therefore, the target IP address can be acquired from the corresponding IP pool more conveniently and accurately by classifying and storing the IP addresses in advance, so that the operation process is more convenient and accurate.
As a preferred embodiment, the present embodiment further includes:
detecting the performance of each IP address in each performance IP pool according to a preset time period;
updating the IP pool of each performance according to the performance condition detected by each IP address;
and deleting the detected unavailable IP address and the outdated IP address.
In this embodiment, the performance of each IP address in the IP pool of each performance is further detected according to a preset time period. Specifically, by detecting the performance of each IP address in each performance IP pool, and judging whether the performance of each IP address reaches a first preset condition or not; if yes, storing the IP address in a second performance IP pool; if not, judging whether the IP address is unavailable or outdated, if so, deleting the IP address. Deleting the IP address includes clearing the IP address or storing the IP address in a discard IP pool.
It should be noted that, the manner of detecting the performance of the IP address may be: after a certain IP address is selected as a target IP address and the operation of accessing the target website is executed, feedback information of the access operation is received according to a feedback strategy, and the performance of the target IP address is calculated according to the time consumption of access, whether errors occur or not and other information in the feedback information.
According to the method and the device, the IP addresses in the IP pools are further updated according to the preset time period, so that the availability and the performance of the IP addresses in the IP pools are maintained, the corresponding IP addresses meeting the performance requirements can be selected according to different IP pools, the subsequent selection of unavailable or outdated IP addresses can be avoided, and the reliability and the stability of the access target website are further improved.
On the basis of the above embodiment, the present embodiment further describes and optimizes a technical solution, specifically, a process of acquiring a plurality of IP addresses in advance and storing the IP addresses in an IP pool, which specifically includes:
the method comprises the steps of collecting the changed IP addresses in the current computer network, and storing the collected IP addresses in an IP pool.
Specifically, the method for acquiring multiple IP addresses provided in this embodiment is to collect the changed IP addresses in the computer network where the client is currently located. It will be appreciated that in a computer network, the proxy servers corresponding to other clients will change their corresponding IP addresses when rejoining the computer network. It is by this feature that the present embodiment collects each changed IP address, so that a plurality of IP addresses can be obtained.
Or, a process of acquiring a plurality of IP addresses in advance and storing the IP addresses in an IP pool specifically includes:
and collecting the IP addresses from the web pages through the Internet, and storing the collected IP addresses into an IP pool.
The method for acquiring the IP address provided in this embodiment acquires the IP address from the web page through the internet. It should be noted that, in the current web page, there are some published IP addresses, so in this embodiment, the IP addresses are obtained from the web page through the internet by retrieving and collecting.
Therefore, the method for acquiring the plurality of IP addresses provided in this embodiment is convenient and reliable to operate.
On the basis of the above embodiment, the present embodiment further describes and optimizes the technical solution, specifically, when an access request is an http request, and when an access request to a target website initiated by a client is received, a target IP address is selected from an IP pool according to a preset allocation policy, and the access request is forwarded to a target proxy server corresponding to the target IP address, which specifically includes:
when an access request to a target website initiated by a client is received, the access request is analyzed to obtain user information;
judging whether the user information is consistent with the user information of the last access request;
if yes, selecting the IP address corresponding to the last access request from the IP pool as a target IP address, and forwarding the access request to a target proxy server corresponding to the target IP address.
In this embodiment, it is further considered that when the access request is an http request, there are cases where two consecutive access requests require the use of the same target IP address, for example, when accessing a shopping website, the logged-in access request and the access request of the order require the same target IP address, otherwise, the order-placing operation cannot be implemented.
Therefore, when an access request to a target website initiated by a client is received, the access request is analyzed first to obtain user information; then judging whether the user information is consistent with the user information of the last access request; if so, the access request initiated by the same user is indicated, so that the IP address corresponding to the last access request is selected from the IP pool as a target IP address, and the access request is forwarded to a target proxy server corresponding to the target IP address, namely, the target IP address identical to the last access request is utilized to realize the revisit of the target website.
Therefore, the embodiment provides another method for determining the target IP address, which can meet more actual requirements for accessing the target website.
In order to enable those skilled in the art to better understand the technical solution in the present application, the technical solution in the embodiment of the present application is described in detail below with reference to a process schematic diagram of a website access method and an actual application scenario shown in fig. 2. In the embodiment of the application, the specific steps are as follows:
collecting IP addresses from web pages through Internet by an IP address collection program, and storing the collected IP addresses in different IP pools according to performances;
detecting the performance of each IP address in each performance IP pool according to a preset time period, updating each performance IP pool according to the performance condition detected by each IP address, and removing the detected unavailable IP address and the outdated IP address into a waste IP pool;
when an access request to a target website initiated by a browser/crawler is received, selecting a target IP address meeting the performance requirement of the target website from an IP pool according to target identification information corresponding to the target website;
controlling a target proxy server to initiate access to a target website;
and receiving feedback information after the target IP address is used for accessing the target website, calculating the performance of the target IP address according to the time consumption, error or other information in the feedback information, and determining an IP pool corresponding to the target IP address according to the obtained performance of the target IP address.
The application provides a website access method, which is characterized in that a plurality of IP addresses are obtained in advance and stored in an IP pool; when an access request to a target website initiated by a client is received, selecting a target IP address from an IP pool according to a preset allocation strategy, and forwarding the access request to a target proxy server corresponding to the target IP address; the target proxy server is then controlled to initiate access to the target website. Therefore, when the same client initiates access requests to the target website for multiple times at high frequency, for the target website, the received access requests with different target IP addresses are received, so that multiple times of high-frequency access to the target website by the same IP address is avoided, and the risk that the IP address is sealed off by the target website is reduced; in addition, even if a certain IP address is sealed, a plurality of IP addresses are prestored in the IP pool, namely a large number of standby IP addresses exist, and access interruption cannot be caused, so that the method can improve the convenience and stability of accessing websites; in addition, the method avoids manual operation of the user, and further improves convenience in accessing the target website.
The embodiments of the method for accessing a website provided by the present application are described in detail, and the present application also provides an apparatus, a device, and a computer readable storage medium for accessing a website corresponding to the method.
Fig. 3 is a block diagram of a website accessing apparatus according to an embodiment of the present application, where, as shown in fig. 3, the website accessing apparatus includes:
an acquisition module 31 for acquiring a plurality of IP addresses in advance and storing in an IP pool;
the setting module 32 is configured to select, when an access request to a target website initiated by a client is received, a target IP address from an IP pool according to a preset allocation policy, and forward the access request to a target proxy server corresponding to the target IP address;
and the access module 33 is used for controlling the target proxy server to initiate access to the target website.
The website access device provided by the embodiment of the application has the beneficial effects of the website access method.
Fig. 4 is a block diagram of a website access device according to an embodiment of the present application, where, as shown in fig. 4, the website access device includes:
a memory 41 for storing a computer program;
a processor 42 for implementing the steps of the method of accessing a web site as described above when executing a computer program.
The website access device provided by the embodiment of the application has the beneficial effects of the website access method.
In order to solve the above technical problem, the present application further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the method for accessing a website as described above.
The computer readable storage medium provided by the embodiment of the application has the beneficial effects of the access method of the website.
The method, the device, the equipment and the computer readable storage medium for accessing the website provided by the application are described in detail above. The principles and embodiments of the present application have been described herein with reference to specific examples, which are intended to be merely illustrative of the methods of the present application and their core ideas. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the application can be made without departing from the principles of the application and these modifications and adaptations are intended to be within the scope of the application as defined in the following claims.
In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

Claims (6)

1. A method for accessing a web site, comprising:
a plurality of IP addresses are obtained in advance and stored in an IP pool;
when an access request to a target website initiated by a client is received, selecting a target IP address from the IP pool according to a preset allocation strategy, and forwarding the access request to a target proxy server corresponding to the target IP address;
controlling the target proxy server to initiate access to a target website;
when the access request is an http request, selecting a target IP address from the IP pool according to a preset allocation policy when the access request initiated by the client to the target website is received, and forwarding the access request to a target proxy server corresponding to the target IP address, where the process specifically includes:
when the access request to the target website initiated by the client is received, analyzing the access request to obtain user information;
judging whether the user information is consistent with the user information of the last access request;
if yes, selecting an IP address corresponding to the last access request from the IP pool as a target IP address, and forwarding the access request to a target proxy server corresponding to the target IP address;
the method for accessing the website further comprises the following steps:
setting identification information according to domain names of different websites in advance; the identification information is used for representing the performance requirement of the website on the IP address;
correspondingly, when an access request to a target website initiated by a client is received, selecting a target IP address from the IP pool according to a preset allocation policy, and forwarding the access request to a target proxy server corresponding to the target IP address, specifically:
when the access request to the target website initiated by the client is received, selecting a target IP address meeting the performance requirement of the target website from the IP pool according to target identification information corresponding to the target website; forwarding the access request to a target proxy server corresponding to the target IP address;
the process of acquiring a plurality of IP addresses in advance and storing the IP addresses in the IP pool specifically comprises the following steps:
acquiring a plurality of IP addresses in advance and storing each IP address in an IP pool of corresponding performance according to the performance of each IP address;
correspondingly, when the access request to the target website initiated by the client is received, selecting a target IP address meeting the performance requirement of the target website from the IP pool according to target identification information corresponding to the target website, specifically:
when the access request to the target website initiated by the client is received, selecting the target IP address meeting the performance requirement of the target website from an IP pool with corresponding performance according to the target identification information;
the method for accessing the website further comprises the following steps:
detecting the performance of each IP address in each IP pool of the performance according to a preset time period;
updating each IP pool of the performance according to the performance condition detected by each IP address;
and deleting the detected unavailable IP address and the outdated IP address.
2. The method according to claim 1, wherein the process of pre-acquiring a plurality of IP addresses and storing the IP addresses in an IP pool specifically comprises:
and collecting the changed IP address in the current computer network, and storing the collected IP address in the IP pool.
3. The method according to claim 1, wherein the process of pre-acquiring a plurality of IP addresses and storing the IP addresses in an IP pool specifically comprises:
and collecting the IP addresses from the web pages through the Internet, and storing the collected IP addresses into the IP pool.
4. An access device for a web site, comprising:
the acquisition module is used for acquiring a plurality of IP addresses in advance and storing the IP addresses in the IP pool;
the setting module is used for selecting a target IP address from the IP pool according to a preset allocation strategy when an access request to a target website initiated by a client is received, and forwarding the access request to a target proxy server corresponding to the target IP address;
the access module is used for controlling the target proxy server to initiate access to a target website;
when the access request is an http request, the setting module is specifically configured to: when the access request to the target website initiated by the client is received, analyzing the access request to obtain user information; judging whether the user information is consistent with the user information of the last access request; if yes, selecting an IP address corresponding to the last access request from the IP pool as a target IP address, and forwarding the access request to a target proxy server corresponding to the target IP address;
wherein, the access device of website is further used for: setting identification information according to domain names of different websites in advance; the identification information is used for representing the performance requirement of the website on the IP address;
correspondingly, the setting module is specifically configured to: when the access request to the target website initiated by the client is received, selecting a target IP address meeting the performance requirement of the target website from the IP pool according to target identification information corresponding to the target website; forwarding the access request to a target proxy server corresponding to the target IP address;
the IP pool comprises two types of IP pools, and the acquisition module is specifically used for: acquiring a plurality of IP addresses in advance and storing each IP address in an IP pool of corresponding performance according to the performance of each IP address;
correspondingly, the process of selecting the target IP address meeting the performance requirement of the target website from the IP pool according to the target identification information corresponding to the target website in the setting module comprises the following steps: when the access request to the target website initiated by the client is received, selecting the target IP address meeting the performance requirement of the target website from an IP pool with corresponding performance according to the target identification information;
wherein, the access device of website is further used for: detecting the performance of each IP address in each IP pool of the performance according to a preset time period; updating each IP pool of the performance according to the performance condition detected by each IP address; and deleting the detected unavailable IP address and the outdated IP address.
5. An access device for a web site, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the method of accessing a website according to any one of claims 1 to 3 when executing the computer program.
6. A computer readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, implements the steps of the method of accessing a website according to any of claims 1 to 3.
CN201911083339.1A 2019-11-07 2019-11-07 Website access method, device, equipment and storage medium Active CN110851753B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911083339.1A CN110851753B (en) 2019-11-07 2019-11-07 Website access method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911083339.1A CN110851753B (en) 2019-11-07 2019-11-07 Website access method, device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN110851753A CN110851753A (en) 2020-02-28
CN110851753B true CN110851753B (en) 2023-09-26

Family

ID=69599699

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911083339.1A Active CN110851753B (en) 2019-11-07 2019-11-07 Website access method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN110851753B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113395368B (en) * 2020-03-13 2024-05-03 阿里巴巴集团控股有限公司 Access configuration method, access method and device
CN111405039A (en) * 2020-03-16 2020-07-10 深圳市网心科技有限公司 Data transparent transmission method, device and system, client and server
CN113438244B (en) * 2021-06-28 2022-08-19 安天科技集团股份有限公司 Penetration testing method and device, computing equipment and storage medium
CN113343155A (en) * 2021-06-29 2021-09-03 北京天空卫士网络安全技术有限公司 Request processing method and device
CN113923260B (en) * 2021-09-28 2024-01-09 盐城天眼察微科技有限公司 Method, device, terminal and storage medium for processing agent environment
CN113905092B (en) * 2021-09-28 2024-03-22 盐城天眼察微科技有限公司 Method, device, terminal and storage medium for determining reusable agent queue
CN114285899A (en) * 2021-11-30 2022-04-05 同程网络科技股份有限公司 Flow proxy method, system, intelligent terminal and storage medium
CN114189385A (en) * 2021-12-14 2022-03-15 杭州安恒信息技术股份有限公司 Flow distribution method, device, equipment and computer readable storage medium
CN115396509A (en) * 2022-08-09 2022-11-25 上海宝创网络科技有限公司 IPv6 network proxy service-based method and equipment for processing access limitation
CN115314462A (en) * 2022-08-09 2022-11-08 上海宝创网络科技有限公司 Processing method and equipment for high-concurrency access based on IPv6 network proxy service

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102624920A (en) * 2012-03-31 2012-08-01 奇智软件（北京）有限公司 Method and device for performing access through proxy server
CN103973682A (en) * 2014-04-30 2014-08-06 北京奇虎科技有限公司 Method and device for having access to webpage
CN104901943A (en) * 2012-03-31 2015-09-09 北京奇虎科技有限公司 Method and system for accessing website

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102624920A (en) * 2012-03-31 2012-08-01 奇智软件（北京）有限公司 Method and device for performing access through proxy server
CN104901943A (en) * 2012-03-31 2015-09-09 北京奇虎科技有限公司 Method and system for accessing website
CN103973682A (en) * 2014-04-30 2014-08-06 北京奇虎科技有限公司 Method and device for having access to webpage

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
储凡静 ; 刘方爱 ; 孙晓玲 ; .一种基于IP限制的网站管理方法.信息技术与信息化.2006,(06),全文. *

Also Published As

Publication number Publication date
CN110851753A (en) 2020-02-28

Similar Documents

Publication Publication Date Title
CN110851753B (en) Website access method, device, equipment and storage medium
US10791190B2 (en) Systems and methods for avoiding server push of objects already cached at a client
US8321503B2 (en) Context-specific network resource addressing model for distributed services
US9769248B1 (en) Performance-based content delivery
US9208097B2 (en) Cache optimization
CN103051740B (en) Domain name analytic method, dns server and domain name analysis system
CN110519409B (en) Domain name dynamic configuration method, device, equipment and storage medium
CN106302842B (en) Domain name resolution method, device and system
US9948709B2 (en) Using resource timing data for server push in multiple web page transactions
US20130346539A1 (en) Client side cache management
US20120185609A1 (en) Presenting multiple possible selectable domain names from a url entry
US8413208B2 (en) Access control and protocol/policy determination based on the client's environment
CN111314499A (en) Domain name proxy method, device, equipment and readable storage medium
US20210336861A1 (en) Detection method and detection device for detecting quality of service of bgp anycast cluster
WO2017166524A1 (en) Domain name parsing method and apparatus
CN112291365A (en) Access balance processing method and device, computer equipment and storage medium
CN107301084B (en) Virtual machine migration method and device of cluster server, server and storage medium
CN110674427B (en) Method, device, equipment and storage medium for responding to webpage access request
CN113938464B (en) Access request method and terminal
EP1387301A2 (en) Establishment of network connections
CN113766013B (en) Session creation method, device, equipment and storage medium
CN109471713B (en) Method and device for inquiring information
CN109446445A (en) A kind of resource acquiring method and device
CN113055420B (en) HTTPS service identification method and device and computing equipment
US7890487B1 (en) Facilitating client-side data-management for web-based applications

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant