CN111355693A - Method and device for realizing proxy service, electronic equipment and storage medium - Google Patents

Method and device for realizing proxy service, electronic equipment and storage medium Download PDF

Info

Publication number
CN111355693A
CN111355693A CN201811581126.7A CN201811581126A CN111355693A CN 111355693 A CN111355693 A CN 111355693A CN 201811581126 A CN201811581126 A CN 201811581126A CN 111355693 A CN111355693 A CN 111355693A
Authority
CN
China
Prior art keywords
proxy
target equipment
service
equipment
authentication information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811581126.7A
Other languages
Chinese (zh)
Other versions
CN111355693B (en
Inventor
赵化强
杨霖
钟健男
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Qihoo Technology Co Ltd
Original Assignee
Beijing Qihoo Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Qihoo Technology Co Ltd filed Critical Beijing Qihoo Technology Co Ltd
Priority to CN201811581126.7A priority Critical patent/CN111355693B/en
Publication of CN111355693A publication Critical patent/CN111355693A/en
Application granted granted Critical
Publication of CN111355693B publication Critical patent/CN111355693B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/0227—Filtering policies
    • H04L63/0236—Filtering by address, protocol, port number or service, e.g. IP-address or URL
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/0281—Proxies
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
    • H04L63/083—Network architectures or network communication protocols for network security for authentication of entities using passwords
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
    • H04L63/0876—Network architectures or network communication protocols for network security for authentication of entities based on the identity of the terminal or configuration, e.g. MAC address, hardware or software configuration or device fingerprint

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Power Engineering (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a method and a device for realizing proxy service, electronic equipment and a storage medium. The method comprises the following steps: receiving a service request containing verification information; determining target equipment used as a proxy outlet according to the verification information, and establishing proxy connection with the target equipment; and realizing the transmission of the service data based on the proxy connection. On one hand, the authentication information is utilized to carry out normal identity authentication of the proxy service, and on the other hand, the target equipment used as the proxy outlet can be determined according to the authentication information, so that when the transmission of the service data is realized based on the proxy connection established with the target equipment, the fixed target equipment can be used as the proxy outlet, and the instability of the access process caused by the change of the outlet ip in the access process is avoided; the target equipment can be determined by using the verification information, advanced screening can be realized, excessive improvement on a primary management protocol is not needed, a new functional module is not needed, the concept is ingenious, the application is wide, the existing network architecture is not needed to be changed greatly, and the method is very convenient.

Description

Method and device for realizing proxy service, electronic equipment and storage medium
Technical Field
The invention relates to the technical field of networks, in particular to a method and a device for realizing proxy service, electronic equipment and a storage medium.
Background
A proxy service is a special network service that allows one network terminal to make an indirect connection with another network terminal through the service. Some network devices such as gateways and routers have network proxy functions. The proxy service is generally considered to be beneficial to ensuring the privacy or security of the network terminal and preventing attacks.
At present, when a network terminal is connected with another network terminal through a proxy server, great randomness exists, so that a plurality of network outlets are adopted in one network access process, and inconvenience is caused. For example, in a process of accessing a specific site using a proxy service, access is performed using a plurality of IP addresses, and the specific site is easily determined to be an illegal access behavior.
Disclosure of Invention
In view of the above, the present invention has been made to provide a method, an apparatus, an electronic device and a storage medium for implementing a proxy service that overcome or at least partially solve the above problems.
According to an aspect of the present invention, there is provided a method for implementing a proxy service, including:
receiving a service request containing verification information;
determining target equipment used as a proxy outlet according to the verification information, and establishing proxy connection with the target equipment;
and realizing the transmission of service data based on the proxy connection.
Optionally, the service request is a service request related to a crawler;
the transmission of the service data based on the proxy connection comprises the following steps:
and taking the IP address of the target device as an exit IP of the crawler, and transmitting the crawling request and the crawling content through the proxy connection.
Optionally, the authentication information includes a user name and a password, and the determining, according to the authentication information, the target device serving as the proxy exit includes:
analyzing the user name, and judging whether new target equipment needs to be allocated or not according to an analysis result;
and if new target equipment needs to be distributed, selecting matched equipment as the target equipment according to the proxy mode corresponding to the analysis result.
Optionally, the parsing the user name includes:
analyzing a session name from the user name;
the judging whether new target equipment needs to be allocated according to the analysis result comprises the following steps:
and judging whether the target equipment corresponding to the session name is recorded, if so, not allocating new target equipment, otherwise, allocating new target equipment.
Optionally, the establishing a proxy connection with the target device includes:
and establishing a proxy session according to the session name, and recording the session name and the corresponding target equipment.
Optionally, the proxy mode includes: a random mode and a binding mode;
the selecting the matched device as the target device according to the proxy mode corresponding to the analysis result comprises:
in a random mode, randomly distributing equipment as target equipment;
in the binding mode, the designated device is taken as the target device.
Optionally, the parsing the user name includes: analyzing service requirements or equipment information of specified equipment from the user name;
the random allocation device as a target device includes: and randomly allocating the equipment meeting the service requirement as target equipment.
Optionally, the proxy service is implemented based on the socks5 protocol.
According to another aspect of the present invention, there is provided an apparatus for implementing a proxy service, including:
a receiving unit adapted to receive a service request containing authentication information;
a proxy connection establishing unit adapted to determine a target device serving as a proxy exit according to the authentication information, and establish a proxy connection with the target device;
and the proxy unit is suitable for realizing the transmission of the service data based on the proxy connection.
Optionally, the service request is a service request related to a crawler;
and the proxy unit is suitable for transmitting the crawling request and the crawling content through the proxy connection by taking the IP address of the target device as an exit IP of the crawler.
Optionally, the authentication information includes a user name and a password;
the proxy connection establishing unit is suitable for analyzing the user name and judging whether new target equipment needs to be allocated or not according to an analysis result; and if new target equipment needs to be distributed, selecting matched equipment as the target equipment according to the proxy mode corresponding to the analysis result.
Optionally, the proxy connection establishing unit is adapted to parse a session name from the user name; and judging whether the target equipment corresponding to the session name is recorded, if so, not allocating new target equipment, otherwise, allocating new target equipment.
Optionally, the proxy connection establishing unit is adapted to establish a proxy session according to the session name, and record the session name and a corresponding target device.
Optionally, the proxy mode includes: a random mode and a binding mode;
the proxy connection establishing unit is suitable for randomly distributing equipment as target equipment in a random mode; in the binding mode, the designated device is taken as the target device.
Optionally, the proxy connection establishing unit is adapted to analyze a service requirement or device information of a specified device from the user name; and randomly allocating the equipment meeting the service requirement as target equipment.
Optionally, the proxy service is implemented based on the socks5 protocol.
In accordance with still another aspect of the present invention, there is provided an electronic apparatus including: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform a method as any one of the above.
According to a further aspect of the invention, there is provided a computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement a method as any one of the above.
Therefore, according to the technical scheme of the invention, after the service request containing the verification information is received, on one hand, the verification information is utilized to carry out normal identity verification of the proxy service, and on the other hand, the target equipment used as the proxy outlet can be determined according to the verification information, so that when the transmission of the service data is realized based on the proxy connection established with the target equipment, the fixed target equipment can be used as the proxy outlet, and the instability of the access process caused by the change of the outlet ip in the access process is avoided; the target equipment can be determined by using the verification information, advanced screening can be realized, for example, certain province, certain city, certain type of target equipment and the like are selected, excessive improvement on a primary management protocol is not needed, a new functional module is not needed to be built, the concept is ingenious, the application is wide, the existing network architecture is not needed to be changed greatly, and the method is very convenient.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart illustrating a method for implementing a proxy service according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an apparatus for implementing a proxy service according to an embodiment of the present invention;
FIG. 3 shows a schematic structural diagram of an electronic device according to one embodiment of the invention;
fig. 4 shows a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Fig. 1 is a flowchart illustrating a method for implementing a proxy service according to an embodiment of the present invention. As shown in fig. 1, the method includes:
step S110, receiving a service request containing authentication information.
Here, the authentication information may include a user name and a password for authentication in the proxy service process.
And step S120, determining the target device used as a proxy exit according to the verification information, and establishing proxy connection with the target device.
In this step, the determination of the target device is achieved using the verification information. That is, in the embodiment of the present invention, the authentication information may be in a specific format, and may be used not only for authentication but also for determination of the target device. The subsequent process of establishing the proxy connection can be implemented by referring to the prior art, and is not described herein again.
And step S130, realizing the transmission of the service data based on the proxy connection.
It can be seen that, in the method shown in fig. 1, after receiving a service request including authentication information, on one hand, the authentication information is used to perform normal identity authentication of proxy service, and on the other hand, a target device serving as a proxy exit can be determined according to the authentication information, so that when service data is transmitted based on a proxy connection established with the target device, the fixed target device can be used as the proxy exit, thereby avoiding instability of an access process due to a change of an exit ip in the access process; the target equipment can be determined by using the verification information, advanced screening can be realized, for example, certain province, certain city, certain type of target equipment and the like are selected, excessive improvement on a primary management protocol is not needed, a new functional module is not needed to be built, the concept is ingenious, the application is wide, the existing network architecture is not needed to be changed greatly, and the method is very convenient.
In an embodiment of the present invention, in the method, the service request is a service request related to a crawler; the transmission of the service data based on the proxy connection comprises the following steps: and taking the IP address of the target device as an exit IP of the crawler, and transmitting the crawling request and the crawling content through the proxy connection.
The crawler is an important tool for acquiring network resources and is also one of core technologies of a search engine. Crawlers, because access behavior is significantly different from normal manual access, often require multiple network exits, where it is often necessary to connect different devices as network exits via a proxy service. In this process, there may be problems pointed out by the background of the present invention due to the large number of available devices: in a crawling process, multiple requests may be generated, for example, crawling resources of several different pages of a certain website; however, for these multiple requests, it is likely that different devices are randomly assigned, resulting in a change in the export IP, which is likely to be unusual for the crawled web site, resulting in a crawl failure.
Therefore, in a specific embodiment of the present invention, a fixed target device is used as a proxy exit in a session process, and the IP address of the target device is used as the exit IP of the crawler, so that the above situation can be effectively avoided, and the stability and reliability of the service are ensured.
In an embodiment of the present invention, in the method, the authentication information includes a user name and a password, and determining the target device serving as the proxy exit according to the authentication information includes: analyzing the user name, and judging whether new target equipment needs to be allocated or not according to an analysis result; and if new target equipment needs to be distributed, selecting matched equipment as the target equipment according to the proxy mode corresponding to the analysis result.
In a specific embodiment of the present invention, the user name may be divided into a plurality of parts, where a part of the content includes information related to target device allocation, and when the user name is analyzed, it may be determined whether a new target device needs to be allocated according to an analysis result as long as the part of the content is analyzed, if not, it indicates that a target device already bound may be used, and if so, a matching device is selected according to a corresponding proxy mode.
In an embodiment of the present invention, in the method, analyzing the user name includes: analyzing a session name from a user name; judging whether new target equipment needs to be allocated according to the analysis result comprises the following steps: and judging whether the target equipment corresponding to the session name is recorded, if so, not allocating new target equipment, otherwise, allocating new target equipment.
In one embodiment of the invention, the session name may be Rand-12345, with the former Rand corresponding to the random pattern in the proxy pattern and the latter 12345 being the session name. If the session name has a corresponding target device, the target device is still used; otherwise the target device needs to be found. In an embodiment of the present invention, in the method, establishing a proxy connection with a target device includes: and establishing a proxy session according to the session name, and recording the session name and the corresponding target equipment.
In an embodiment of the present invention, in the method, the proxy mode includes: a random mode and a binding mode; selecting the matched equipment as the target equipment according to the proxy mode corresponding to the analysis result comprises the following steps: in a random mode, randomly distributing equipment as target equipment; in the binding mode, the designated device is taken as the target device.
In a specific embodiment, the random assignment may randomly select one target device from all available target devices as a proxy egress, and the binding mode designates a certain device as a proxy egress.
Certainly, the random allocation may also follow a certain rule, and in an embodiment of the present invention, the analyzing the user name includes: analyzing a service requirement or equipment information of specified equipment from the user name; the random allocation device as a target device includes: and randomly allocating the equipment meeting the service requirement as target equipment.
For example, if a province, a city, or a certain type of target device is selected, it is only necessary to write information on the province, the city, or the type of device into the user name. For the binding mode, device information of a specified device needs to be written in the user name.
In the embodiment of the invention, the designated equipment can be IoT equipment, namely intelligent equipment in the Internet of things, so that the resources of the Internet of things are effectively utilized; the server side only needs to specify the format of the user name and can accurately analyze the necessary information from the user name. The proxy protocol used can be a common protocol in the prior art, and does not need to add a new functional module or need to be improved too much.
For example, in one embodiment of the present invention, the method described above, the proxy service is implemented based on the socks5 protocol. socks5 may play an intermediary role between a front-end machine and a server machine that communicate using TCP/IP protocols, enabling the front-end machine in an intranet to access a server in the Internet, or making communications more secure.
In a specific embodiment, a crawler program arranged on a server side establishes proxy connection with a target device for crawling through proxy service in a connection transfer server, and a business process is realized.
Fig. 2 is a schematic structural diagram of an apparatus for implementing a proxy service according to an embodiment of the present invention. As shown in fig. 2, the device 200 for implementing the proxy service includes:
the receiving unit 210 is adapted to receive a service request comprising authentication information.
Here, the authentication information may include a user name and a password for authentication in the proxy service process.
A proxy connection establishing unit 220 adapted to determine a target device serving as a proxy exit according to the authentication information, and establish a proxy connection with the target device.
The proxy connection establishing unit 220 achieves the determination of the target device using the authentication information. That is, in the embodiment of the present invention, the authentication information may be in a specific format, and may be used not only for authentication but also for determination of the target device. The subsequent process of establishing the proxy connection can be implemented by referring to the prior art, and is not described herein again.
The proxy unit 230 is adapted to implement transmission of service data based on the proxy connection.
It can be seen that, after receiving a service request including authentication information, the apparatus shown in fig. 2, on one hand, performs normal identity authentication of proxy service using the authentication information, and on the other hand, may determine a target device serving as a proxy exit according to the authentication information, so that when implementing service data transmission based on proxy connection established with the target device, it may use a fixed target device as the proxy exit, thereby avoiding unstable access process due to change of an exit ip in the access process; the target equipment can be determined by using the verification information, advanced screening can be realized, for example, certain province, certain city, certain type of target equipment and the like are selected, excessive improvement on a primary management protocol is not needed, a new functional module is not needed to be built, the concept is ingenious, the application is wide, the existing network architecture is not needed to be changed greatly, and the method is very convenient.
In an embodiment of the present invention, in the above apparatus, the service request is a service request related to a crawler; the proxy unit 230 is adapted to transmit the crawling request and the crawled content through the proxy connection with the IP address of the target device as the exit IP of the crawler.
The crawler is an important tool for acquiring network resources and is also one of core technologies of a search engine. Crawlers, because access behavior is significantly different from normal manual access, often require multiple network exits, where it is often necessary to connect different devices as network exits via a proxy service. In this process, there may be problems pointed out by the background of the present invention due to the large number of available devices: in a crawling process, multiple requests may be generated, for example, crawling resources of several different pages of a certain website; however, for these multiple requests, it is likely that different devices are randomly assigned, resulting in a change in the export IP, which is likely to be unusual for the crawled web site, resulting in a crawl failure.
Therefore, in a specific embodiment of the present invention, a fixed target device is used as a proxy exit in a session process, and the IP address of the target device is used as the exit IP of the crawler, so that the above situation can be effectively avoided, and the stability and reliability of the service are ensured.
In an embodiment of the present invention, in the above apparatus, the authentication information includes a user name and a password; the proxy connection establishing unit 220 is adapted to analyze the user name and determine whether new target equipment needs to be allocated according to an analysis result; and if new target equipment needs to be distributed, selecting matched equipment as the target equipment according to the proxy mode corresponding to the analysis result.
In a specific embodiment of the present invention, the user name may be divided into a plurality of parts, where a part of the content includes information related to target device allocation, and when the user name is analyzed, it may be determined whether a new target device needs to be allocated according to an analysis result as long as the part of the content is analyzed, if not, it indicates that a target device already bound may be used, and if so, a matching device is selected according to a corresponding proxy mode.
In an embodiment of the present invention, in the above apparatus, the proxy connection establishing unit 220 is adapted to parse a session name from a user name; and judging whether the target equipment corresponding to the session name is recorded, if so, not allocating new target equipment, otherwise, allocating new target equipment.
In one embodiment of the invention, the session name may be Rand-12345, with the former Rand corresponding to the random pattern in the proxy pattern and the latter 12345 being the session name. If the session name has a corresponding target device, the target device is still used; otherwise the target device needs to be found. In an embodiment of the present invention, in the above apparatus, the proxy connection establishing unit 220 is adapted to establish a proxy session according to a session name, and record the session name and a corresponding target device.
In an embodiment of the present invention, in the above apparatus, the proxy mode includes: a random mode and a binding mode; a proxy connection establishing unit 220 adapted to randomly allocate a device as a target device in a random mode; in the binding mode, the designated device is taken as the target device.
In a specific embodiment, the random assignment may randomly select one target device from all available target devices as a proxy egress, and the binding mode designates a certain device as a proxy egress.
Certainly, the random allocation may also follow a certain rule, and in an embodiment of the present invention, in the apparatus, the proxy connection establishing unit 220 is adapted to analyze the service requirement or the device information of the specified device from the user name; and randomly allocating the equipment meeting the service requirement as target equipment.
For example, if a province, a city, or a certain type of target device is selected, it is only necessary to write information on the province, the city, or the type of device into the user name. For the binding mode, device information of a specified device needs to be written in the user name.
In the embodiment of the invention, the designated equipment can be IoT equipment, namely intelligent equipment in the Internet of things, so that the resources of the Internet of things are effectively utilized; the server side only needs to specify the format of the user name and can accurately analyze the necessary information from the user name. The proxy protocol used can be a common protocol in the prior art, and does not need to add a new functional module or need to be improved too much.
For example, in one embodiment of the present invention, in the above apparatus, the proxy service is implemented based on the socks5 protocol. socks5 may play an intermediary role between a front-end machine and a server machine that communicate using TCP/IP protocols, enabling the front-end machine in an intranet to access a server in the Internet, or making communications more secure.
In a specific embodiment, a crawler program arranged on a server side establishes proxy connection with a target device for crawling through proxy service in a connection transfer server, and a business process is realized.
In summary, according to the technical solution of the present invention, after receiving a service request including authentication information, on one hand, the authentication information is used to perform normal identity authentication of a proxy service, and on the other hand, a target device serving as a proxy exit can be determined according to the authentication information, so that when service data is transmitted based on a proxy connection established with the target device, the fixed target device can be used as the proxy exit, thereby avoiding instability of an access process caused by a change of an exit ip in the access process; the target equipment can be determined by using the verification information, advanced screening can be realized, for example, certain province, certain city, certain type of target equipment and the like are selected, excessive improvement on a primary management protocol is not needed, a new functional module is not needed to be built, the concept is ingenious, the application is wide, the existing network architecture is not needed to be changed greatly, and the method is very convenient.
It should be noted that:
the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing such a device will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in an implementation of a proxy service according to an embodiment of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
For example, fig. 3 shows a schematic structural diagram of an electronic device according to an embodiment of the invention. The electronic device comprises a processor 310 and a memory 320 arranged to store computer executable instructions (computer readable program code). The memory 320 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. The memory 320 has a storage space 330 storing computer readable program code 331 for performing any of the method steps described above. For example, the storage space 330 for storing the computer readable program code may comprise respective computer readable program codes 331 for respectively implementing various steps in the above method. The computer readable program code 331 may be read from or written to one or more computer program products. These computer program products comprise a program code carrier such as a hard disk, a Compact Disc (CD), a memory card or a floppy disk. Such a computer program product is typically a computer readable storage medium such as described in fig. 4. Fig. 4 shows a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention. The computer readable storage medium 400 has stored thereon a computer readable program code 331 for performing the steps of the method according to the invention, readable by a processor 310 of the electronic device 300, which computer readable program code 331, when executed by the electronic device 300, causes the electronic device 300 to perform the steps of the method described above, in particular the computer readable program code 331 stored on the computer readable storage medium may perform the method shown in any of the embodiments described above. The computer readable program code 331 may be compressed in a suitable form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
The embodiment of the invention discloses A1 and a method for realizing proxy service, which comprises the following steps:
receiving a service request containing verification information;
determining target equipment used as a proxy outlet according to the verification information, and establishing proxy connection with the target equipment;
and realizing the transmission of service data based on the proxy connection.
A2, the method as in A1, wherein the service request is a crawler-related service request;
the transmission of the service data based on the proxy connection comprises the following steps:
and taking the IP address of the target device as an exit IP of the crawler, and transmitting the crawling request and the crawling content through the proxy connection.
A3, the method as in A1, wherein the authentication information includes a username and a password, and the determining the target device to act as a proxy exit from the authentication information includes:
analyzing the user name, and judging whether new target equipment needs to be allocated or not according to an analysis result;
and if new target equipment needs to be distributed, selecting matched equipment as the target equipment according to the proxy mode corresponding to the analysis result.
A4, the method as in A3, wherein the parsing the username comprises:
analyzing a session name from the user name;
the judging whether new target equipment needs to be allocated according to the analysis result comprises the following steps:
and judging whether the target equipment corresponding to the session name is recorded, if so, not allocating new target equipment, otherwise, allocating new target equipment.
A5, the method as in A4, wherein the establishing a proxy connection with the target device comprises:
and establishing a proxy session according to the session name, and recording the session name and the corresponding target equipment.
A6, the method of A3, wherein the proxy mode comprises: a random mode and a binding mode;
the selecting the matched device as the target device according to the proxy mode corresponding to the analysis result comprises:
in a random mode, randomly distributing equipment as target equipment;
in the binding mode, the designated device is taken as the target device.
A7, the method as in A6, wherein the parsing the username comprises: analyzing service requirements or equipment information of specified equipment from the user name;
the random allocation device as a target device includes: and randomly allocating the equipment meeting the service requirement as target equipment.
A8, the method as in A1, wherein the proxy service is implemented based on socks5 protocol.
The embodiment of the invention also discloses B9, a device for realizing the proxy service, which comprises:
a receiving unit adapted to receive a service request containing authentication information;
a proxy connection establishing unit adapted to determine a target device serving as a proxy exit according to the authentication information, and establish a proxy connection with the target device;
and the proxy unit is suitable for realizing the transmission of the service data based on the proxy connection.
B10, the device as B9, wherein the service request is a service request related to a crawler;
and the proxy unit is suitable for transmitting the crawling request and the crawling content through the proxy connection by taking the IP address of the target device as an exit IP of the crawler.
B11, the apparatus as in B9, wherein the authentication information includes a username and password;
the proxy connection establishing unit is suitable for analyzing the user name and judging whether new target equipment needs to be allocated or not according to an analysis result; and if new target equipment needs to be distributed, selecting matched equipment as the target equipment according to the proxy mode corresponding to the analysis result.
B12, the device of B11, wherein,
the proxy connection establishing unit is suitable for analyzing a session name from the user name; and judging whether the target equipment corresponding to the session name is recorded, if so, not allocating new target equipment, otherwise, allocating new target equipment.
B13, the device of B12, wherein,
the proxy connection establishing unit is suitable for establishing a proxy session according to the session name and recording the session name and the corresponding target equipment.
B14, the apparatus as in B11, wherein the proxy mode comprises: a random mode and a binding mode;
the proxy connection establishing unit is suitable for randomly distributing equipment as target equipment in a random mode; in the binding mode, the designated device is taken as the target device.
B15, the device of B14, wherein,
the proxy connection establishing unit is suitable for analyzing the service requirement or the equipment information of the specified equipment from the user name; and randomly allocating the equipment meeting the service requirement as target equipment.
B16, the apparatus as in B9, wherein the proxy service is implemented based on socks5 protocol.
The embodiment of the invention also discloses C17 and electronic equipment, wherein the electronic equipment comprises: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the method of any one of a 1-A8.
Embodiments of the invention also disclose D18, a computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method as any one of a1-a 8.

Claims (10)

1. A method for realizing proxy service includes:
receiving a service request containing verification information;
determining target equipment used as a proxy outlet according to the verification information, and establishing proxy connection with the target equipment;
and realizing the transmission of service data based on the proxy connection.
2. The method of claim 1, wherein the service request is a crawler-related service request;
the transmission of the service data based on the proxy connection comprises the following steps:
and taking the IP address of the target device as an exit IP of the crawler, and transmitting the crawling request and the crawling content through the proxy connection.
3. The method of claim 1, wherein the authentication information includes a username and a password, and wherein determining a target device to act as a proxy egress from the authentication information comprises:
analyzing the user name, and judging whether new target equipment needs to be allocated or not according to an analysis result;
and if new target equipment needs to be distributed, selecting matched equipment as the target equipment according to the proxy mode corresponding to the analysis result.
4. The method of claim 3, wherein the parsing the username comprises:
analyzing a session name from the user name;
the judging whether new target equipment needs to be allocated according to the analysis result comprises the following steps:
and judging whether the target equipment corresponding to the session name is recorded, if so, not allocating new target equipment, otherwise, allocating new target equipment.
5. An apparatus for implementing a proxy service, comprising:
a receiving unit adapted to receive a service request containing authentication information;
a proxy connection establishing unit adapted to determine a target device serving as a proxy exit according to the authentication information, and establish a proxy connection with the target device;
and the proxy unit is suitable for realizing the transmission of the service data based on the proxy connection.
6. The apparatus of claim 5, wherein the service request is a crawler-related service request;
and the proxy unit is suitable for transmitting the crawling request and the crawling content through the proxy connection by taking the IP address of the target device as an exit IP of the crawler.
7. The apparatus of claim 5, wherein the authentication information comprises a username and a password;
the proxy connection establishing unit is suitable for analyzing the user name and judging whether new target equipment needs to be allocated or not according to an analysis result; and if new target equipment needs to be distributed, selecting matched equipment as the target equipment according to the proxy mode corresponding to the analysis result.
8. The apparatus of claim 7, wherein,
the proxy connection establishing unit is suitable for analyzing a session name from the user name; and judging whether the target equipment corresponding to the session name is recorded, if so, not allocating new target equipment, otherwise, allocating new target equipment.
9. An electronic device, wherein the electronic device comprises: a processor; and a memory arranged to store computer-executable instructions that, when executed, cause the processor to perform the method of any one of claims 1-4.
10. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of claims 1-4.
CN201811581126.7A 2018-12-24 2018-12-24 Proxy service realization method, device, electronic equipment and storage medium Active CN111355693B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811581126.7A CN111355693B (en) 2018-12-24 2018-12-24 Proxy service realization method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811581126.7A CN111355693B (en) 2018-12-24 2018-12-24 Proxy service realization method, device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN111355693A true CN111355693A (en) 2020-06-30
CN111355693B CN111355693B (en) 2023-10-31

Family

ID=71193749

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811581126.7A Active CN111355693B (en) 2018-12-24 2018-12-24 Proxy service realization method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN111355693B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080183889A1 (en) * 2007-01-31 2008-07-31 Dmitry Andreev Method and system for preventing web crawling detection
CN102708173A (en) * 2012-05-02 2012-10-03 奇智软件（北京）有限公司 Method and system for processing user requests of accessing to web pages
CN102891858A (en) * 2012-10-22 2013-01-23 北京瓦力网络科技有限公司 Method and system for network data transmission and proxy server
US20130067085A1 (en) * 2011-03-11 2013-03-14 Qualcomm Incorporated System and method using a client-local proxy-server to access a device having an assigned network address
CN103914568A (en) * 2014-04-24 2014-07-09 厦门市美亚柏科信息股份有限公司 Method and device for dispatching HTTP proxy
CN106021375A (en) * 2016-05-11 2016-10-12 深圳市永兴元科技有限公司 Database access method and database agent node
CN106612261A (en) * 2015-10-26 2017-05-03 北京国双科技有限公司 Website data obtaining method, devices and system
CN107770138A (en) * 2016-08-22 2018-03-06 阿里巴巴集团控股有限公司 Specify the method and proxy server, client of IP address
CN108063714A (en) * 2016-11-09 2018-05-22 北京国双科技有限公司 A kind of processing method and processing device of network request
CN108551452A (en) * 2018-04-18 2018-09-18 平安科技（深圳）有限公司 Web crawlers method, terminal and storage medium

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080183889A1 (en) * 2007-01-31 2008-07-31 Dmitry Andreev Method and system for preventing web crawling detection
US20130067085A1 (en) * 2011-03-11 2013-03-14 Qualcomm Incorporated System and method using a client-local proxy-server to access a device having an assigned network address
CN102708173A (en) * 2012-05-02 2012-10-03 奇智软件（北京）有限公司 Method and system for processing user requests of accessing to web pages
CN102891858A (en) * 2012-10-22 2013-01-23 北京瓦力网络科技有限公司 Method and system for network data transmission and proxy server
CN103914568A (en) * 2014-04-24 2014-07-09 厦门市美亚柏科信息股份有限公司 Method and device for dispatching HTTP proxy
CN106612261A (en) * 2015-10-26 2017-05-03 北京国双科技有限公司 Website data obtaining method, devices and system
CN106021375A (en) * 2016-05-11 2016-10-12 深圳市永兴元科技有限公司 Database access method and database agent node
CN107770138A (en) * 2016-08-22 2018-03-06 阿里巴巴集团控股有限公司 Specify the method and proxy server, client of IP address
CN108063714A (en) * 2016-11-09 2018-05-22 北京国双科技有限公司 A kind of processing method and processing device of network request
CN108551452A (en) * 2018-04-18 2018-09-18 平安科技（深圳）有限公司 Web crawlers method, terminal and storage medium

Also Published As

Publication number Publication date
CN111355693B (en) 2023-10-31

Similar Documents

Publication Publication Date Title
CN111934918A (en) Network isolation method and device for container instances in same container cluster
CN102137059A (en) Method and system for blocking malicious accesses
CN108833450B (en) Method and device for preventing server from being attacked
CN110247857B (en) Current limiting method and device
CN106685949A (en) Container access method, container access device and container access system
CN104158818A (en) Single sign-on method and system
CN103973682A (en) Method and device for having access to webpage
CN108347731A (en) A kind of method, medium, equipment and terminal carrying out secure binding
CN104219230A (en) Method and device for identifying malicious websites
CN111064804A (en) Network access method and device
CN108712428A (en) A kind of method and device carrying out device type identification to terminal
CN105991624A (en) Safety management method and device of server
CN110619022B (en) Node detection method, device, equipment and storage medium based on block chain network
CN104506667B (en) A kind of distribution method and device of port resource, user authentication management device
CN106453663B (en) Improved storage expansion method and device based on cloud service
CN115333863B (en) Internet of things system building method based on dynamic domain name service and related equipment
CN108512889A (en) A kind of application response method for pushing and proxy server based on HTTP
CN111355693B (en) Proxy service realization method, device, electronic equipment and storage medium
CN113472545B (en) Equipment network access method, device, equipment, storage medium and communication system
CN111970250B (en) Method for identifying account sharing, electronic device and storage medium
CN101707535A (en) Method and device for detecting counterfeit network equipment
CN113973093B (en) Data transmission method and device, electronic equipment and readable storage medium
CN112468356B (en) Router interface testing method, device, electronic equipment and storage medium
CN108650274A (en) A kind of network inbreak detection method and system
CN114244555A (en) Method for adjusting security policy

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant