CN111818038A

CN111818038A - Network data acquisition and identification method and device

Info

Publication number: CN111818038A
Application number: CN202010627703.2A
Authority: CN
Inventors: 范晓锋; 方庆远
Original assignee: Lazas Network Technology Shanghai Co Ltd
Current assignee: Lazas Network Technology Shanghai Co Ltd
Priority date: 2020-07-01
Filing date: 2020-07-01
Publication date: 2020-10-23
Anticipated expiration: 2040-07-01
Also published as: CN111818038B

Abstract

The method comprises the steps of decrypting a target encryption parameter carried by a second network request to obtain a first network behavior background mark, obtaining a second network behavior background mark corresponding to the second network request, comparing the first network behavior background mark with the second network behavior background mark, and determining whether an access main body corresponding to the second network request is a network data acquisition main body or not based on a comparison result, wherein the second network request is realized based on network response data corresponding to the first network request, and the target encryption parameter is a parameter obtained after encryption processing is carried out on the first network behavior background mark corresponding to the first network request. By using the method, the network data acquisition task scheduling system and the data acquisition operation under the acquisition strategy distribution mechanism can be accurately identified.

Description

Network data acquisition and identification method and device

Technical Field

The application relates to the technical field of computers, in particular to a network data acquisition and identification method. The application also relates to a network data acquisition and identification device, an electronic device and a computer readable storage medium.

Background

The network data acquisition refers to automatically acquiring world wide web information through a program or a script according to a preset data acquisition rule, and the process may cause network resources, computing resources and the like of a website to be improperly occupied, so that the function of the website is damaged. For example, in an online shopping scenario or a life service type network application scenario, the network data acquisition process may cause that merchant information, commodity information, and the like of a network platform are maliciously acquired in batches; for a content distribution website, it may cause the intellectual property rights of the website to be maliciously infringed.

The network data acquisition main body can usually acquire target data content through an HTTP request, corresponding parameters in a URL address (Uniform Resource Locator) of the HTTP request correspond to the target data content, and the network data acquisition main body continuously extracts a new URL from a URL corresponding to one or a plurality of initial webpages and puts the new URL into an acquisition queue according to a webpage acquisition policy after acquiring the initial webpage URL, and acquires the target data content based on the URL until a certain stop condition is satisfied.

The existing identification and interception technology aiming at the network data acquisition behavior comprises modes of IP dimension countermeasure, account dimension countermeasure, verification codes, a dynamic page structure, access limiting amount and downloading amount, text-to-picture conversion, dynamic parameters and the like, wherein the identification and interception technology aiming at the network data acquisition behavior based on the dynamic parameters increases the cost of the network data acquisition behavior to a certain extent and increases the opportunity of identifying and intercepting the network data acquisition behavior. For example, the takeout website performs dynamic processing on the URL corresponding to the merchant, which is specifically to differentiate the merchant ID in the URL corresponding to the merchant into an internal merchant ID and an external dynamic ID, where the internal ID is a transfer parameter used for specifying a merchant object between systems in the website, which is generally a fixed character string, the external dynamic ID is generally a dynamic character string with variable timing or non-timing obtained by mapping numbers, so that the network data acquisition subject cannot request the data content of the merchant from the website through the same static URL, instead, at different points in time, the exact value of the dynamic URL corresponding to the merchant needs to be found, for example, by searching the website for the name of the merchant, the method increases the interaction times of the network data acquisition main body and the website for acquiring one target data content, thereby increasing the acquisition cost and increasing the opportunity of identifying and intercepting network data acquisition behaviors by the website.

In order to overcome the above-mentioned identification and interception technology for network data acquisition behavior based on dynamic parameters and implement efficient traversal of the whole content of a website, a task scheduling system and an acquisition policy allocation mechanism are added in a network data acquisition system, that is, a preamble page of the target data content of a target website is periodically acquired by a preamble network data acquisition instance, and a URL acquired by the preamble network data acquisition instance is allocated to other network data acquisition instances to continue to execute subsequent acquisition operations, for example, a merchant list page is acquired by the preamble network data acquisition instance before a merchant detail page is acquired, and a dynamic URL corresponding to the merchant detail page is allocated to the subsequent network data acquisition instance, the merchant detail page is acquired by the subsequent network data acquisition instance, because different network data acquisition instances generally use different IP addresses and different accounts, in view of a target website, each IP and each account have no obvious intensive operation, so that a network data acquisition main body bypasses the limitation of the access amount of a single IP or a single account in the identification and interception technology for network data acquisition behaviors, and the aim of acquiring a large amount of network data is fulfilled.

Based on this, how to accurately identify the network data acquisition task scheduling system and the data acquisition operation under the acquisition strategy allocation mechanism thereof is a problem which needs to be solved urgently at present.

Disclosure of Invention

The embodiment of the application provides a network data acquisition identification method and device, electronic equipment and a computer readable storage medium, which aim to accurately identify a network data acquisition task scheduling system and data acquisition operation under an acquisition strategy allocation mechanism of the network data acquisition task scheduling system.

The embodiment of the application provides a network data acquisition and identification method, which comprises the following steps: acquiring a target encryption parameter carried by a second network request, wherein the second network request is realized based on network response data corresponding to the first network request, and the target encryption parameter is obtained after encryption processing is performed on a first network behavior background identifier corresponding to the first network request; decrypting the target encryption parameter to obtain a first network behavior background identifier; acquiring a second network behavior background identifier corresponding to the second network request; and comparing the first network behavior background identification with the second network behavior background identification, and determining whether the access subject corresponding to the second network request is a network data acquisition subject or not based on the comparison result.

Optionally, the first network behavior context identifier includes: a first network requests corresponding user context information; the second network behavior context identification comprises: the second network requests the corresponding user context information; comparing the first network behavior background identifier with the second network behavior background identifier, and determining whether the access subject corresponding to the second network request is a network data acquisition subject based on the comparison result, including: comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request; and if the user context information corresponding to the first network request is not matched with the user context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject.

Optionally, the user context information includes at least one of the following: user identification information; network operating environment information; comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request, comprising: comparing the user identification information corresponding to the first network request with the user identification information corresponding to the second network request; and/or comparing the network operation environment information corresponding to the first network request with the network operation environment information corresponding to the second network request. Optionally, the user identification information includes: user identification information, or login session identification information for anonymous users. The network operating environment information includes at least one of: network device identification information; network device IP address.

Optionally, the first network behavior context identifier includes: the first network requests corresponding application context information; the second network behavior context identification comprises: the second network requests corresponding application context information; comparing the first network behavior background identifier with the second network behavior background identifier, and determining whether the access subject corresponding to the second network request is a network data acquisition subject based on the comparison result, including: comparing the application context information corresponding to the first network request with the application context information corresponding to the second network request; and if the application context information corresponding to the first network request is not matched with the application context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject.

Optionally, the first network behavior context identifier includes: the user context information corresponding to the first network request and the application context information corresponding to the first network request; the second network behavior context identification comprises: the user context information corresponding to the second network request and the application context information corresponding to the second network request; comparing the first network behavior background identifier with the second network behavior background identifier, and determining whether the access subject corresponding to the second network request is a network data acquisition subject based on the comparison result, including: comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request, and comparing the application context information corresponding to the first network request with the application context information corresponding to the second network request; and if the user context information corresponding to the first network request is not matched with the user context information corresponding to the second network request, and/or the application context information corresponding to the first network request is not matched with the application context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject.

Optionally, the application context information includes at least one of the following: applying channel information; application interface information. Optionally, the method further includes: a second network request is intercepted. Optionally, the obtaining of the target encryption parameter carried by the second network request includes: acquiring a target encryption parameter carried by a second network request aiming at target network data; the second network request is realized based on the network response data corresponding to the first network request, and comprises the following steps: the second network request is implemented based on the access policy data for the target network data contained in the network response data corresponding to the first network request.

Optionally, the method further includes: obtaining network response data in response to the first network request, the network response data including initial access policy data for the target network data; acquiring a first network behavior background identifier corresponding to a first network request; encrypting a first network behavior background identifier corresponding to the first network request to obtain a target encryption parameter; obtaining target access policy data aiming at the target network data according to the initial access policy data and the target encryption parameter, and providing the target access policy data to an access subject corresponding to the first network request; or, the initial access policy data and the target encryption parameter are provided to the access subject corresponding to the first network request, so that the access subject obtains the target access policy data for the target network data based on the initial access policy data and the target encryption parameter.

Optionally, the access policy data for the target network data includes: and URL corresponding to the target network data. Optionally, the target encryption parameter is included in a parameter of the URL. Optionally, the encrypting the first network behavior context identifier corresponding to the first network request includes: and encrypting the identification information of the target network data and the first network behavior background identification. The initial access strategy data aiming at the target network data is an initial URL corresponding to the target network data, and the target access strategy data aiming at the target network data is a target URL obtained after the identification information of the target network data contained in the initial URL is replaced by a target encryption parameter; providing the target access policy data to an access subject corresponding to the first network request, including: and providing the target URL to an access subject corresponding to the first network request.

Optionally, providing the initial access policy data and the target encryption parameter to the access subject corresponding to the first network request includes: and sending the data structure file containing the initial access policy data and the target encryption parameter to an access subject corresponding to the first network request. The target encryption parameter is included in a request header or a request body of the second network request.

Optionally, the first network request includes: an access main body corresponding to the first network request shares target network data with a sharing request of a target sharing main body; the first network behavior context identifier corresponding to the first network request comprises: a network behavior background identifier corresponding to the target sharing subject; the method for encrypting the first network behavior background identification corresponding to the first network request comprises the following steps: and encrypting the user identification information of the access subject corresponding to the first network request and the network behavior background identification corresponding to the target sharing subject.

Optionally, the encrypting the first network behavior context identifier corresponding to the first network request includes: encrypting the first network behavior background identification by using a target encryption key in the multiple sets of keys, wherein the second network request comprises a target encryption key corresponding to the target key identification; correspondingly, decrypting the target encryption parameter comprises the following steps: and decrypting the target encryption parameter by using the target decryption key corresponding to the target key identifier. Optionally, the encrypting the first network behavior context identifier corresponding to the first network request includes: and encrypting the first network behavior background identification corresponding to the first network request based on a symmetric key cryptosystem. The first network behavior context identification comprises: time stamp information. Optionally, the first network request is a network access request for a merchant list page, and the second network request is a network access request for a merchant detail page.

An embodiment of the present application further provides a network data acquiring and identifying device, including: the target encryption parameter acquiring unit is used for acquiring a target encryption parameter carried by a second network request, wherein the second network request is realized based on network response data corresponding to the first network request, and the target encryption parameter is obtained after encryption processing is performed on a first network behavior background identifier corresponding to the first network request; the target encryption parameter decryption unit is used for decrypting the target encryption parameter to obtain a first network behavior background identifier; a second network behavior background identifier obtaining unit, configured to obtain a second network behavior background identifier corresponding to the second network request; and the network behavior background identifier comparison unit is used for comparing the first network behavior background identifier with the second network behavior background identifier and determining whether the access subject corresponding to the second network request is a network data acquisition subject or not based on the comparison result.

Optionally, the user context information includes at least one of the following: user identification information; network operating environment information; comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request, comprising: comparing the user identification information corresponding to the first network request with the user identification information corresponding to the second network request; and/or comparing the network operation environment information corresponding to the first network request with the network operation environment information corresponding to the second network request. Optionally, the user identification information includes: user identification information, or login session identification information for anonymous users. Optionally, the network operating environment information includes at least one of the following: network device identification information; network device IP address.

Optionally, the first network behavior context identifier includes: the user context information corresponding to the first network request and the application context information corresponding to the first network request; the second network behavior context identification comprises: the user context information corresponding to the second network request and the application context information corresponding to the second network request; comparing the first network behavior background identifier with the second network behavior background identifier, and determining whether the access subject corresponding to the second network request is a network data acquisition subject based on the comparison result, including: comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request, and comparing the application context information corresponding to the first network request with the application context information corresponding to the second network request; and if the user context information corresponding to the first network request is not matched with the user context information corresponding to the second network request, and/or the application context information corresponding to the first network request is not matched with the application context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject. Optionally, the application context information includes at least one of the following: applying channel information; application interface information. Optionally, the method further includes: and the second network request intercepting unit is used for intercepting the second network request.

Optionally, the obtaining of the target encryption parameter carried by the second network request includes: acquiring a target encryption parameter carried by a second network request aiming at target network data; the second network request is realized based on the network response data corresponding to the first network request, and comprises the following steps: the second network request is implemented based on the access policy data for the target network data contained in the network response data corresponding to the first network request.

Optionally, the method further includes: a network response data acquisition unit configured to acquire network response data in response to the first network request, the network response data including initial access policy data for the target network data; a first network behavior background identifier obtaining unit, configured to obtain a first network behavior background identifier corresponding to the first network request; the encryption processing unit is used for encrypting the first network behavior background identifier corresponding to the first network request to obtain a target encryption parameter; the data providing unit is used for obtaining target access policy data aiming at the target network data according to the initial access policy data and the target encryption parameter and providing the target access policy data to an access subject corresponding to the first network request; or, the initial access policy data and the target encryption parameter are provided to the access subject corresponding to the first network request, so that the access subject obtains the target access policy data for the target network data based on the initial access policy data and the target encryption parameter.

Optionally, the access policy data for the target network data includes: and URL corresponding to the target network data. Optionally, the target encryption parameter is included in a parameter of the URL.

Optionally, the encrypting the first network behavior context identifier corresponding to the first network request includes: and encrypting the identification information of the target network data and the first network behavior background identification.

Optionally, the initial access policy data for the target network data is an initial URL corresponding to the target network data, and the target access policy data for the target network data is a target URL obtained after replacing the identification information of the target network data included in the initial URL with the target encryption parameter; the providing the target access policy data to the access subject corresponding to the first network request includes: and providing the target URL to an access subject corresponding to the first network request. Optionally, the providing the initial access policy data and the target encryption parameter to the access subject corresponding to the first network request includes: and sending a data structure file containing the initial access policy data and the target encryption parameter to an access subject corresponding to the first network request.

Optionally, the target encryption parameter is included in a request header or a request body of the second network request.

Optionally, the first network request includes: an access main body corresponding to a first network request shares the target network data with a sharing request of a target sharing main body; the first network behavior context identifier corresponding to the first network request comprises: and the target shares the network behavior background identification corresponding to the main body. The encrypting the first network behavior background identifier corresponding to the first network request includes: and encrypting the user identification information of the access subject corresponding to the first network request and the network behavior background identification corresponding to the target sharing subject.

Optionally, the encrypting the first network behavior context identifier corresponding to the first network request includes: encrypting the first network behavior background identifier by using a target encryption key in a plurality of sets of keys, wherein the second network request comprises a target key identifier corresponding to the target encryption key; correspondingly, the decrypting the target encryption parameter includes: and decrypting the target encryption parameter by using a target decryption key corresponding to the target key identifier.

Optionally, the encrypting the first network behavior context identifier corresponding to the first network request includes: and encrypting the first network behavior background identification corresponding to the first network request based on a symmetric key cryptosystem. The first network behavior context identification comprises: time stamp information.

Optionally, the first network request is a network access request for a merchant list page, and the second network request is a network access request for a merchant detail page.

The embodiment of the application also provides an electronic device, which comprises a processor and a memory; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the above-described method.

Embodiments of the present application also provide a computer-readable storage medium having one or more computer instructions stored thereon, which are executed by a processor to implement the above-mentioned method.

Compared with the prior art, the embodiment of the application has the following advantages:

in the network data acquisition and identification method provided by the embodiment of the application, after a target encryption parameter carried by a second network request is acquired, the target encryption parameter is decrypted to obtain a first network behavior background identifier, a second network behavior background identifier corresponding to the second network request is acquired, the first network behavior background identifier is compared with the second network behavior background identifier, whether an access subject corresponding to the second network request is a network data acquisition subject is determined based on a comparison result, the second network request is realized based on network response data corresponding to the first network request, and the target encryption parameter is a parameter obtained after encryption processing is performed on the first network behavior background identifier corresponding to the first network request. Since the second network request is implemented based on the network response data corresponding to the first network request, it may be determined whether the second network request is from the network data obtaining subject based on whether the first network request and the second network request correspond to the same network behavior context identifier, for example, if the second network request is a network request from a normal user, the first network request and the second network request should correspond to the same network behavior context identifier, and if the first network request and the second network request correspond to different network behavior context identifiers, it indicates that the network data obtaining operation implemented based on the network data obtaining task scheduling system is between the first network request and the second network request, that is, the access subject corresponding to the second network request is the network data obtaining subject. By using the method, the network data acquisition task scheduling system and the data acquisition operation under the acquisition strategy distribution mechanism can be accurately identified.

Drawings

Fig. 1 is a flowchart of a network data acquisition and identification method according to a first embodiment of the present application;

FIG. 2 is a schematic view of a scenario provided by a first embodiment of the present application;

fig. 3 is a block diagram of elements of a network data acquisition and identification device according to a second embodiment of the present application;

fig. 4 is a schematic logical structure diagram of an electronic device according to a third embodiment of the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.

Aiming at a network data acquisition identification scene, in order to accurately identify network data acquisition behaviors, the application provides a network data acquisition identification method, a network data acquisition identification device corresponding to the method, electronic equipment and a computer readable storage medium. The following provides embodiments for detailed description of the method, apparatus, electronic device, and computer-readable storage medium.

A first embodiment of the present application provides a network data acquisition and identification method, an application body of the method may be a computing device application for identifying a network data acquisition behavior, the computing device application may be run in a network server (for example, a network platform server in an online shopping scenario or a life service type network application scenario) or run in a network middleware (for example, a terminal device dedicated to identifying a network data acquisition behavior) independently deployed between the network server and a client, fig. 1 is a flowchart of a network data acquisition and identification method provided in the first embodiment of the present application, fig. 2 is a schematic view of a scenario provided in the first embodiment of the present application, and the network data acquisition and identification method provided in this embodiment is described in detail below with reference to fig. 1 and fig. 2. The following description refers to embodiments for the purpose of illustrating the principles of the methods, and is not intended to be limiting in actual use.

As shown in fig. 1, the network data acquiring and identifying method provided in this embodiment includes the following steps:

s101, acquiring a target encryption parameter carried by the second network request.

The second network request is implemented based on the network response data corresponding to the first network request, that is, for the network access user, the first network request and the second network request have a network access hierarchy relationship, the second network request is implemented based on the network response data corresponding to the first network request, the first network request is a preamble request of the second network request, and the page accessed by the first network request and the page accessed by the second network request are continuous pages. As shown in fig. 2, in an online shopping scenario or a life service type network application scenario, the first network request may be a network access request of a user for a merchant list page, and the second network request may be a network access request of a user for a merchant detail page of a merchant in the merchant list page.

The target encryption parameter is a parameter obtained after encrypting a first network behavior background identifier corresponding to the first network request, that is, the target encryption parameter can be obtained by using the first network behavior background identifier as encrypted data and encrypting the encrypted data by using a predetermined encryption function, so that the encryption processing of the first network behavior is performed to avoid that a network data acquisition main body autonomously generates an access parameter, thereby avoiding the network data acquisition and identification method of the embodiment.

The network behavior background identifier is used for identifying background information corresponding to the network access behavior, and due to differences of factors such as a network access subject, a network environment, an application link and the like, each network access behavior corresponds to the network behavior background identifier, which may be user context information corresponding to the network access behavior, may also be application context information corresponding to the network access behavior, and may also include user context information and application context information corresponding to the same network access behavior.

The user context information is used for identifying a network access subject, and for access requests from different users, the user context information corresponds to different user context information, in this embodiment, the user context information may be at least one of user identification information and network operation environment information, the user identification information may be user identification information (e.g., a user ID) of a registered user, or may be login session identification information allocated for an anonymous user (e.g., in order to identify a user identity, a website allocates a login session ID to an anonymous user lacking a user ID and lacking an associated login session in advance, and stores the login session ID in a Cookie, which indicates state information transferred between a network server and a client and may be used for tracking the user); the network operating environment information is used to identify a network operating environment corresponding to the network access behavior, and may be at least one of network device identification information (e.g., a network device ID) and a network device IP address.

The application context information is used to identify an application link corresponding to the network access behavior, and may be at least one of application channel information corresponding to the network request (e.g., network requests from different network platforms correspond to different application channel information) and application interface information (e.g., network requests from different application interfaces of the same network platform correspond to different application interface information).

In this embodiment, the obtaining of the target encryption parameter carried by the second network request may refer to: acquiring a target encryption parameter carried by a second network request aiming at target network data; correspondingly, the second network request is implemented based on the network response data corresponding to the first network request, and may refer to: the second network request is implemented based on the access policy data for the target network data included in the network response data corresponding to the first network request, in this embodiment, the target encryption parameter may be included in the access policy data, the access policy data may be a URL (Uniform Resource Locator) corresponding to the target network data, for example, a merchant list page (first network request) of the network shopping platform accessed by the user, in the merchant list content to be returned by the network platform server, the merchant detail page of each merchant corresponds to an initial URL, after adding the target encryption parameter to each initial URL or replacing part of the parameters in the initial URL with the target encryption parameter, the target URL corresponding to the merchant detail page is obtained, and the target URL is returned to the user as the network response data of the first network request, when the user accesses the merchant detail page (second network request) of the target merchant based on the target URL (clicking the link of the target URL), the merchant detail page of the target merchant is the target network data, and the target URL includes the target encryption parameter.

It should be noted that the network response data corresponding to the first network request may also be a data structure file with a predetermined format, and correspondingly, the access policy data may also be included in a data structure file with a predetermined format returned by the network server, for example, the access policy data may be included in a JSON file returned by the network server, and the client may assemble a URL corresponding to the target network data based on the JSON file, add a target encryption parameter to the URL, or replace a part of the parameter in the URL with a target encryption parameter, and then obtain a target URL corresponding to the target network data, and access the target network data based on the target URL (the second network request).

It should be noted that, the target encryption parameter may be included in a request header (header) or a request body (body) of the second network request, besides the parameter of the URL, and only the second network request needs to carry the target encryption parameter, which is not limited herein.

Corresponding to the above, before executing step S101, the application main body of the present embodiment further needs to perform the following operations:

firstly, network response data responding to a first network request is obtained, the network response data comprises initial access policy data aiming at target network data, the initial access policy data can be an initial URL corresponding to the target network data, for example, a merchant list page (first network request) of a network shopping platform accessed by a user, and in merchant list contents to be returned by a network platform server, the merchant list page of each merchant corresponds to the initial URL;

secondly, acquiring a first network behavior background identifier corresponding to the first network request, for example, acquiring a user ID, a login session ID, a network device IP address, application channel information, and the like corresponding to the first network request;

then, encrypting a first network behavior background identifier corresponding to the first network request to obtain a target encryption parameter;

finally, according to the initial access policy data and the target encryption parameter, target access policy data for the target network data is obtained, and the target access policy data is provided to the access subject corresponding to the first network request, for example, the target access policy data may be a target URL obtained after the implementation subject replaces identification information of the target network data included in the initial URL with the target encryption parameter, or a target URL obtained after the implementation subject adds the target encryption parameter on the basis of the original parameter in the initial URL, and correspondingly, the providing of the target access policy data to the access subject corresponding to the first network request may refer to the implementation subject providing the target URL to the access subject corresponding to the first network request. The identification information of the target network data is used to identify the target network data in the content addressing process of the network platform, and may be one of parameters in the initial URL, for example, a delivery parameter (e.g., a merchant internal ID) preset by the network platform for the resident merchant, for specifying the merchant between systems inside the network platform.

In this embodiment, in addition to obtaining target access policy data for target network data based on the initial access policy data and the target encryption parameter as described above, and provides the target access policy data to the access subject corresponding to the first network request, and also provides the initial access policy data and the target encryption parameter to the access subject corresponding to the first network request, for the accessing agent to obtain target access policy data for the target network data based on the initial access policy data and the target encryption parameters, for example, the target access policy data may be a target URL obtained after the client (the access subject corresponding to the first network request) replaces the identification information of the target network data contained in the initial URL with the target encryption parameter, or a target URL obtained after adding target encryption parameters on the basis of the original parameters in the initial URL for the client (the access subject corresponding to the first network request). The providing the initial access policy data and the target encryption parameter to the access subject corresponding to the first network request may also refer to: and sending a data structure file containing initial access strategy data and target encryption parameters to an access main body corresponding to the first network request, so that the access main body assembles the URL corresponding to the target network data based on the data structure file, adds the target encryption parameters to the URL, obtains the target URL, and accesses the target network data based on the target URL.

In this embodiment, in order to increase the applicability (e.g., make the encryption process adapt to different application lines of the network platform) and the security (e.g., avoid the influence on the encryption and decryption processes due to factors such as application system upgrade and data update of the network platform), the encryption process may be preferentially implemented based on multiple sets of keys preset by the network platform, for example, a target encryption key in the multiple sets of keys is used to encrypt a first network behavior background identifier, each set of key corresponds to a key identifier (e.g., a key ID corresponding to each set of key or a hash value obtained by performing hash calculation on the key), in this case, a second network request includes the target key identifier corresponding to the target encryption key, e.g., the target key identifier is used as a new parameter in an initial URL or returned to the client in another way, the destination key identification may also be part of the encrypted data. Subsequent decryption processes may be implemented based on the target key identification.

It should be noted that the encryption process may be implemented based on a symmetric key cryptosystem (the encryption key and the decryption key use the same cryptosystem) or a public key cryptosystem (different encryption key and decryption key are used), in this embodiment, in order to improve convenience of the encryption and decryption process, the encryption process is preferentially performed on the first network behavior background identifier corresponding to the first network request based on the symmetric key cryptosystem, for example, for the same network platform, the convenience of the encryption and decryption process may be improved by performing encryption and decryption using the symmetric key cryptosystem.

In this embodiment, the first network behavior context identifier may further include timestamp information corresponding to the first network request, and the timestamp information is used as a part of the encrypted data, so that the validity period of the encryption result may be limited in a time dimension, and replay attack may be prevented.

S102, the target encryption parameter is decrypted to obtain a first network behavior background identifier.

After the target encryption parameter carried by the second network request is obtained in the above step, the step is used for performing decryption processing on the target encryption parameter, so as to obtain a first network behavior background identifier corresponding to the first network request.

Corresponding to the above digital encryption based on multiple sets of keys, in this embodiment, decrypting the target encryption parameter may specifically refer to: and decrypting the target encryption parameter by using the target decryption key corresponding to the target key identifier.

And S103, acquiring a second network behavior background identifier corresponding to the second network request.

After the target encryption parameter is decrypted and the first network behavior background identifier is obtained in the above steps, this step is used to obtain the second network behavior background identifier corresponding to the second network request, for example, obtain one or more of information such as a user ID, a login session ID, a network device IP address, and application channel information corresponding to the second network request.

It should be noted that, in this embodiment, the implementation sequence of the step S102 and the step S103 is not limited, that is, after the second network behavior context identifier corresponding to the second network request is obtained, a decryption operation may be performed on the target encryption parameter carried by the second network request, and the first network behavior context identifier is obtained. If step S103 is executed after step S102 is executed, the second network behavior context identifier corresponding to the second network request may be obtained in a manner suitable for the decryption result, that is, based on the first network behavior context identifier obtained by decryption in step S102, the second network behavior context identifier of the same category as the first network behavior context identifier may be selectively obtained.

And S104, comparing the first network behavior background mark with the second network behavior background mark, and determining whether an access subject corresponding to the second network request is a network data acquisition subject or not based on the comparison result.

On the basis of obtaining the first network behavior background identifier and the second network behavior background identifier in the above steps, the step is used for comparing the first network behavior background identifier with the second network behavior background identifier, and determining whether an access subject corresponding to the second network request is a network data acquisition subject based on a comparison result.

The purpose of comparing the first network behavior background identifier with the second network behavior background identifier is to determine whether the two network requests correspond to the same network access subject, or correspond to the same network operating environment, or correspond to the same application link. For a normal network request, the probability of cross-user access, or cross-network operating environment access, or cross-application link access between a first network request and a second network request having a network access hierarchical relationship is low, and therefore, whether an access subject corresponding to the second network request is a network data acquisition subject can be determined based on a preconfigured network data acquisition identification dimension and the result of the comparison.

In this embodiment, the above process can be implemented as follows:

the first method is as follows: the first network behavior background mark is user context information corresponding to the first network request; the second network behavior background mark is user context information corresponding to the second network request; the comparing the first network behavior background identifier with the second network behavior background identifier, and determining whether the access subject corresponding to the second network request is the network data acquisition subject based on the comparison result specifically means: comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request to determine whether the user contexts corresponding to the two network requests are matched, for example, comparing the user identification information (the user identification information of the registered user or the login session identification information distributed to the anonymous user) corresponding to the first network request with the user identification information corresponding to the second network request based on the user context comparison dimension pre-configured for the identification and interception requirements of the network platform for the network data acquisition behavior, or comparing the network operation environment information (at least one of the network device ID and the network device IP address) corresponding to the first network request with the network operation environment information corresponding to the second network request, or comparing the user identification information and the network operation environment information corresponding to the first network request with the user identification information corresponding to the second network request Comparing the network operation environment information, and further determining whether the two network requests correspond to the same access subject or the same network operation environment; and if the user context information corresponding to the first network request is not matched with the user context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject. By the method, the characteristic that the network data acquisition task system performs page access in a mode of switching user contexts among a plurality of webpages of the network platform can be resisted, for example, the characteristic that the network data acquisition task system frequently changes users to acquire webpage data can be resisted by comparing user IDs of two network requests; by comparing the login session identification information distributed by the anonymous user, the characteristic that the network data acquisition system frequently changes the anonymous user to acquire the webpage can be resisted; by comparing the network operation environment information such as the network equipment ID, the network equipment IP address and the like, the characteristic that the network operation environment is frequently changed by the network data acquisition task system can be resisted.

The second method comprises the following steps: the first network behavior background identifier is application context information corresponding to the first network request, the second network behavior background identifier is application context information corresponding to the second network request, the first network behavior background identifier is compared with the second network behavior background identifier, and whether an access subject corresponding to the second network request is a network data acquisition subject is determined based on a comparison result, specifically, the method includes: comparing the application context information corresponding to the first network request with the application context information corresponding to the second network request; and if the application context information corresponding to the first network request is not matched with the application context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject. By the method, the characteristic that the network data acquisition task system frequently changes an application link (an application channel or an application interface) to acquire the webpage can be resisted.

The third method comprises the following steps: the process of comparing the first network behavior background identifier with the second network behavior background identifier and determining whether the access subject corresponding to the second network request is the network data acquisition subject based on the comparison result may also be implemented based on a comparison dimension pre-configured for the identification and interception requirements of the network data acquisition behavior of the network platform, where the process specifically may refer to: comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request, and comparing the application context information corresponding to the first network request with the application context information corresponding to the second network request; if the user context information corresponding to the first network request is not matched with the user context information corresponding to the second network request, or the application context information corresponding to the first network request is not matched with the application context information corresponding to the second network request, and the user context information corresponding to the first network request is not matched with the user context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject.

It should be noted that, because the data sharing operation of the normal user also has a situation that the first network request and the second network request correspond to different network access subjects, different network operation environments, and different application links, for example, the user a shares the link information corresponding to the merchant detail page to the user B, and the user B accesses the merchant detail page based on the link information, in order to avoid the data sharing operation of the normal user and the data acquisition operation of the shared user being mistakenly identified as the data acquisition operation of the network data acquisition subject, the first network request may also refer to a sharing request that the access subject corresponding to the first network request shares the target network data to the target sharing subject, and the target sharing subject is the shared user, in this case, the first network behavior background identifier corresponding to the first network request is the network behavior background identifier corresponding to the target sharing subject, for example, the access subject corresponding to the first network request is user a, the shared user is user B, and the first network behavior context identifier is information such as a user ID corresponding to user B and a device ID of a terminal device used by user B. In this scenario, the encrypting the first network behavior context identifier corresponding to the first network request means: the network behavior background identifier corresponding to the target sharing subject is encrypted, for example, information such as a user ID corresponding to the user B and a device ID of a terminal device used by the user B is encrypted. When the first network behavior background identifier is compared with the second network behavior background identifier, if the user is a normal user, the network behavior background identifier corresponding to the target sharing subject is consistent with the network behavior background identifier corresponding to the second network request. In this embodiment, the encryption process may further be: the user identification information of the access subject corresponding to the first network request and the network behavior background identification corresponding to the target sharing subject are encrypted (for example, the user ID of the user a, the user ID of the user B, and the device ID of the terminal device used by the user B are encrypted), and the user identification information of the access subject corresponding to the first network request is used as a part of encrypted data, so that the source tracing of the sharing behavior can be realized.

In this embodiment, the encrypting the first network behavior context identifier corresponding to the first network request in step S101 may further refer to: the identification information (such as a merchant internal ID) of the target network data and the first network behavior background identifier are encrypted, so that the identification information of the target network data is used as a part of the encrypted data, and the verification link is set for the response process of the target network data.

In this embodiment, after determining that the access subject corresponding to the second network request is the network data acquisition subject through the above steps, intercepting or performing early warning processing on the second network request is further required.

In the method for acquiring and identifying network data provided in this embodiment, after a target encryption parameter carried by a second network request is acquired, the target encryption parameter is decrypted to obtain a first network behavior background identifier (one or more of user context information such as a user ID, a login session ID, a network device IP address, and application context information such as application channel information and application interface information corresponding to the first network request), and obtain a second network behavior background identifier corresponding to the second network request (one or more of user context information such as a user ID, a login session ID, a network device IP address, and application context information such as application channel information and application interface information corresponding to the second network request), the first network behavior background identifier is compared with the second network behavior background identifier, and determining whether an access subject corresponding to a second network request is a network data acquisition subject or not based on the comparison result, wherein the second network request is realized based on network response data corresponding to the first network request, and the target encryption parameter is a parameter obtained after encryption processing is performed on a first network behavior background identifier corresponding to the first network request. Because the second network request is implemented based on the network response data corresponding to the first network request, it may be determined whether the second network request is from a network data acquisition subject based on whether the first network request and the second network request correspond to the same network behavior context identifier, for example, if the second network request is a network request from a normal user, the first network request and the second network request correspond to the same network behavior context identifier, that is, the first network request and the second network request correspond to the same user context information such as a user ID, a login session ID, a network device IP address, and application context information such as application channel information, application interface information, and the like; if the first network request and the second network request correspond to different network behavior background identifications, for example, the first network request and the second network request correspond to different user IDs or login session IDs, the first network request and the second network request are cross-user network requests; the first network request and the second network request correspond to different network equipment IDs or network equipment IP addresses, and the first network request and the second network request are network requests of a cross-network operation environment; the first network request and the second network request correspond to different application channel information or application interface information, and the first network request and the second network request are network requests crossing application links; however, for the first network request and the second network request having a network access hierarchical relationship, the possibility of the cross-user, cross-network operating environment and cross-application link is low, and for the network data acquisition task scheduling system and the data acquisition operation under the acquisition policy allocation mechanism thereof, in order to bypass the limitation of the access amount of a single IP or a single account in the identification and interception technology for the network data acquisition behavior and achieve the purpose of acquiring a large amount of network data, when performing the acquisition policy allocation for the first network request and the second network request having a network access hierarchical relationship, it is necessary to implement the network data allocation across user, cross-network operating environment and cross-application link, so if the first network request and the second network request correspond to different network behavior context identifiers, it indicates that the network data acquisition task scheduling system implements the network data between the first network request and the second network request based on the network data acquisition task scheduling system And acquiring operation, namely, the access subject corresponding to the second network request is a network data acquisition subject. By using the method, the network data acquisition task scheduling system and the data acquisition operation under the acquisition strategy distribution mechanism can be accurately identified.

In order to avoid the network data acquisition and identification method provided by this embodiment, the network data acquisition system has to use the same account to execute the first network request and the second network request having the network access hierarchical relationship under the same network operating environment and the same application link, so as to acquire consecutive pages, in this case, there are two possible results as follows: the method comprises the steps that firstly, the request times of a single account are increased, so that each IP and each account have obvious intensive operation, and the network data acquisition behavior is easy to be identified and intercepted by the existing identification and interception technology aiming at the network data acquisition behavior; II, secondly: the total request number of the network data acquisition behaviors is increased, and the data acquisition cost is increased.

For example, when the network data acquisition system wants to acquire a merchant detail page corresponding to a first page and a second page of a merchant list page in an online shopping platform, each merchant list page includes links of 20 merchant detail pages, under the existing network data acquisition task scheduling system and an acquisition policy allocation mechanism thereof, the acquisition of the merchant detail page can be completed only by allocating 42(2+20 × 2, wherein 2 network data acquisition examples acquire the merchant list page) network data acquisition examples and executing 42 page access requests; if the network data acquisition and identification method provided by the embodiment is adopted to protect the online shopping platform, the network data acquiring system needs to improve its task system to avoid the solution, for example, the network data acquiring system needs to allocate 20 network data acquiring instances to attack the merchant detail page corresponding to the home page of the merchant list page, each network data acquiring instance needs to acquire the home page and the corresponding merchant detail page respectively, execute 40 page access requests in total, and 20 network data acquisition instances are required to be assigned to attack the merchant detail page corresponding to the second page of the merchant list page, each network data acquisition instance is required to acquire the first page, the second page and the corresponding merchant detail page of the merchant list page, 20 x 3 page access requests are required to be executed, that is, the network data acquisition system needs to execute 100 page access requests in total, and the data acquisition cost is obviously increased; in order to reduce the data acquisition cost and avoid the network data acquisition and identification method provided by this embodiment, the network data acquisition system may use a single network data acquisition instance to attack all targets, that is, the same network data acquisition subject is used to execute 42 page access requests to acquire all merchant detail pages corresponding to the first page and the second page of the merchant list page, and the total request number is consistent with the total request number under the existing network data acquisition task scheduling system and its acquisition policy allocation mechanism, however, the intensive operation of a single account is easily identified and intercepted by the existing identification and interception technology for network data acquisition behavior based on IP dimension and account dimension.

The second embodiment of the present application further provides a network data acquisition and identification device, which is substantially similar to the method embodiment and therefore is relatively simple to describe, and the details of the related technical features may be obtained by referring to the corresponding description of the method embodiment provided above, and the following description of the device embodiment is only illustrative. Referring to fig. 3, to understand the embodiment, fig. 3 is a block diagram of a unit of the apparatus provided in the embodiment, and as shown in fig. 3, the apparatus provided in the embodiment includes: a target encryption parameter obtaining unit 201, configured to obtain a target encryption parameter carried by a second network request, where the second network request is implemented based on network response data corresponding to a first network request, and the target encryption parameter is obtained after a first network behavior background identifier corresponding to the first network request is encrypted; the target encryption parameter decryption unit 202 is configured to decrypt the target encryption parameter to obtain a first network behavior background identifier; a second network behavior background identifier obtaining unit 203, configured to obtain a second network behavior background identifier corresponding to the second network request; the network behavior background identifier comparing unit 204 is configured to compare the first network behavior background identifier with the second network behavior background identifier, and determine whether the access subject corresponding to the second network request is a network data obtaining subject based on a comparison result.

The first network behavior context identification comprises: a first network requests corresponding user context information; the second network behavior context identification comprises: the second network requests the corresponding user context information; comparing the first network behavior background identifier with the second network behavior background identifier, and determining whether the access subject corresponding to the second network request is a network data acquisition subject based on the comparison result, including: comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request; and if the user context information corresponding to the first network request is not matched with the user context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject.

The user context information includes at least one of: user identification information; network operating environment information; comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request, comprising: comparing the user identification information corresponding to the first network request with the user identification information corresponding to the second network request; and/or comparing the network operation environment information corresponding to the first network request with the network operation environment information corresponding to the second network request. The user identification information includes: user identification information, or login session identification information for anonymous users. The network operating environment information includes at least one of: network device identification information; network device IP address.

The first network behavior context identification comprises: the first network requests corresponding application context information; the second network behavior context identification comprises: the second network requests corresponding application context information; comparing the first network behavior background identifier with the second network behavior background identifier, and determining whether the access subject corresponding to the second network request is a network data acquisition subject based on the comparison result, including: comparing the application context information corresponding to the first network request with the application context information corresponding to the second network request; and if the application context information corresponding to the first network request is not matched with the application context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject.

The first network behavior context identification comprises: the user context information corresponding to the first network request and the application context information corresponding to the first network request; the second network behavior context identification comprises: the user context information corresponding to the second network request and the application context information corresponding to the second network request; comparing the first network behavior background identifier with the second network behavior background identifier, and determining whether the access subject corresponding to the second network request is a network data acquisition subject based on the comparison result, including: comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request, and comparing the application context information corresponding to the first network request with the application context information corresponding to the second network request; and if the user context information corresponding to the first network request is not matched with the user context information corresponding to the second network request, and/or the application context information corresponding to the first network request is not matched with the application context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject.

The application context information includes at least one of: applying channel information; application interface information. Further comprising: and the second network request intercepting unit is used for intercepting the second network request. Acquiring a target encryption parameter carried by the second network request, including: acquiring a target encryption parameter carried by a second network request aiming at target network data; the second network request is realized based on the network response data corresponding to the first network request, and comprises the following steps: the second network request is implemented based on the access policy data for the target network data contained in the network response data corresponding to the first network request.

Further comprising: a network response data acquisition unit configured to acquire network response data in response to the first network request, the network response data including initial access policy data for the target network data; a first network behavior background identifier obtaining unit, configured to obtain a first network behavior background identifier corresponding to the first network request; the encryption processing unit is used for encrypting the first network behavior background identifier corresponding to the first network request to obtain a target encryption parameter; the data providing unit is used for obtaining target access policy data aiming at the target network data according to the initial access policy data and the target encryption parameter and providing the target access policy data to an access subject corresponding to the first network request; or, the initial access policy data and the target encryption parameter are provided to the access subject corresponding to the first network request, so that the access subject obtains the target access policy data for the target network data based on the initial access policy data and the target encryption parameter.

The access policy data for the target network data includes: and URL corresponding to the target network data. The target encryption parameter is included in the parameters of the URL. The method for encrypting the first network behavior background identification corresponding to the first network request comprises the following steps: and encrypting the identification information of the target network data and the first network behavior background identification. The initial access strategy data aiming at the target network data is an initial URL corresponding to the target network data, and the target access strategy data aiming at the target network data is a target URL obtained after the identification information of the target network data contained in the initial URL is replaced by a target encryption parameter; providing the target access policy data to an access subject corresponding to the first network request, including: and providing the target URL to an access subject corresponding to the first network request.

Providing the initial access policy data and the target encryption parameter to an access subject corresponding to the first network request, including: and sending the data structure file containing the initial access policy data and the target encryption parameter to an access subject corresponding to the first network request. The target encryption parameter is included in a request header or a request body of the second network request. The first network request includes: an access main body corresponding to the first network request shares target network data with a sharing request of a target sharing main body; the first network behavior context identifier corresponding to the first network request comprises: and the target shares the network behavior background mark corresponding to the main body. The method for encrypting the first network behavior background identification corresponding to the first network request comprises the following steps: and encrypting the user identification information of the access subject corresponding to the first network request and the network behavior background identification corresponding to the target sharing subject.

The method for encrypting the first network behavior background identification corresponding to the first network request comprises the following steps: encrypting the first network behavior background identification by using a target encryption key in the multiple sets of keys, wherein the second network request comprises a target encryption key corresponding to the target key identification; correspondingly, decrypting the target encryption parameter comprises the following steps: and decrypting the target encryption parameter by using the target decryption key corresponding to the target key identifier.

The method for encrypting the first network behavior background identification corresponding to the first network request comprises the following steps: and encrypting the first network behavior background identification corresponding to the first network request based on a symmetric key cryptosystem. The first network behavior context identification comprises: time stamp information. The first network request is a network access request for a merchant listing page and the second network request is a network access request for a merchant details page.

The network data acquiring and identifying device provided in this embodiment decrypts a target encryption parameter after acquiring the target encryption parameter carried by a second network request, obtains a first network behavior background identifier (one or more of user context information such as a user ID, a login session ID, a network device IP address, and application context information such as application channel information and application interface information corresponding to the first network request), and obtains a second network behavior background identifier corresponding to the second network request (one or more of user context information such as a user ID, a login session ID, a network device IP address, and application context information such as application channel information and application interface information corresponding to the second network request), compares the first network behavior background identifier with the second network behavior background identifier, and determining whether an access subject corresponding to a second network request is a network data acquisition subject or not based on the comparison result, wherein the second network request is realized based on network response data corresponding to the first network request, and the target encryption parameter is a parameter obtained after encryption processing is performed on a first network behavior background identifier corresponding to the first network request. By using the device, accurate identification can be carried out on the network data acquisition task scheduling system and the data acquisition operation under the acquisition strategy distribution mechanism.

In the embodiments described above, a network data acquisition and identification method and a network data acquisition and identification device are provided, and in addition, a third embodiment of the present application also provides an electronic device, which is basically similar to the method embodiment and therefore is relatively simple to describe, and for details of relevant technical features, reference may be made to the corresponding description of the method embodiment provided above, and the following description of the electronic device embodiment is only illustrative. The embodiment of the electronic equipment is as follows: please refer to fig. 4 for understanding the present embodiment, fig. 4 is a schematic view of an electronic device provided in the present embodiment. As shown in fig. 4, the electronic device provided in this embodiment includes: a processor 301 and a memory 302; the memory 302 is used for storing computer instructions of the network data acquisition identification method, and when the computer instructions are read and executed by the processor 301, the computer instructions perform the following operations: acquiring a target encryption parameter carried by a second network request, wherein the second network request is realized based on network response data corresponding to the first network request, and the target encryption parameter is obtained after encryption processing is performed on a first network behavior background identifier corresponding to the first network request; decrypting the target encryption parameter to obtain a first network behavior background identifier; acquiring a second network behavior background identifier corresponding to the second network request; and comparing the first network behavior background identification with the second network behavior background identification, and determining whether the access subject corresponding to the second network request is a network data acquisition subject or not based on the comparison result.

The first network behavior context identification comprises: the user context information corresponding to the first network request and the application context information corresponding to the first network request; the second network behavior context identification comprises: the user context information corresponding to the second network request and the application context information corresponding to the second network request; comparing the first network behavior background identifier with the second network behavior background identifier, and determining whether the access subject corresponding to the second network request is a network data acquisition subject based on the comparison result, including: comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request, and comparing the application context information corresponding to the first network request with the application context information corresponding to the second network request; and if the user context information corresponding to the first network request is not matched with the user context information corresponding to the second network request, and/or the application context information corresponding to the first network request is not matched with the application context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject. The application context information includes at least one of: applying channel information; application interface information. Further comprising: a second network request is intercepted. Acquiring a target encryption parameter carried by the second network request, including: acquiring a target encryption parameter carried by a second network request aiming at target network data; the second network request is realized based on the network response data corresponding to the first network request, and comprises the following steps: the second network request is implemented based on the access policy data for the target network data contained in the network response data corresponding to the first network request. Further comprising: obtaining network response data in response to the first network request, the network response data including initial access policy data for the target network data; acquiring a first network behavior background identifier corresponding to a first network request; encrypting a first network behavior background identifier corresponding to the first network request to obtain a target encryption parameter; obtaining target access policy data aiming at the target network data according to the initial access policy data and the target encryption parameter, and providing the target access policy data to an access subject corresponding to the first network request; or, the initial access policy data and the target encryption parameter are provided to the access subject corresponding to the first network request, so that the access subject obtains the target access policy data for the target network data based on the initial access policy data and the target encryption parameter.

The access policy data for the target network data includes: and URL corresponding to the target network data. The target encryption parameter is included in the parameters of the URL. The method for encrypting the first network behavior background identification corresponding to the first network request comprises the following steps: and encrypting the identification information of the target network data and the first network behavior background identification. The initial access strategy data aiming at the target network data is an initial URL corresponding to the target network data, and the target access strategy data aiming at the target network data is a target URL obtained after the identification information of the target network data contained in the initial URL is replaced by a target encryption parameter; providing the target access policy data to an access subject corresponding to the first network request, including: and providing the target URL to an access subject corresponding to the first network request. Providing the initial access policy data and the target encryption parameter to an access subject corresponding to the first network request, including: and sending the data structure file containing the initial access policy data and the target encryption parameter to an access subject corresponding to the first network request. The target encryption parameter is included in a request header or a request body of the second network request.

The first network request includes: an access main body corresponding to the first network request shares target network data with a sharing request of a target sharing main body; the first network behavior context identifier corresponding to the first network request comprises: a network behavior background identifier corresponding to the target sharing subject; the method for encrypting the first network behavior background identification corresponding to the first network request comprises the following steps: and encrypting the user identification information of the access subject corresponding to the first network request and the network behavior background identification corresponding to the target sharing subject. The method for encrypting the first network behavior background identification corresponding to the first network request comprises the following steps: encrypting the first network behavior background identification by using a target encryption key in the multiple sets of keys, wherein the second network request comprises a target encryption key corresponding to the target key identification; correspondingly, decrypting the target encryption parameter comprises the following steps: and decrypting the target encryption parameter by using the target decryption key corresponding to the target key identifier. The method for encrypting the first network behavior background identification corresponding to the first network request comprises the following steps: and encrypting the first network behavior background identification corresponding to the first network request based on a symmetric key cryptosystem. The first network behavior context identification comprises: time stamp information. The first network request is a network access request for a merchant listing page and the second network request is a network access request for a merchant details page.

The electronic device provided in this embodiment, after obtaining a target encryption parameter carried by a second network request, decrypts the target encryption parameter to obtain a first network behavior background identifier (one or more of user context information such as a user ID, a login session ID, a network device ID, and a network device IP address corresponding to the first network request, and application context information such as application channel information and application interface information), and obtains a second network behavior background identifier corresponding to the second network request (one or more of user context information such as a user ID, a login session ID, a network device ID, and a network device IP address corresponding to the second network request, and application context information such as application channel information and application interface information), compares the first network behavior background identifier with the second network behavior background identifier, and determining whether an access subject corresponding to a second network request is a network data acquisition subject or not based on the comparison result, wherein the second network request is realized based on network response data corresponding to the first network request, and the target encryption parameter is obtained after encryption processing is performed on a first network behavior background identifier corresponding to the first network request. By using the electronic equipment, accurate identification can be performed on the network data acquisition task scheduling system and the data acquisition operation under the acquisition strategy distribution mechanism.

In the foregoing embodiments, a network data acquisition and identification method, a network data acquisition and identification device, and an electronic device are provided, and in addition, a fourth embodiment of the present application further provides a computer-readable storage medium for implementing the network data acquisition and identification method. The embodiments of the computer-readable storage medium provided in the present application are described relatively simply, and for relevant portions, reference may be made to the corresponding descriptions of the above method embodiments, and the embodiments described below are merely illustrative.

The present embodiments provide a computer readable storage medium having stored thereon computer instructions that, when executed by a processor, perform the steps of: acquiring a target encryption parameter carried by a second network request, wherein the second network request is realized based on network response data corresponding to the first network request, and the target encryption parameter is obtained after encryption processing is performed on a first network behavior background identifier corresponding to the first network request; decrypting the target encryption parameter to obtain a first network behavior background identifier; acquiring a second network behavior background identifier corresponding to the second network request; and comparing the first network behavior background identification with the second network behavior background identification, and determining whether the access subject corresponding to the second network request is a network data acquisition subject or not based on the comparison result.

After the computer instruction stored in the computer-readable storage medium provided in this embodiment is executed, after obtaining a target encryption parameter carried by a second network request, the computer instruction decrypts the target encryption parameter to obtain a first network behavior context identifier (one or more of user context information such as a user ID, a login session ID, a network device ID, and a network device IP address corresponding to the first network request, and application context information such as application channel information and application interface information), and obtains a second network behavior context identifier corresponding to the second network request (one or more of user context information such as a user ID, a login session ID, a network device ID, and a network device IP address corresponding to the second network request, and application context information such as application channel information and application interface information), compares the first network behavior context identifier with the second network behavior context identifier, and determining whether an access subject corresponding to a second network request is a network data acquisition subject or not based on the comparison result, wherein the second network request is realized based on network response data corresponding to the first network request, and the target encryption parameter is obtained after encryption processing is performed on a first network behavior background identifier corresponding to the first network request. Through the process, the accurate identification can be carried out on the network data acquisition task scheduling system and the data acquisition operation under the acquisition strategy distribution mechanism.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

1. Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.

2. As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Although the present application has been described with reference to the preferred embodiments, it is not intended to limit the present application, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present application, therefore, the scope of the present application should be determined by the claims that follow.

Claims

1. A network data acquisition and identification method is characterized by comprising the following steps:

acquiring a target encryption parameter carried by a second network request, wherein the second network request is realized based on network response data corresponding to a first network request, and the target encryption parameter is obtained after encryption processing is performed on a first network behavior background identifier corresponding to the first network request;

decrypting the target encryption parameter to obtain the first network behavior background identifier;

acquiring a second network behavior background identifier corresponding to the second network request;

and comparing the first network behavior background identification with the second network behavior background identification, and determining whether an access subject corresponding to the second network request is a network data acquisition subject or not based on a comparison result.

2. The method of claim 1, wherein the first network behavior context identification comprises: the first network requests corresponding user context information;

the second network behavior context identification comprises: the second network requests corresponding user context information;

the comparing the first network behavior background identifier with the second network behavior background identifier, and determining whether an access subject corresponding to the second network request is a network data acquisition subject based on a comparison result, includes:

comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request;

and if the user context information corresponding to the first network request is not matched with the user context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject.

3. The method of claim 2, wherein the user context information comprises at least one of: user identification information; network operating environment information;

the comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request includes:

comparing the user identification information corresponding to the first network request with the user identification information corresponding to the second network request; and/or comparing the network operation environment information corresponding to the first network request with the network operation environment information corresponding to the second network request.

4. The method of claim 3, wherein the user identification information comprises: user identity identification information, or login session identification information for anonymous users.

5. The method of claim 3, wherein the network operating environment information comprises at least one of: network device identification information; network device IP address.

6. The method of claim 1, wherein the first network behavior context identification comprises: the first network requests corresponding application context information;

the second network behavior context identification comprises: the second network requests corresponding application context information;

comparing the application context information corresponding to the first network request with the application context information corresponding to the second network request;

and if the application context information corresponding to the first network request is not matched with the application context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject.

7. The method of claim 1, wherein the first network behavior context identification comprises: the first network request corresponds to user context information and the first network request corresponds to application context information;

the second network behavior context identification comprises: the second network request corresponds to user context information and the second network request corresponds to application context information;

comparing the user context information corresponding to the first network request with the user context information corresponding to the second network request, and comparing the application context information corresponding to the first network request with the application context information corresponding to the second network request;

and if the user context information corresponding to the first network request is not matched with the user context information corresponding to the second network request, and/or the application context information corresponding to the first network request is not matched with the application context information corresponding to the second network request, determining that the access subject corresponding to the second network request is a network data acquisition subject.

8. A network data acquisition identification device, comprising:

a target encryption parameter obtaining unit, configured to obtain a target encryption parameter carried by a second network request, where the second network request is implemented based on network response data corresponding to a first network request, and the target encryption parameter is obtained after a first network behavior context identifier corresponding to the first network request is encrypted;

the target encryption parameter decryption unit is used for decrypting the target encryption parameter to obtain the first network behavior background identifier;

a second network behavior background identifier obtaining unit, configured to obtain a second network behavior background identifier corresponding to the second network request;

and the network behavior background identifier comparison unit is used for comparing the first network behavior background identifier with the second network behavior background identifier and determining whether the access subject corresponding to the second network request is a network data acquisition subject or not based on a comparison result.

9. An electronic device comprising a processor and a memory; wherein the content of the first and second substances,

the memory is to store one or more computer instructions, wherein the one or more computer instructions are to be executed by the processor to implement the method of claims 1-7.

10. A computer-readable storage medium having stored thereon one or more computer instructions for execution by a processor to perform the method of claims 1-7.