CN118158123A - Method and device for detecting crawler behaviors, electronic equipment and computer storage medium - Google Patents
Method and device for detecting crawler behaviors, electronic equipment and computer storage medium Download PDFInfo
- Publication number
- CN118158123A CN118158123A CN202410244771.9A CN202410244771A CN118158123A CN 118158123 A CN118158123 A CN 118158123A CN 202410244771 A CN202410244771 A CN 202410244771A CN 118158123 A CN118158123 A CN 118158123A
- Authority
- CN
- China
- Prior art keywords
- crawler
- target application
- sensor data
- target
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000006399 behavior Effects 0.000 title claims abstract description 311
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000010801 machine learning Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012549 training Methods 0.000 claims description 7
- 238000001514 detection method Methods 0.000 abstract description 12
- 230000002265 prevention Effects 0.000 abstract description 5
- 238000000605 extraction Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 238000011176 pooling Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000036544 posture Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013515 script Methods 0.000 description 2
- 241000239290 Araneae Species 0.000 description 1
- 241000257303 Hymenoptera Species 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Landscapes
- Testing Or Calibration Of Command Recording Devices (AREA)
Abstract
The application discloses a method, a device, electronic equipment and a computer storage medium for detecting crawler behaviors, wherein the method comprises the following steps: acquiring device sensor data of a target device running a target application, wherein the device sensor data is used for representing spatial characteristics of the target device; obtaining a judging result of whether the target application has a crawler behavior according to the equipment sensor data; and if the judgment result of the target application with the crawler behavior is obtained, outputting prompt information of the target application with the crawler behavior. The detection method of the crawler behaviors can accurately judge whether the target application has the crawler behaviors or not, and the crawler prevention and control level of the target application is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and apparatus for detecting crawler behaviors, an electronic device, and a computer storage medium.
Background
Web crawlers are automated programs that browse fixed websites by writing software or scripts, and with the development of internet technology, there are a large number of malicious crawlers in addition to search engine crawlers. The damage caused by malicious crawlers is that on one hand, the server bandwidth is occupied, and the traffic of normal users is squeezed; on the other hand, information of the enterprise is crawled, resulting in the leakage or misuse of the invaluable information assets.
The current common anti-crawler system is used for judging a crawler based on the access frequency of equipment running a target application, when the access frequency of certain equipment exceeds a preset threshold value, the access request of the equipment is intercepted, but when the equipment is normally accessed in actual use, the access frequency also exceeds the preset threshold value, and at the moment, the normal access of a user can be intercepted. Therefore, the anti-crawler system cannot accurately judge whether the target application running on the equipment has crawler behaviors or not, and the phenomenon of interception easily occurs.
Therefore, how to provide a method for detecting the crawler behavior, so as to accurately identify whether the target application running on the device has the crawler behavior, is a problem to be solved at present.
Disclosure of Invention
The embodiment of the application provides a method for detecting crawler behaviors, which can accurately judge whether the target application running on equipment has the crawler behaviors or not, and improves the crawler prevention and control level of the target application.
The embodiment of the application provides a method for detecting crawler behaviors, which comprises the following steps: acquiring device sensor data of a target device running a target application, wherein the device sensor data is used for representing spatial characteristics of the target device; obtaining a judging result of whether the target application has a crawler behavior according to the equipment sensor data; and if the judgment result of the target application with the crawler behavior is obtained, outputting prompt information of the target application with the crawler behavior.
Optionally, the obtaining, according to the device sensor data, a result of determining whether the target application has a crawler behavior includes:
Inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors or not, and obtaining a prediction result for indicating whether the target application has crawler behaviors or not;
And obtaining a judging result of whether the target application has the crawler behaviors according to the predicting result.
Optionally, the crawler behavior recognition model includes a first recognition model and a second recognition model;
Inputting the device sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors, and obtaining a prediction result for indicating whether the target application has crawler behaviors, wherein the method comprises the following steps of:
inputting the equipment sensor data into a first recognition model to obtain a first predicted value used for representing whether the target application has crawler behaviors;
and inputting the device sensor data into a second recognition model to obtain a second predicted value for representing whether the target application has the crawler behaviors.
Optionally, the obtaining, according to the prediction result, a determination result of whether the target application has a crawler behavior includes:
and if at least one of the first predicted value and the second predicted value accords with the preset condition of the existence of the crawler behavior, obtaining a judging result of the existence of the crawler behavior of the target application.
Optionally, the device sensor data comprises a device sensor data sequence.
Optionally, the type of the device sensor data sequence includes at least one of:
four-dimensional data consisting of X-axis data, Y-axis data, Z-axis data and electronic compass data of the target equipment;
Or three-dimensional data consisting of any three of X-axis data, Y-axis data, Z-axis data and electronic compass data of the target equipment;
Or two-dimensional data consisting of any two of X-axis data, Y-axis data, Z-axis data and electronic compass data of the target device.
Optionally, the acquiring device sensor data of the target device running the target application includes:
acquiring device sensor data of the target device through a device sensor data acquisition device set by the target application;
Or receiving an access request sent by the target application, and acquiring the equipment sensor data of the target equipment according to the access request, wherein the access request contains the equipment sensor data of the target equipment.
Optionally, the first recognition model is a machine learning model, and the machine learning model is trained in the following manner:
Obtaining sample data of device sensor data of target devices of target applications, wherein the sample data comprises device sensor data samples of target applications with crawler behaviors, device sensor data samples of target applications without crawler behaviors and identification result tags of the target applications, and the identification result tags of the target applications comprise: the target application has an identification tag of a crawler behavior or the target application does not have an identification tag of the crawler behavior;
And training an initial machine learning model based on the sample data to obtain the machine learning model as the first recognition model.
Optionally, the first recognition model includes: a sequence feature extraction layer and an identification output layer;
The inputting the device sensor data into a first recognition model to obtain a first predicted value for representing whether the target application has crawler behaviors, including:
Inputting the device sensor data into the sequence feature extraction layer to obtain the device sensor data sequence features output by the sequence feature extraction layer;
and inputting the data sequence characteristics of the equipment sensor into the identification output layer to obtain a first predicted value of whether the crawler behaviors exist or not corresponding to the target application output by the identification output layer.
Optionally, the sequence feature extraction layer includes a pooling layer, and the sequence feature extraction layer uses maximum pooling as the pooling layer;
The identification output layer includes a residual layer.
Optionally, the method further comprises:
When acquiring device sensor data sample data of a target device running a target application, if the number of the sample data is lower than a preset threshold value, segmenting longer device sensor data in the device sensor data sample data into a plurality of device sensor data serving as a plurality of device sensor data sample data of the target device.
Optionally, the second recognition model is used for recognizing whether the target application running on the device with the variation of the spatial feature within the preset range has the crawler behavior.
Optionally, the inputting the device sensor data into a second recognition model, to obtain a second predicted value for indicating whether the target application has a crawler behavior, includes:
acquiring a plurality of device sensor data of a target device running a target application;
acquiring a reference value of the plurality of device sensor data;
Comparing each device sensor data of the plurality of device sensor data with the reference value to obtain a plurality of difference values;
acquiring a first quantity value of the device sensor data;
The plurality of difference values and the first number of values of the device sensor data are input into a second recognition model, and a second predicted value for representing whether the target application has crawler behaviors is obtained.
Optionally, the inputting the plurality of difference values and the first number of values of the device sensor data into a second recognition model, to obtain a second predicted value for indicating whether the target application has a crawler behavior, includes:
acquiring a second quantity value of the difference values, wherein the difference value of the plurality of difference values is smaller than a preset threshold value;
Acquiring a ratio of the second quantity value to the first quantity value;
And obtaining a second predicted value for representing whether the target application has the crawler behaviors according to the ratio.
Optionally, the obtaining, according to the ratio, a second predicted value for indicating whether the target application has a crawler behavior includes:
and if the ratio exceeds a preset threshold, obtaining a second predicted value for representing that the target application has the crawler behavior.
Optionally, the method further comprises:
acquiring a plurality of device sensor data of a plurality of target devices running a target application at the same time;
Inputting each device sensor data into the crawler behavior recognition model to obtain a plurality of prediction results used for representing whether the target application has crawler behaviors;
Acquiring position data of a plurality of target applications;
And if the difference values of the plurality of prediction results and the difference values of the position data of the plurality of target applications are smaller than a preset threshold value, obtaining a judgment result of the existence of the crawler behaviors of the group device.
Optionally, the method further comprises:
when acquiring a plurality of device sensor data of target devices running a target application, inputting each device sensor data into a crawler behavior recognition model to acquire a plurality of prediction results used for indicating whether the target application has crawler behaviors;
If more than preset number of the prediction results meet the preset conditions of the existence of the crawler behaviors, obtaining the existence of the crawler behaviors of the target application corresponding to the sensor data of the plurality of devices.
Optionally, the method further comprises:
if the prediction result does not accord with the preset condition of the existence of the crawler behavior, acquiring historical equipment sensor data of a plurality of target applications within the preset range of the target application position data;
Inputting sensor data of each historical equipment into a crawler behavior recognition model to obtain a historical prediction result used for indicating whether the target application has crawler behaviors;
Judging whether the history prediction result accords with preset conditions of the existence of the crawler behaviors or not;
If yes, judging that the target application has crawler behaviors;
If not, marking the target application as suspected crawler behaviors.
Optionally, the method further comprises: acquiring a first auxiliary judgment result, wherein the first auxiliary judgment result is a judgment result for judging whether the target application has a crawler behavior;
Inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors, and obtaining a prediction result for indicating whether the target application has crawler behaviors; according to the prediction result, obtaining a determination result of whether the target application has a crawler behavior, including:
Inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors or not, and obtaining a model judgment result;
And obtaining a judging result of whether the target application has the crawler behaviors according to the model judging result and the first auxiliary judging result.
Optionally, the obtaining the first auxiliary judging result includes:
obtaining the number of merchants accessed by a user and sent by the target application;
Obtaining a threshold value of the number of merchants visited by a normal user;
And acquiring a first auxiliary judgment result according to the number of merchants visited by the user and the threshold value of the number of merchants visited by the normal user.
Optionally, the obtaining a first auxiliary judgment result according to the number of merchants visited by the user and the threshold value of the number of merchants visited by the normal user includes:
And if the number of merchants visited by the user in the preset time exceeds the threshold value of the number of merchants visited by the normal user, judging that the target application has the crawler behavior.
Optionally, the method further comprises: acquiring a second auxiliary judgment result, wherein the second auxiliary judgment result is a judgment result for judging whether the target application has a crawler behavior or not;
Inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors, and obtaining a prediction result for indicating whether the target application has crawler behaviors; according to the prediction result, obtaining a determination result of whether the target application has a crawler behavior, including:
Inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors or not, and obtaining a model judgment result;
And obtaining a judging result of whether the target application has the crawler behaviors according to the model judging result and the second auxiliary judging result.
Optionally, the obtaining the second auxiliary judging result includes:
acquiring the distribution characteristics of the distribution resources sent by the target application, wherein the distribution characteristics comprise distribution duration and distribution distance;
If the variation range of the distribution time length and the distribution distance is within a preset threshold, judging that the target application does not have the crawler behavior, and taking the target application as a second auxiliary judgment result.
Optionally, the method further comprises: acquiring a third auxiliary judgment result, wherein the third auxiliary judgment result is a judgment result for judging whether the target application has a crawler behavior;
Inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors, and obtaining a prediction result for indicating whether the target application has crawler behaviors; according to the prediction result, obtaining a determination result of whether the target application has a crawler behavior, including:
Inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors or not, and obtaining a model judgment result;
and obtaining a judging result of whether the target application has the crawler behaviors according to the model judging result and the third auxiliary judging result.
Optionally, the obtaining the third auxiliary judging result includes:
If the number of times of access requests sent by the target application exceeds a preset threshold value within a preset time and the ordering operation corresponding to the target application is not recognized, determining that the target application has crawler behaviors, and taking the crawler behaviors as a third auxiliary determination result.
The embodiment of the application also provides a device for detecting the crawler behaviors, which comprises the following steps: an acquisition unit configured to acquire device sensor data of a target device running a target application, the device sensor data being used to represent a spatial feature of the target device; the obtaining unit is used for obtaining a judging result of whether the target application has the crawler behaviors according to the equipment sensor data; and the input unit is used for outputting prompt information of the existence of the crawler behavior of the target application if the judgment result of the existence of the crawler behavior of the target application is obtained.
The application also provides electronic equipment, which comprises a processor and a memory; the memory stores a computer program, and the processor executes the method after running the computer program.
The present application also provides a computer storage medium storing a computer program which, when executed by a processor, performs the above method.
Compared with the prior art, the embodiment of the application has the following advantages:
The embodiment of the application provides a method for detecting crawler behaviors, which comprises the following steps: acquiring device sensor data of a target device running a target application, wherein the device sensor data is used for representing spatial characteristics of the target device; obtaining a judging result of whether the target application has a crawler behavior according to the equipment sensor data; and if the judgment result of the target application with the crawler behavior is obtained, outputting prompt information of the target application with the crawler behavior.
According to the method for detecting the crawler behaviors, the judgment result of whether the target application has the crawler behaviors can be obtained according to the equipment sensor data of the target application, and when the target application has the crawler behaviors, prompt information of the target application having the crawler behaviors is output. Therefore, the crawler behavior detection method can accurately judge whether the target application has crawler behaviors or not.
In a preferred embodiment of the present application, the method for detecting a crawler behavior includes inputting the device sensor data into a first recognition model and a second recognition model for predicting whether the target application has a crawler behavior, and obtaining a first predicted value and a second predicted value for indicating whether the target application has a crawler behavior; and further, according to the first predicted value and the second predicted value, a judging result of whether the target application has the crawler behaviors or not is obtained. According to the crawler behavior detection method, the data of the equipment sensor are input into the first recognition model and the second recognition model, so that whether the target application has the crawler behavior or not can be accurately judged, and the crawler prevention and control level of the target application is improved.
Drawings
Fig. 1 is a schematic diagram of an application scenario of a crawler behavior detection method according to a first embodiment of the present application;
FIG. 2 is a flowchart of a method for detecting crawler behavior according to a first embodiment of the present application;
FIG. 3 is a schematic diagram of device sensor data provided by a first embodiment of the present application;
FIG. 4 is a schematic illustration of a swarm device crawler provided in accordance with a first embodiment of the present application;
FIG. 5 is a schematic diagram of a crawler behavior detection apparatus according to a second embodiment of the present application;
fig. 6 is a schematic diagram of an electronic device according to a third embodiment of the present application.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. The present application may be embodied in many other forms than those herein described, and those skilled in the art will readily appreciate that the present application may be similarly embodied without departing from the spirit or essential characteristics thereof, and therefore the present application is not limited to the specific embodiments disclosed below.
Firstly, in order to enable those skilled in the art to better understand the scheme of the present application, a specific application scenario of an embodiment of the method for detecting crawler behavior provided by the present application is described in detail below. Fig. 1 is a schematic application scenario diagram of a method for detecting crawler behaviors according to a first embodiment of the present application.
Before introducing the application scenario of the embodiment of the present application, the terms related to the embodiment of the present application, namely, web crawlers, web spiders, web robots, etc., are explained first, and in the middle of a community of FOAFs (i.e., friend-of-a-Friend, FOAF. FOAF is an XML/RDF vocabulary describing information such as personal information that may be typically placed on a main Web page in a computer readable form), more often referred to as Web chasers, a Web crawler is a program or script that automatically captures Web information according to a certain rule. Other less commonly used names for crawlers are ants, auto-indexing, simulators or worms, etc. The gyroscope sensor is an electronic component for acquiring the placing posture of the mobile phone in the mobile phone, and acquires the numerical values of X, Y and Z axes of the mobile phone in a three-dimensional space, wherein the numerical values are used for representing the placing posture of the mobile phone in the three-dimensional space and are common electronic components of modern mobile phones. The electronic compass is an electronic component for judging the north-south orientation of the current equipment by measuring the earth magnetic field, is also called as an electronic compass, and is a common electronic component of modern mobile phones.
Based on the background, with the development of internet technology, there are a large number of malicious crawlers in addition to search engine crawlers. The damage caused by malicious crawlers is that on one hand, the server bandwidth is occupied, and the traffic of normal users is squeezed; on the other hand, information of the enterprise is crawled, resulting in unbiased information assets being revealed/abused. The current common anti-crawler system is used for judging crawlers based on the access frequency of equipment, when the access frequency of certain equipment exceeds a preset threshold value, the access request of the equipment is intercepted, but when the equipment is normally accessed in actual use, the access frequency also exceeds the preset threshold value, and at the moment, the normal access of a user can be intercepted. Therefore, the anti-crawler system cannot accurately judge whether the crawler behavior exists in the application running on the equipment, and the phenomenon of interception easily occurs.
Based on the method, the application provides a method for detecting crawler behaviors, which comprises the steps of obtaining device sensor data of target devices running target applications, wherein the device sensor data are used for representing spatial characteristics of the target devices; obtaining a judging result of whether the target application has a crawler behavior according to the equipment sensor data; and if the judgment result of the target application with the crawler behavior is obtained, outputting prompt information of the target application with the crawler behavior. The detection method of the crawler behaviors can accurately judge whether the target application has the crawler behaviors or not, and the crawler prevention and control level of the target application is improved.
In specific implementation, referring to fig. 1, the execution body of the embodiment of the present application may be a server, a user terminal for identifying a crawler behavior, or a terminal device of a worker running a target application. Taking the server as an example, first, step S101 is executed: the server may obtain device sensor data of a target device running the target application, where the device sensor data includes gyroscope data and electronic compass data, such as X, Y, Z axis data of the device and electronic compass data. The device sensor data includes a device sensor data sequence having a length that characterizes an access duration of the target application. The device mainly refers to mobile terminal devices used by users, such as mobile phones, tablet computers and the like, and the Application can be an Application (APP), an applet, a webpage and the like running on the device. When acquiring the device sensor data, specifically, the target device may acquire the device sensor data once every a certain period of time, for example, the target device acquires the device sensor value once every N seconds, and all requests in N seconds carry the same value, so that the gyroscope sequence length is obtained after the device sensor data in all intervals of N seconds are de-duplicated, where the size of N can be set according to actual requirements.
After acquiring the device sensor data, step S102-1 and step S102-2 are performed, step S102-1: inputting the equipment sensor data into a first recognition model to obtain a first predicted value used for representing whether the target application has crawler behaviors; step S102-2: and inputting the device sensor data into a second recognition model to obtain a second predicted value for representing whether the target application has the crawler behaviors. After obtaining the device sensor data of the target device running the target application, the server side inputs the device sensor data into a first recognition model and a second recognition model for predicting whether the target application has a crawler behavior, wherein the first recognition model is a machine learning model and is mainly used for recognizing the application crawled in a fixed mode, and the second recognition model is used for recognizing the crawler behavior of which the gesture of the device is kept unchanged for a long time. After the device sensor data is input into the first recognition model and the second recognition model, a first predicted value and a second predicted value which are used for indicating whether the target application has the crawler behaviors or not are obtained, wherein the first predicted value and the second predicted value can be percentage values used for indicating that the target application has the crawler behaviors.
Finally, step S103 is performed: and obtaining a judging result of whether the target application has the crawler behaviors according to the first predicted value and the second predicted value. The method comprises the following steps: and if at least one of the first predicted value and the second predicted value accords with the preset condition of the existence of the crawler behavior, obtaining a judging result of the existence of the crawler behavior of the target application. And if the first predicted value and the second predicted value do not meet the preset conditions of the existence of the crawler behaviors, obtaining a judging result that the target application does not exist the crawler behaviors. And if the judgment result of the existence of the crawler behavior of the target application is obtained, outputting prompt information of the existence of the crawler behavior of the target application.
The whole process analysis of the detection method of the crawler behaviors is carried out, and device sensor data of target devices running the target applications are obtained, wherein the device sensor data are used for representing the spatial characteristics of the target devices; obtaining a judging result of whether the target application has a crawler behavior according to the equipment sensor data; and if the judgment result of the target application with the crawler behavior is obtained, outputting prompt information of the target application with the crawler behavior. The method for detecting the crawler behaviors can accurately judge whether the target application has the crawler behaviors or not.
In a preferred embodiment of the present application, the method for detecting a crawler behavior includes inputting the device sensor data into a first recognition model and a second recognition model for predicting whether the target application has a crawler behavior, and obtaining a first predicted value and a second predicted value for indicating whether the target application has a crawler behavior; and further, according to the first predicted value and the second predicted value, a judging result of whether the target application has the crawler behaviors or not is obtained. According to the crawler behavior detection method, the data of the equipment sensor are input into the first recognition model and the second recognition model, so that whether the target application has the crawler behavior or not can be accurately judged, and the crawler prevention and control level of the target application is improved.
The present application will be described in detail below with reference to a number of examples and drawings.
First embodiment
The first embodiment of the present application provides a method for detecting a crawler behavior, and the method for detecting a crawler behavior is described in detail below with reference to fig. 2.
Step S201: device sensor data of a target device running a target application is acquired, the device sensor data being used to represent spatial characteristics of the target device.
This step is used to obtain device sensor data of a target device running the target application.
In the embodiment of the application, the equipment mainly refers to mobile terminal equipment used by a user, such as a mobile phone, a tablet personal computer and the like; the application may specifically be an APP, applet, web page, etc. running on the device. The device sensor data is used to represent spatial features of the target device, which may be referred to as position gestures. Thus, the device sensor data of the target device may specifically be data representing a position and orientation of the target device.
In specific implementation, the execution body of the embodiment of the application can be a server, a user side for identifying the behavior of the crawler, or a terminal device of a worker running the target application. Taking the server as an example, when the server acquires the device sensor data, the target device may specifically acquire the device sensor data once every a certain period of time, and then the target device performs de-duplication processing on the acquired device sensor data and uploads the device sensor data to the server for analysis.
The device sensor data comprises a device sensor data sequence, the device sensor data comprises gyroscope data and electronic compass data, the gyroscope data are used for representing the placing postures of the target device in the three-dimensional space in the X, Y and Z axes, and the electronic compass data are used for representing the north-south orientation of the target device. The type of device sensor data sequence includes at least one of: four-dimensional data consisting of X-axis data, Y-axis data, Z-axis data and electronic compass data of the target equipment; or three-dimensional data consisting of any three of X-axis data, Y-axis data, Z-axis data and electronic compass data of the target equipment; or two-dimensional data consisting of any two of X-axis data, Y-axis data, Z-axis data and electronic compass data of the target device.
The length of the device sensor data sequence reflects the access duration of the target device, and when the device sensor data is acquired, a device sensor data sequence with the length of the device sensor data sequence being greater than a preset threshold value can be selected as the device sensor data. Fig. 3 is a schematic diagram of device sensor data provided in the first embodiment of the present application, in fig. 3, the ordinate is device sensor data, the abscissa is sequence position, an X-axis data fluctuation curve 301 represents the fluctuation of X-axis data, a Y-axis data fluctuation curve 302 represents the fluctuation of Y-axis data, a Z-axis data fluctuation curve 303 represents the fluctuation of Z-axis data, and an electronic compass data fluctuation curve 304 represents the fluctuation of electronic compass data. The X-axis data, the Y-axis data, the Z-axis data and the electronic compass data are all normalized data, and the X-axis data, the Y-axis data, the Z-axis data and the electronic compass data are normalized within the range of-1 to 1. The interval between the sequence position data is a preset duration, such as N seconds, the target device collects the device sensor data every N seconds, and each sequence position can correspond to four values, namely, X-axis data, Y-axis data, Z-axis data and electronic compass data.
In an embodiment of the present application, the acquiring device sensor data of a target device running a target application includes: acquiring device sensor data of the target device through a device sensor data acquisition device set by the target application; or receiving an access request sent by the target application, and acquiring the equipment sensor data of the target equipment according to the access request, wherein the access request contains the equipment sensor data of the target equipment. When acquiring the device sensor data, the server side is actually acquired through a device sensor data acquisition device set by the target application, for example, an SDK (Software Development Kit, software development kit, abbreviated as SDK). Or the access request carrying the device sensor data sent by the target application is received, and the device sensor data is obtained by analyzing the access request.
The above steps are used to introduce a process of acquiring device sensor data of the target device, and how to determine whether the target device has a crawler behavior according to the device sensor data is described next.
Step S202: and obtaining a judging result of whether the target application has the crawler behaviors according to the equipment sensor data.
The method is used for judging whether the target application has the crawler behaviors according to the device sensor data.
The step of obtaining a result of judging whether the target application has the crawler behavior according to the device sensor data comprises the following steps: inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors or not, and obtaining a prediction result for indicating whether the target application has crawler behaviors or not; and obtaining a judging result of whether the target application has the crawler behaviors according to the predicting result. The crawler behavior recognition model comprises a first recognition model and a second recognition model; inputting the device sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors, and obtaining a prediction result for indicating whether the target application has crawler behaviors, wherein the method comprises the following steps of: inputting the equipment sensor data into a first recognition model to obtain a first predicted value used for representing whether the target application has crawler behaviors; and inputting the device sensor data into a second recognition model to obtain a second predicted value for representing whether the target application has the crawler behaviors. The step of obtaining a determination result of whether the target application has the crawler behavior according to the prediction result includes: and if at least one of the first predicted value and the second predicted value accords with the preset condition of the existence of the crawler behavior, obtaining a judging result of the existence of the crawler behavior of the target application.
When the method is implemented, whether the target application has the crawler behaviors or not is judged according to the equipment sensor data, the crawler behaviors are judged through a crawler behavior recognition model, the acquired equipment sensor data are input into the crawler behavior recognition model for predicting whether the target application has the crawler behaviors or not, the crawler behavior recognition model comprises a first recognition model and a second recognition model, the first recognition model is a machine learning model and is mainly used for recognizing the application crawled in a fixed mode, and the second recognition model is used for recognizing the crawler behaviors of equipment with the states kept unchanged for a long time. After the device sensor data is input into the first recognition model and the second recognition model, a first predicted value and a second predicted value which are used for indicating whether the target application has the crawler behaviors or not are obtained, wherein the first predicted value and the second predicted value can be percentage values used for indicating that the target application has the crawler behaviors. The first predicted value is obtained by inputting the equipment sensor data into a first recognition model, the second predicted value is obtained by inputting the equipment sensor data into a second recognition model, and a judging result of whether the target application has the crawler behaviors can be obtained according to the first predicted value and the second predicted value. Specifically, if at least one of the first predicted value and the second predicted value accords with a preset condition of existence of the crawler behavior, a judging result of existence of the crawler behavior of the target application is obtained; and if the first predicted value and the second predicted value do not meet the preset conditions of the existence of the crawler behaviors, obtaining a judging result that the target application does not exist the crawler behaviors. The preset condition for existence of the crawler behavior may be set according to an actual requirement, for example, a threshold may be set, and when the first predicted value and the second predicted value exceed the preset threshold, it may be determined that the target application has the crawler behavior.
In the embodiment of the present application, the first recognition model is a machine learning model, and the first predicted value is obtained after the device sensor data is input into the first recognition model, where the first predicted value may be a Y value. The Y value is different from Y in the Y-axis data, where Y in the Y-axis data refers to a Y coordinate axis in the XYZ coordinate axis of the coordinate system, and here, the Y value represents a first predicted value obtained after the device sensor data is input into the first recognition model, and the range of the Y value is normalized to a range of 0to 1. And if the equipment sensor data is identified by the first identification model, the output Y value is 0, and the target application is proved to have no crawler behaviors, and if the equipment sensor data is identified by the first identification model, the output Y value is closer to 1, the target application is proved to have the crawler behaviors possibly. The machine learning model is trained as follows: obtaining sample data of device sensor data of target devices of target applications, wherein the sample data comprises device sensor data samples of target applications with crawler behaviors, device sensor data samples of target applications without crawler behaviors and identification result tags of the target applications, and the identification result tags of the target applications comprise: the target application has an identification tag of a crawler behavior or the target application does not have an identification tag of the crawler behavior; and training an initial machine learning model based on the sample data to obtain the machine learning model as the first recognition model.
When acquiring device sensor data sample data of a target device running a target application, if the number of the sample data is lower than a preset threshold, splitting longer device sensor data in the device sensor data sample data into a plurality of device sensor data serving as a plurality of device sensor data sample data of the target device. When the model training is actually performed, the situation that the model training samples are insufficient may exist, and the training sample size can be expanded through a data enhancement means at this time, specifically, a longer equipment sensor data sequence is segmented into sequences with several standard lengths, so that the training sample size can be greatly expanded.
The first recognition model adopts InceptionTime structure, inception network is composed of different residual blocks, each block is composed of 3 Inception sub-modules instead of a traditional full connection layer. The input of each residual block is transferred to the input of the next block through a fast linear connection, thereby alleviating the gradient vanishing problem by allowing a direct flow of gradients. For Inception modules, when the input is an M-dimensional multivariate time series, the "bottleneck" layer in Inception module is first traversed. In this layer, M filters with length of 1 and step length of 1 are used to convert a signal of M dimension into M dimension (M < M), so as to reduce dimension, reduce complexity of model and alleviate over fitting problem. The filters used in this layer may be large, even 10 times larger than ResNet, where multiple different size filters are used simultaneously to process the input, such as 3 different size convolutions. Meanwhile, in order to keep the model unchanged from small changes, an operation similar to MaxPooling is added after bottleneck layers, so that the dimension can be further reduced. Finally, a plurality of parallel convolutional layer-max-pooling layers are combined to form the current Inception module output.
The first recognition model includes: a sequence feature extraction layer and an identification output layer; the inputting the device sensor data into a first recognition model to obtain a first predicted value for representing whether the target application has crawler behaviors, including: inputting the device sensor data into the sequence feature extraction layer to obtain the device sensor data sequence features output by the sequence feature extraction layer; and inputting the data sequence characteristics of the equipment sensor into the identification output layer to obtain a first predicted value of whether the crawler behaviors exist or not corresponding to the target application output by the identification output layer. The sequence feature extraction layer comprises a pooling layer, and the sequence feature extraction layer uses maximum pooling as the pooling layer; the identification output layer includes a residual layer.
The above is an introduction to the first recognition model, i.e., the machine learning model, and the second recognition model is described below.
The second recognition model is used for recognizing whether the target application running on the equipment with the variable quantity of the spatial characteristics within the preset range has the crawler behaviors or not. Specifically, the inputting the device sensor data into a second recognition model to obtain a second predicted value for indicating whether the target application has a crawler behavior, including: acquiring a plurality of device sensor data of a target device running a target application; acquiring a reference value of the plurality of device sensor data; comparing each device sensor data of the plurality of device sensor data with the reference value to obtain a plurality of difference values; acquiring a first quantity value of the device sensor data; the plurality of difference values and the first number of values of the device sensor data are input into a second recognition model, and a second predicted value for representing whether the target application has crawler behaviors is obtained. The inputting the plurality of difference values and the first number of values of the device sensor data into a second recognition model to obtain a second predicted value for indicating whether the target application has crawler behavior, comprising: acquiring a second quantity value of the difference values, wherein the difference value of the plurality of difference values is smaller than a preset threshold value; acquiring a ratio of the second quantity value to the first quantity value; and obtaining a second predicted value for representing whether the target application has the crawler behaviors according to the ratio. The obtaining, according to the ratio, a second predicted value for indicating whether the target application has a crawler behavior, including: and if the ratio exceeds a preset threshold, obtaining a second predicted value for representing that the target application has the crawler behavior.
Here, the second recognition model is mainly used for recognizing the crawler behavior that the posture of the device remains unchanged for a long time, so that when the second recognition model is used for crawler recognition, a plurality of device sensor data of the target device can be acquired first, and all the device sensor data are differenced from a preset reference value to obtain a plurality of difference values. If the proportion of the equipment sensor data with small difference value exceeds a preset threshold value, the equipment corresponding to the equipment sensor data is proved to be in a preset range, and the equipment belongs to the group equipment crawler behaviors. The preset threshold value can be set specifically according to actual requirements.
When the crawler identification is performed by using the second identification model, a plurality of device sensor data of a plurality of target devices running the target application at the same time can be acquired first; inputting each device sensor data into the crawler behavior recognition model to obtain a plurality of prediction results used for representing whether the target application has crawler behaviors; acquiring position data of a plurality of target applications; and if the difference values of the plurality of prediction results and the difference values of the position data of the plurality of target applications are smaller than a preset threshold value, obtaining a judgment result of the existence of the crawler behaviors of the group device. The position data of the plurality of target applications may be specifically obtained by a satellite positioning system, for example, GPS (global positioning system, global Positioning System, abbreviated as GPS) data of the target applications are obtained, and when the difference value of the GPS data of the plurality of target applications is smaller than a preset threshold value, it is proved that the plurality of target applications are in the same preset range.
It should be noted that, referring to fig. 4, fig. 4 is a schematic diagram of a group device crawler according to the first embodiment of the present application, in fig. 4, taking a first device 401, a second device 402, a third device 403, a fourth device 404, a fifth device 405, a sixth device 406, a seventh device 407, and an eighth device 408 as examples, differences of position data of the eight devices are smaller than a preset threshold, and differences of device sensor data corresponding to each device and a preset reference value are also smaller than the preset threshold, it may be determined that the group device crawler behavior exists.
When a crawler behavior recognition model is used for judging, if multiple device sensor data of target devices running a target application are acquired, inputting each device sensor data into the crawler behavior recognition model, and acquiring multiple prediction results for representing whether the target application has crawler behaviors; if more than preset number of the prediction results meet the preset conditions of the existence of the crawler behaviors, obtaining the existence of the crawler behaviors of the target application corresponding to the sensor data of the plurality of devices. It should be noted that one device may have a plurality of device sensor data sequences, and after inputting the plurality of device sensor data sequences into the crawler behavior recognition model, a plurality of prediction results are correspondingly obtained, and if some of the prediction results conform to the condition that the crawler behavior exists, it may be determined that the target application exists the crawler behavior.
When the crawler identification is performed by using the second identification model, if the prediction result does not accord with the preset condition of existence of the crawler behavior, acquiring historical equipment sensor data of a plurality of target applications within the preset range of the target application position data; inputting sensor data of each historical equipment into a crawler behavior recognition model to obtain a historical prediction result used for indicating whether the target application has crawler behaviors; judging whether the history prediction result accords with preset conditions of the existence of the crawler behaviors or not; if yes, judging that the target application has crawler behaviors; if not, marking the target application as suspected crawler behaviors. In practical application, there may be an unstable condition of the device, so that the identified prediction result points to a situation that no crawler behavior exists, at this time, the historical device sensor data sequence of the next device may be focused, the crawler behavior identification is performed according to the historical device sensor data, if the historical prediction result is that the crawler behavior exists, it may be determined that the target application exists, if the historical prediction result does not conform to the preset condition that the crawler behavior exists, the target application may be marked, and then focus on whether the crawler behavior exists or not.
In the embodiment of the application, when judging whether the target application has the crawler behavior, a first auxiliary judgment result can be obtained, wherein the first auxiliary judgment result is a judgment result for judging whether the target application has the crawler behavior; inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors, and obtaining a prediction result for indicating whether the target application has crawler behaviors; according to the prediction result, obtaining a determination result of whether the target application has a crawler behavior, including: inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors or not, and obtaining a model judgment result; and obtaining a judging result of whether the target application has the crawler behaviors according to the model judging result and the first auxiliary judging result.
The obtaining the first auxiliary judging result includes: obtaining the number of merchants accessed by a user and sent by the target application; obtaining a threshold value of the number of merchants visited by a normal user; and acquiring a first auxiliary judgment result according to the number of merchants visited by the user and the threshold value of the number of merchants visited by the normal user. The step of obtaining a first auxiliary judgment result according to the number of merchants visited by the user and the threshold value of the number of merchants visited by the normal user comprises the following steps: and if the number of merchants visited by the user in the preset time exceeds the threshold value of the number of merchants visited by the normal user, judging that the target application has the crawler behavior. It should be noted that, the first auxiliary determination result is used for assisting in determining whether the target application has the crawler behavior, and under normal conditions, the number of merchants browsed by the user is relatively fixed, generally, the number of merchants browsed by the user is usually not too large, and if the user is identified to visit a large number of merchants in a short time, it is indicated that the crawler behavior may exist in the normal operation of the user.
In the embodiment of the application, when judging whether the target application has the crawler behavior, a second auxiliary judgment result can be obtained, wherein the second auxiliary judgment result is a judgment result for judging whether the target application has the crawler behavior; inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors, and obtaining a prediction result for indicating whether the target application has crawler behaviors; according to the prediction result, obtaining a determination result of whether the target application has a crawler behavior, including: inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors or not, and obtaining a model judgment result; and obtaining a judging result of whether the target application has the crawler behaviors according to the model judging result and the second auxiliary judging result.
The obtaining the second auxiliary judging result includes: acquiring the distribution characteristics of the distribution resources sent by the target application, wherein the distribution characteristics comprise distribution duration and distribution distance; if the variation range of the distribution time length and the distribution distance is within a preset threshold, judging that the target application does not have the crawler behavior, and taking the target application as a second auxiliary judgment result. It should be noted that, in general, the geographic location of the user is relatively fixed, so after the user places an order, the duration and distance of delivering the delivering resource are relatively stable, if the delivering duration and delivering distance span are recognized to be relatively large, the delivering characteristic after the user places an order conventionally is not met, and the crawler behavior may exist when the user is not normal operation.
In the embodiment of the application, when judging whether the target application has the crawler behavior, a third auxiliary judgment result can be obtained, wherein the third auxiliary judgment result is a judgment result for judging whether the target application has the crawler behavior; inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors, and obtaining a prediction result for indicating whether the target application has crawler behaviors; according to the prediction result, obtaining a determination result of whether the target application has a crawler behavior, including: inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors or not, and obtaining a model judgment result; and obtaining a judging result of whether the target application has the crawler behaviors according to the model judging result and the third auxiliary judging result.
The obtaining the third auxiliary judging result includes: if the number of times of access requests sent by the target application exceeds a preset threshold value within a preset time and the ordering operation corresponding to the target application is not recognized, determining that the target application has crawler behaviors, and taking the crawler behaviors as a third auxiliary determination result. It should be noted that, in general, a user is purposely browsing a merchant, selecting a suitable commodity to make an order, if the user submits a plurality of requests in a short time, and does not perform an order operation, or the user frequently changes search terms, it is explained that a crawler behavior may exist in a normal operation of a user. It should be noted that, through setting up supplementary judgement result, can further assist and judge whether target application has the crawler action, through combining supplementary judgement result with model judgement result, can make the judgement result that whether target application has the crawler action more accurate.
The above is a process of obtaining a result of determining whether the target application has a crawler behavior according to the device sensor data.
Step S203: and if the judgment result of the target application with the crawler behavior is obtained, outputting prompt information of the target application with the crawler behavior.
The step is used for outputting prompt information of the existence of the crawler behavior of the target application if the judgment result of the existence of the crawler behavior of the target application is obtained.
The embodiment of the application provides a method for detecting crawler behaviors, which comprises the following steps: acquiring device sensor data of a target device running a target application, wherein the device sensor data is used for representing spatial characteristics of the target device; obtaining a judging result of whether the target application has a crawler behavior according to the equipment sensor data; and if the judgment result of the target application with the crawler behavior is obtained, outputting prompt information of the target application with the crawler behavior.
According to the method for detecting the crawler behaviors, the judgment result of whether the target application has the crawler behaviors can be obtained according to the equipment sensor data of the target application, and when the target application has the crawler behaviors, prompt information of the target application having the crawler behaviors is output. Therefore, the crawler behavior detection method can accurately judge whether the target application has crawler behaviors or not.
In a preferred embodiment of the present application, the method for detecting a crawler behavior includes inputting the device sensor data into a first recognition model and a second recognition model for predicting whether the target application has a crawler behavior, and obtaining a first predicted value and a second predicted value for indicating whether the target application has a crawler behavior; and further, according to the first predicted value and the second predicted value, a judging result of whether the target application has the crawler behaviors or not is obtained. According to the crawler behavior detection method, the data of the equipment sensor are input into the first recognition model and the second recognition model, so that whether the target application has the crawler behavior can be accurately judged.
Second embodiment
In the first embodiment, a method for detecting a crawler behavior is provided, and correspondingly, a second embodiment of the present application provides a device for detecting a crawler behavior. Since the apparatus embodiment is substantially similar to the first embodiment of the method, the description is relatively simple, and reference is made to the description of the method embodiment for relevant points. The device embodiments described below are merely illustrative.
Fig. 5 is a schematic diagram of a crawler behavior detection device according to a second embodiment of the present application.
The crawler behavior detection apparatus 500 includes: an acquisition unit 501 configured to acquire device sensor data of a target device running a target application, the device sensor data being used to represent a spatial feature of the target device; an obtaining unit 502, configured to obtain a result of determining whether the target application has a crawler behavior according to the device sensor data; and an input unit 503, configured to output prompt information that the target application has the crawler behavior if the determination result that the target application has the crawler behavior is obtained.
Third embodiment
Corresponding to the above method embodiment of the present application, a third embodiment of the present application further provides an electronic device. Fig. 6 is a schematic diagram of an electronic device according to a third embodiment of the present application. The electronic device includes: at least one processor 601, at least one communication interface 602, at least one memory 603 and at least one communication bus 604; alternatively, the communication interface 602 may be an interface of a communication module, such as an interface of a GSM module; the processor 601 may be a processor CPU or an Application specific integrated Circuit ASIC (Application SPECIFIC INTEGRATED Circuit) or one or more integrated circuits configured to implement embodiments of the present application. The memory 603 may comprise high-speed RAM memory or may further comprise non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 603 stores a program, and the processor 601 calls the program stored in the memory 603 to execute the method provided in the above embodiment of the present application.
Fourth embodiment
The fourth embodiment of the present application also provides a computer storage medium corresponding to the above-described method of the present application. The computer storage medium stores a computer program that is executed by a processor to perform the method provided in the above-described embodiments of the present application.
While the application has been described in terms of preferred embodiments, it is not intended to be limiting, but rather, it will be apparent to those skilled in the art that various changes and modifications can be made herein without departing from the spirit and scope of the application as defined by the appended claims.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
1. Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer readable media, as defined herein, does not include non-transitory computer readable media (transmission media), such as modulated data signals and carrier waves.
2. It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It should be noted that, in the embodiment of the present application, the use of user data may be involved, and in practical application, the user specific personal data may be used in the solution described herein within the scope allowed by the applicable legal regulations in the country under the condition of meeting the applicable legal regulations in the country (for example, the user explicitly agrees to the user to notify practically, etc.).
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region, and provide corresponding operation entries for the user to select authorization or rejection.
Claims (10)
1. A method for detecting crawler behavior, comprising:
acquiring device sensor data of a target device running a target application, wherein the device sensor data is used for representing spatial characteristics of the target device;
Obtaining a judging result of whether the target application has a crawler behavior according to the equipment sensor data;
and if the judgment result of the target application with the crawler behavior is obtained, outputting prompt information of the target application with the crawler behavior.
2. The method for detecting the crawler behavior according to claim 1, wherein the obtaining, according to the device sensor data, a result of determining whether the target application has the crawler behavior includes:
Inputting the equipment sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors or not, and obtaining a prediction result for indicating whether the target application has crawler behaviors or not;
And obtaining a judging result of whether the target application has the crawler behaviors according to the predicting result.
3. The method of claim 1, wherein the crawler behavior recognition model comprises a first recognition model and a second recognition model;
Inputting the device sensor data into a crawler behavior recognition model for predicting whether the target application has crawler behaviors, and obtaining a prediction result for indicating whether the target application has crawler behaviors, wherein the method comprises the following steps of:
inputting the equipment sensor data into a first recognition model to obtain a first predicted value used for representing whether the target application has crawler behaviors;
and inputting the device sensor data into a second recognition model to obtain a second predicted value for representing whether the target application has the crawler behaviors.
4. The method for detecting crawler behaviors according to claim 3, wherein the obtaining, according to the prediction result, a determination result of whether the target application has crawler behaviors includes:
and if at least one of the first predicted value and the second predicted value accords with the preset condition of the existence of the crawler behavior, obtaining a judging result of the existence of the crawler behavior of the target application.
5. The method of claim 1, wherein the device sensor data comprises a device sensor data sequence.
6. The method of claim 5, wherein the type of device sensor data sequence comprises at least one of:
four-dimensional data consisting of X-axis data, Y-axis data, Z-axis data and electronic compass data of the target equipment;
Or three-dimensional data consisting of any three of X-axis data, Y-axis data, Z-axis data and electronic compass data of the target equipment;
Or two-dimensional data consisting of any two of X-axis data, Y-axis data, Z-axis data and electronic compass data of the target device.
7. The method of claim 1, wherein the first recognition model is a machine learning model that is trained as follows:
Obtaining sample data of device sensor data of target devices of target applications, wherein the sample data comprises device sensor data samples of target applications with crawler behaviors, device sensor data samples of target applications without crawler behaviors and identification result tags of the target applications, and the identification result tags of the target applications comprise: the target application has an identification tag of a crawler behavior or the target application does not have an identification tag of the crawler behavior;
And training an initial machine learning model based on the sample data to obtain the machine learning model as the first recognition model.
8. A device for detecting crawler behavior, comprising:
An acquisition unit configured to acquire device sensor data of a target device running a target application, the device sensor data being used to represent a spatial feature of the target device;
The obtaining unit is used for obtaining a judging result of whether the target application has the crawler behaviors according to the equipment sensor data;
and the input unit is used for outputting prompt information of the existence of the crawler behavior of the target application if the judgment result of the existence of the crawler behavior of the target application is obtained.
9. An electronic device comprising a processor and a memory;
the memory has stored therein a computer program, which, when executed by the processor, performs the method of any of claims 1-7.
10. A computer storage medium, characterized in that the computer storage medium stores a computer program which, when executed by a processor, performs the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410244771.9A CN118158123A (en) | 2024-03-04 | 2024-03-04 | Method and device for detecting crawler behaviors, electronic equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410244771.9A CN118158123A (en) | 2024-03-04 | 2024-03-04 | Method and device for detecting crawler behaviors, electronic equipment and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118158123A true CN118158123A (en) | 2024-06-07 |
Family
ID=91294005
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410244771.9A Pending CN118158123A (en) | 2024-03-04 | 2024-03-04 | Method and device for detecting crawler behaviors, electronic equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118158123A (en) |
-
2024
- 2024-03-04 CN CN202410244771.9A patent/CN118158123A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106897334A (en) | A kind of question pushing method and equipment | |
| CN111163072B (en) | Method and device for determining characteristic value in machine learning model and electronic equipment | |
| WO2015117560A1 (en) | Web page recognizing method and apparatus | |
| CN109086377B (en) | Equipment portrait generation method and device and computing equipment | |
| CN106604362B (en) | A kind of Wireless Fidelity Wi-Fi scan method and mobile terminal | |
| CN109739700B (en) | Interface testing method and device | |
| US20170277622A1 (en) | Web Page Automated Testing Method and Apparatus | |
| CN108399125B (en) | Automatic testing method and device, computer equipment and storage medium | |
| CN105160246A (en) | Method for identifying hijacked browser and browser | |
| CN111224923A (en) | Detection method, device and system for counterfeit websites | |
| CN113132181B (en) | Method and device for detecting network protocol support degree of IPv6 mobile application program | |
| CN105354490A (en) | Method and device for processing hijacked browser | |
| CN105243134B (en) | A kind of method and apparatus handling browser of being held as a hostage | |
| CN110706035B (en) | Updating effect evaluation method and device, storage medium and electronic equipment | |
| CN110708270B (en) | Abnormal link detection method and device | |
| CN118158123A (en) | Method and device for detecting crawler behaviors, electronic equipment and computer storage medium | |
| CN113626483B (en) | Front-end caching method, system, equipment and storage medium for filling forms | |
| CN115080552A (en) | Data quality evaluation method, device, equipment and computer readable storage medium | |
| CN110059272B (en) | Page feature recognition method and device | |
| CN115081006A (en) | Sensitive data processing method, device and equipment | |
| CN104462140A (en) | Webpage data collecting method and device | |
| CN112417252B (en) | Crawler path determination method and device, storage medium and electronic equipment | |
| CN103177100A (en) | Method and device for searching sets of images | |
| CN105205393A (en) | Method and equipment for processing hijacked browser | |
| CN115510235A (en) | Text classification model training method, text classification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |