CN113709159A - Access data detection method, device, equipment and storage medium - Google Patents
Access data detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113709159A CN113709159A CN202110998200.0A CN202110998200A CN113709159A CN 113709159 A CN113709159 A CN 113709159A CN 202110998200 A CN202110998200 A CN 202110998200A CN 113709159 A CN113709159 A CN 113709159A
- Authority
- CN
- China
- Prior art keywords
- access data
- sequence
- detected
- data detection
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
Abstract
The disclosure relates to an access data detection method, device, equipment and storage medium. The method comprises the following steps: the method comprises the steps of obtaining access data to be detected, wherein the access data to be detected comprises a stay time sequence of a plurality of network addresses, inputting the stay time sequence into a pre-trained access data detection model, obtaining a prediction probability output by the pre-trained access data detection model, wherein the prediction probability is the sum of the probabilities of stay time sequences of at least one network address sequence formed by the plurality of network addresses, and if the prediction probability is smaller than a preset probability threshold, determining that the access data to be detected is abnormal access data. By the technical scheme, the abnormal access data with higher accuracy and robustness can be predicted, so that the detection requirement of complicated and diversified access data detection is met.
Description
Technical Field
The present disclosure relates to the field of network communication and network security technologies, and in particular, to a method, an apparatus, a device, and a storage medium for detecting access data.
Background
With the development of society and the continuous progress of internet technology, the internet brings more and more convenience to users. However, due to the lack of network security awareness of users and the continuous development of network attack technology towards complication and diversification, many network applications suffer from various network attacks and security threats, and a lot of network security vulnerabilities are exposed.
In order to improve the security and availability of the internet, the internet needs to be supervised by a network. Among them, access data detection is an important means for network supervision. However, the existing access data detection method has low detection accuracy, and is difficult to meet the detection requirement of complicated and diversified access data detection.
Disclosure of Invention
To solve the technical problem or at least partially solve the technical problem, the present disclosure provides an access data detection method, apparatus, device and storage medium.
In a first aspect, the present disclosure provides an access data detection method, including:
acquiring access data to be detected, wherein the access data to be detected comprises a retention time sequence of a plurality of network addresses;
inputting the residence time sequence into a pre-trained access data detection model to obtain a prediction probability output by the pre-trained access data detection model, wherein the prediction probability is the sum of the probability that the predicted residence time sequence of at least one network address sequence consisting of a plurality of network addresses is the residence time sequence;
and if the prediction probability is smaller than a preset probability threshold, determining that the access data to be detected is abnormal access data.
In a second aspect, the present disclosure provides an access data detection apparatus, comprising:
the system comprises a to-be-detected access data acquisition module, a to-be-detected access data acquisition module and a to-be-detected access data acquisition module, wherein the to-be-detected access data acquisition module is used for acquiring to-be-detected access data which comprises a retention time sequence of a plurality of network addresses;
the prediction probability determination module is used for inputting the retention time sequence into the pre-trained access data detection model to obtain the prediction probability output by the pre-trained access data detection model, wherein the prediction probability is the sum of the probability of the retention time sequence of the prediction retention time sequence of at least one network address sequence formed by a plurality of network addresses;
and the abnormal access data determining module is used for determining the access data to be detected as abnormal access data if the prediction probability is smaller than a preset probability threshold.
In a third aspect, an embodiment of the present disclosure further provides an access data detection device, where the device includes:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the access data detection method provided by the first aspect.
In a fourth aspect, the disclosed embodiments also provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the access data detection method provided in the first aspect.
Compared with the prior art, the technical scheme provided by the embodiment of the disclosure has the following advantages:
the access data detection method, the device, the equipment and the storage medium of the embodiment can acquire access data to be detected, the access data to be detected comprises a stay time sequence of a plurality of network addresses, the stay time sequence is input into a pre-trained access data detection model to obtain the prediction probability output by the pre-trained access data detection model, the prediction probability is that the prediction stay time sequence of at least one network address sequence consisting of the plurality of network addresses is the probability sum of the stay time sequence, if the prediction probability is smaller than a preset probability threshold value, the access data to be detected is determined to be abnormal access data, therefore, the probability sum of the stay time sequence and the pre-trained access data detection model can be utilized to accurately predict the prediction stay time sequence of at least one network address sequence consisting of the plurality of network addresses as the stay time sequence, and the probability sum is used as a prediction probability to realize the prediction of abnormal access data with higher accuracy and robustness so as to meet the detection requirement of complicated and diversified access data detection.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present disclosure, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic flowchart of a method for detecting access data according to an embodiment of the present disclosure;
fig. 2 is a logic diagram of a method for detecting access data according to an embodiment of the present disclosure;
FIG. 3 is a schematic flow chart diagram illustrating a training method for accessing a data detection model according to an embodiment of the present disclosure;
fig. 4a is a schematic diagram of transition probability between an initial state and a hidden state according to an embodiment of the present disclosure;
FIG. 4b is a schematic diagram illustrating transition probabilities between hidden states according to an embodiment of the disclosure;
FIG. 4c is a schematic diagram illustrating transition probabilities between hidden states and visible states according to an embodiment of the disclosure;
fig. 4d is a diagram of a correspondence between a dwell time sequence and a network address sequence according to an embodiment of the present disclosure;
FIG. 5 is a logic diagram of a training method for accessing a data detection model according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an apparatus for accessing data detection according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an access data detection device according to an embodiment of the present disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, aspects of the present disclosure will be further described below. It should be noted that the embodiments and features of the embodiments of the present disclosure may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced in other ways than those described herein; it is to be understood that the embodiments disclosed in the specification are only a few embodiments of the present disclosure, and not all embodiments.
With the development of society and the continuous progress of internet technology, the internet brings more and more convenience to users. However, due to the lack of network security awareness of users and the continuous development of network attack technology towards complication and diversification, many network applications suffer from various network attacks and security threats, and a lot of network security vulnerabilities are exposed.
According to the newly published '2020 malicious machine traffic report', 37.2% of the traffic in 2019 came from bot traffic, with malicious bot traffic accounting for 24.1%. From the trend, the bot traffic fraction will be larger and larger, and it is expected that even the human behavioral traffic will be exceeded at 2025.
The malicious bot flow can cause the problems of unavailable service of a business website platform, reduced user experience, vulnerability security of a website, business failure and the like, so that enterprise data is crawled, an interface is brushed, the service is unavailable due to challenge of a challenge black hole (CC) attack and the like, and extremely high risk and loss which is difficult to estimate are brought to an enterprise.
In order to improve the security and availability of the internet, the internet needs to be supervised by a network. The access data detection is an important means for network supervision, namely, the detection of malicious traffic has important significance for the network supervision.
In the related art, access data is generally detected by analyzing a browser version, analyzing user behavior, and analyzing a user access time.
However, the existing access data detection method has low detection accuracy, and is difficult to meet the detection requirement of complicated and diversified access data detection.
In order to solve the above problem, embodiments of the present disclosure provide an access data detection method, apparatus, device, and storage medium, which can accurately predict, by using a dwell time sequence and a pre-trained access data detection model, a predicted dwell time sequence of at least one network address sequence composed of a plurality of network addresses as a sum of probabilities of the dwell time sequence, and use the sum of the probabilities as a prediction probability, so as to implement prediction of abnormal access data with higher accuracy and robustness, so as to meet detection requirements of complicated and diversified access data detection.
The following first describes an access data detection method provided by an embodiment of the present disclosure with reference to fig. 1 to 5.
Fig. 1 shows a schematic flowchart of an access data detection method provided by an embodiment of the present disclosure.
In some embodiments of the present disclosure, the access data detection method illustrated in fig. 1 may be performed by a server. The server may be a cloud server or a server cluster or other devices with storage and computing functions.
As shown in fig. 1, the access data detection method may include the following steps.
S110, obtaining access data to be detected, wherein the access data to be detected comprises a stay time sequence of a plurality of network addresses.
Specifically, the server may obtain the access data to be detected as the access data to be detected, so as to detect whether the access data to be detected is abnormal access data.
In this disclosure, the access data to be detected may be access traffic received by the network server.
Optionally, the access data to be detected may include user access data and abnormal access data.
In the disclosed embodiment, the access data to be detected may include a dwell time sequence of a plurality of network addresses.
Wherein the dwell time sequence may comprise dwell times of a plurality of network addresses having a jump relationship. The dwell time may be the time to jump from one network address to another.
And S120, inputting the retention time sequence into the access data detection model trained in advance to obtain the prediction probability output by the access data detection model trained in advance.
In the embodiment of the present disclosure, the access data detection model may be a statistical analysis model, and the statistical analysis model may perform statistics on the historical access data to obtain a rule of the historical access data, and predict the probability that the access data to be detected is abnormal access data based on the rule.
Alternatively, the access data detection model may be embodied as a hidden markov model in a statistical model.
In an embodiment of the disclosure, the predicted probability may be a sum of probabilities that a predicted dwell time sequence of at least one network address sequence consisting of a plurality of network addresses is a dwell time sequence.
Specifically, when the server uses the hidden markov model as the access data detection model to perform probability prediction on the to-be-detected model, the server may input the retention time sequence in the to-be-detected access data to the access data detection model trained in advance, so as to use the retention time sequence as the visible state sequence of the access data detection model, predict, by using the access data detection model, that the predicted retention time sequence of at least one network address sequence composed of a plurality of network addresses in the to-be-detected access data is the sum of probabilities of the retention time sequences, and use the sum of the probabilities as the predicted probability that the to-be-detected access data is abnormal access data.
And S130, if the prediction probability is smaller than a preset probability threshold, determining that the access data to be detected is abnormal access data.
Specifically, after the server obtains the prediction probability, the prediction probability is compared with a preset probability threshold, if the prediction probability is smaller than the preset probability threshold, the data to be detected is determined to be abnormal access data, and otherwise, the data to be detected is determined to be user access data.
In this disclosure, the preset probability threshold may be a maximum probability for determining whether the access data to be detected is abnormal access data.
Optionally, the preset probability threshold may be a probability value set as required, and the probability threshold may be a numerical value such as 80% or 85%, which is not limited herein.
Fig. 2 illustrates a logic diagram of an access data detection method according to an embodiment of the present disclosure.
As shown in fig. 2, the access data detection method may include:
s210, acquiring data to be detected.
In the disclosed embodiment, the access data to be detected may include a dwell time sequence of a plurality of network addresses.
And S220, judging whether the data to be detected is abnormal access data, if so, executing S230, and otherwise, executing S240.
In the embodiment of the disclosure, the server may input the retention time sequence in the data to be detected to the pre-trained access data detection model to obtain the prediction probability output by the pre-trained access data detection model, and if the prediction probability is smaller than a preset probability threshold, it is determined that the access data to be detected is abnormal access data, otherwise, the access data to be detected is user access data.
And S230, releasing the access data to be detected.
Specifically, if the server determines that the access data to be detected is the user access data, the access data to be detected is released, so that the message corresponding to the access data to be detected passes through, and the subsequent access data to be detected is continuously detected.
And S240, intercepting the access data to be detected.
Specifically, if the server determines that the access data to be detected is abnormal access data, the server intercepts the access data to be detected, so that messages corresponding to the access data to be detected are intercepted, abnormal detection information is generated, statistics is carried out on the access data to be detected, and subsequent detection is continuously carried out on the access data to be detected.
In the embodiment of the disclosure, the access data to be detected can be obtained, the access data to be detected includes a dwell time sequence of a plurality of network addresses, the dwell time sequence is input into a pre-trained access data detection model, a prediction probability output by the pre-trained access data detection model is obtained, the prediction probability is a sum of probabilities of the dwell time sequence being a prediction dwell time sequence of at least one network address sequence composed of the plurality of network addresses, and if the prediction probability is smaller than a preset probability threshold, the access data to be detected is determined to be abnormal access data, therefore, the sum of probabilities of the dwell time sequence being the prediction dwell time sequence of the at least one network address sequence composed of the plurality of network addresses can be accurately predicted by using the dwell time sequence and the pre-trained access data detection model, and the sum of the probabilities is used as the prediction probability, the method and the device can realize the prediction of abnormal access data with higher accuracy and robustness so as to meet the detection requirement of complicated and diversified access data detection.
In another embodiment of the present disclosure, in order to further improve the accuracy of detecting the access data to be detected, the access data to be detected further includes a network address sequence to be detected, which is composed of a plurality of network addresses, and the network address sequence is used as a hidden state sequence of a pre-trained access data detection model, and the dwell time sequence is used as a visible state sequence of the pre-trained access data detection model, so that the predicted dwell time sequence of at least one network address sequence composed of a plurality of network addresses in the access data to be detected is predicted as a sum of probabilities of the dwell time sequences by using the pre-trained access data detection model, and the sum of the probabilities is used as a predicted probability that the access data to be detected is abnormal access data.
In some embodiments of the present disclosure, the access data to be detected further includes a network address sequence to be detected, where each two adjacent network addresses in the network address sequence to be detected have a jump relationship.
Wherein S120 may include:
and inputting the residence time sequence and the network address sequence into a pre-trained access data detection model to obtain the prediction probability output by the pre-trained access data detection model, wherein the prediction probability is the probability that the predicted residence time of the network address sequence is the residence time.
The network address sequence may be a sequence of network addresses corresponding to each dwell time in the dwell time sequence.
In some embodiments, the server may extract a header field of a request message of the to-be-detected access data to obtain a network address, and form a network address sequence from the extracted multiple network addresses having a skip relationship.
In other embodiments, the server may extract a header field of the response message of the to-be-detected access data to obtain a network address, and form a network address sequence from the extracted multiple network addresses having a skip relationship.
In still other embodiments, the server may extract a header field in the reference source information of the to-be-detected access data to obtain a network address, and form a network address sequence from the extracted multiple network addresses having a jump relationship.
Alternatively, the access data detection model may be embodied as a hidden markov model in a statistical model.
Specifically, when the server uses the hidden markov model as the access data detection model to perform probability prediction on the model to be detected, the server may input the retention time sequence and the network address sequence in the access data to be detected into the access data detection model trained in advance, so as to use the retention time sequence as the visible state sequence of the access data detection model, use the network address sequence as the hidden state sequence of the access data detection model trained in advance, predict the probability that the predicted retention time of the network address sequence is the retention time by using the access data detection model, and use the predicted probability as the prediction probability that the access data to be detected is abnormal access data.
Therefore, in the embodiment of the present disclosure, the dwell time sequence in the access data to be detected may be used as the visible state sequence of the access data detection model trained in advance, the network address sequence may be used as the hidden state sequence of the access data detection model trained in advance, since the hidden state sequence of the pre-trained access data detection model is known, based on the dwell time sequence and the network address sequence, and using the pre-trained access data detection model, the method can more accurately determine the prediction probability of the access data to be detected as abnormal access data so as to realize the prediction of the abnormal access data with higher accuracy and robustness and meet the detection requirement of complicated and diversified access data detection, therefore, the cost of abnormal access data access is increased, and the abnormal access data can be effectively intercepted.
In some embodiments of the present disclosure, in order to further improve the detection accuracy of the access data detection model, after the prediction probability output by the access data detection model trained in advance is obtained, if the access data to be detected is user access data, the access data detection model trained in advance may be updated by using the dwell time sequence and the network address sequence to be detected in the access data to be detected, so that the access data detection model is continuously updated by using new data, thereby improving the accuracy and robustness of the access data detection model.
In this embodiment of the present disclosure, after S130, the method may further include:
if the first prediction probability is greater than or equal to a preset probability threshold, determining that the access data to be detected is user access data;
and updating the pre-trained access data detection model by using the retention time sequence and the network address sequence to be detected.
Specifically, after the server obtains the prediction probability, the prediction probability is compared with a preset probability threshold, if the prediction probability is greater than or equal to the preset probability threshold, the access data to be detected is determined to be user access data, the stay time sequence is used as a visible state sequence of a pre-trained access data detection model, the network address sequence to be detected is used as a hidden state sequence of the pre-trained access data detection model, and the pre-trained access data detection model is subjected to updating training to obtain an updated access data detection model.
Therefore, in the embodiment of the disclosure, after it is detected that the access data to be detected is the user access data, the access data detection model trained in advance can be updated by using the dwell time sequence and the network address sequence to be detected, so that the access data detection model is continuously updated by using new data, and the accuracy and robustness of the access data detection model are improved.
In another embodiment of the present disclosure, to ensure that the access data detection can be implemented by using the access data detection model, the server may further perform a model training step on the access data detection model before performing S110.
Fig. 3 shows a flowchart of a training method for accessing a data detection model according to an embodiment of the present disclosure.
As shown in fig. 3, the training method of the visit data detection model may further include the following steps before acquiring the visit data to be detected.
S310, obtaining sample access data of a plurality of network addresses accessed by historical users.
Specifically, the server may obtain access data of a plurality of network addresses accessed by a historical user as sample access data, so as to train an access data detection model by using the sample access data.
In the embodiment of the present disclosure, the history user may be a collected user who normally accesses a network address.
In the embodiment of the disclosure, the sample access data includes a sample network address sequence composed of a plurality of network addresses and a sample dwell time sequence, each two adjacent network addresses in the sample network address sequence have a jump relationship, and the sample dwell time sequence includes a dwell time of each network address when each two adjacent network addresses jump.
Wherein each network address in the sample network address sequence sum and each dwell time in the sample dwell time sequence have a correspondence.
S320, training the preset network based on the sample access data to obtain an access data detection model.
Optionally, the preset network may be a statistical analysis model, and the statistical analysis model may perform statistics on the historical access data to obtain a rule of the historical access data, and predict the probability that the access data to be detected is abnormal access data based on the rule.
It is understood that when the user normally visits the website, the state transition of the access data is relevant, and the next network address is connected from the currently accessed network address, so that the conversion of the network address can be regarded as a first-order markov process. In addition, when a user normally accesses a network address, jump between adjacent network addresses exists, when a malicious machine accesses the network address, the jump of the network address does not occur, and the time interval of the user normally accessing the network address is different from the time interval of the malicious machine accessing the network address.
For example, the malicious machines of the ticket swiping class frequently access the same network address, and the jump of the network address does not occur, and the normal user does not have the behavior, so that whether the jump exists between the network addresses and the stay time of the access network address can be utilized to predict whether the access data to be detected is the access data of the normal user or the access data of the malicious machine.
Alternatively, the accessed data detection model may be a hidden markov model in a statistical analysis model.
It will be appreciated that the sequence of observable dwell times and the sequence of hidden network addresses are probabilistically related when a user accesses a network address, and thus the process of accessing a network address can be modeled as a hidden markov process and a set of states probabilistically related and observable to this hidden markov process, resulting in a hidden markov model.
The hidden markov model may include five key components, namely, a hidden state set, a visible state set, a probability of an initial state to a hidden state, a probability of a transition between hidden states, and a probability of a transition between a hidden state to a visible state.
Fig. 4a shows a schematic diagram of a transition probability between an initial state and a hidden state provided by an embodiment of the present disclosure, fig. 4b shows a schematic diagram of a transition probability between hidden states provided by an embodiment of the present disclosure, fig. 4c shows a schematic diagram of a transition probability between a hidden state and a visible state provided by an embodiment of the present disclosure, and fig. 4d shows a corresponding relationship diagram of a dwell time sequence and a network address sequence provided by an embodiment of the present disclosure. Wherein the dwell time of the network address URL1 is 0-1s, the dwell time of the network address URL3 is 2-3s, and the dwell time of the network address URL2 is 4-5 s.
The hidden markov model training process calculates the probability of the initial state to the hidden state, the probability of the transition between the hidden states, and the probability of the transition between the hidden states to the visible state.
In the embodiment of the present disclosure, S320 may include:
taking the sample network address sequence as a hidden state sequence of a preset network;
taking the sample residence time sequence as a visible state sequence of a preset network;
and iteratively training the preset network by using the hidden state sequence and the visible state sequence until the currently trained preset network meets a preset iteration condition, and obtaining a trained access data detection model.
In some embodiments, the server may extract a header field in request information of sample access data of a plurality of network addresses accessed by a historical user to obtain the network addresses, and form a sample network address sequence by the plurality of extracted network addresses with a jump relation.
In other embodiments, the server may extract a header field in response information of sample access data of a plurality of network addresses accessed by a historical user to obtain the network addresses, and form a sample network address sequence by the plurality of extracted network addresses with a jump relationship.
In still other embodiments, the server may extract a header field in reference source information of sample access data of a plurality of network addresses accessed by a historical user to obtain the network addresses, and form a sample network address sequence by the plurality of extracted network addresses with a jump relation.
In the embodiment of the disclosure, the server may perform time sequencing and equally dividing on the residence time corresponding to the plurality of network addresses accessed by the historical user, and take the head and tail time of the equally divided single group of data as a visible state, where if the shortest time of the first group of data is 0.1ms and the longest time is 0.5ms, the visible state sequence is 0.1ms-0.5 ms; and if the shortest time of the last group of data is 30s and the longest time is 3600s, the visible state sequence is 30s-3600s, in order to include all the time, the visible state sequence corresponding to the first group of data can be changed into 0-0.5ms, and the visible state sequence corresponding to the last group of data can be changed into 30 s-max, and the visible state sequence can be specifically determined according to the actual timeout time, so that the sample residence time sequence can be obtained.
Specifically, after the server acquires a sample network address sequence and a sample dwell time sequence composed of a plurality of network addresses in the sample access data, taking the sample network address sequence as a hidden state sequence of the preset network, taking the sample retention time sequence as a visible state sequence of the preset network, iteratively training the preset network by utilizing the hidden state sequence and the visible state sequence, and calculating the probability from the initial state to the hidden state, the probability of the conversion between the hidden states and the visible state by using a maximum likelihood estimation method until the probability from the initial state to the hidden state, the probability of the conversion between the hidden states and the visible state are stable, determining that the preset network trained at present meets the preset iteration condition, and obtaining the trained access data detection model.
In the disclosed embodiment, the trained visited data detection model may be u ═ (a, B, pi), a ═ aijIs the probability of a transition between hidden states, B ═ Bj(k) Is hiddenProbability of transition between hidden state to visible state, pi ═ pi { pi }iThe probability of the initial state to the hidden state.
Alternatively, the hidden state sequence may be Q ═ s1s2...snThe visible state sequence may be O ═ v1v2...vmWhere δ is an impact function (when two parameters of the function are the same, the function value is 1), each parameter value can be calculated by the following formula (δ (q)t,si) Indicating that the hidden state is si at time t).
Optionally, ni＝δ(q1,si)
Wherein s isi、sj、qi、qjAnd q istMay be a network address, vkMay be the dwell time corresponding to the network address.
It should be noted that the trained access data detection model may be a hidden markov model, and the trained access data detection model is used to obtain a sum of probabilities that a predicted residence time sequence of at least one network address sequence composed of a plurality of network addresses is a residence time sequence, or after obtaining the probability that the predicted residence time of the network address sequence is the residence time, the sum of the probabilities or the probabilities of the residence time sequence may be used as a prediction probability, and whether the access data to be detected is abnormal access data may be determined by using the prediction probability and a preset probability threshold.
Therefore, in the embodiment of the disclosure, sample access data of a plurality of network addresses accessed by a historical user can be obtained, the sample access data includes a sample network address sequence and a sample staying time sequence, the sample network address sequence is composed of a plurality of network addresses, each two adjacent network addresses in the sample network address sequence have a jump relation, the sample staying time sequence includes a staying time of each network address when each two adjacent network addresses jump, the sample network address sequence is used as a hidden state sequence of a preset network, the sample staying time sequence is used as a visible state sequence of the preset network, the preset network is iteratively trained by using the hidden state sequence and the visible state sequence until the preset network currently trained meets a preset iteration condition, a trained access data detection model is obtained, since the hidden state sequence can be known in the training process, the visible state sequence can also be known, so that when the access data detection model is trained by using the hidden state sequence and the visible state sequence, the access data detection model can be a perfect corpus model, the training difficulty and cost of the access data detection model are reduced, and the detection accuracy of the access data detection model is improved.
In order to further improve the accuracy and robustness of the trained access data detection model, after S320 is executed, a model verification step for the access data detection model may also be executed.
In some embodiments of the present disclosure, after S320, the training method for accessing a data detection model further includes:
acquiring verification access data, wherein the verification access data comprises user access data and abnormal access data, the user access data comprises a verification network address sequence and a verification stay time sequence, the verification network address sequence is composed of a plurality of network addresses, each two adjacent network addresses in the verification network address sequence have a jump relation, and the verification stay time sequence comprises the stay time of each network address when each two adjacent network addresses jump;
detecting the accuracy of the trained access data detection model by using the verification access data;
and under the condition that the accuracy is smaller than the accuracy threshold, returning to execute the training of the preset network based on the sample access data until the accuracy is larger than or equal to the accuracy threshold, and obtaining a trained access data detection model.
In embodiments of the present disclosure, the validation access data may be access data used to validate the robustness and accuracy of a trained access data detection model.
In embodiments of the present disclosure, the accuracy threshold may be a minimum accuracy for verifying whether the trained access data detection model passes verification.
Alternatively, the accuracy threshold may be 90%, 95%, etc. of data, and is not limited herein.
In an embodiment of the present disclosure, detecting the accuracy of the trained access data detection model by using the verification access data includes:
inputting the verification access data into a trained access data detection model to obtain a prediction probability;
if the prediction probability is smaller than a preset probability threshold, determining that the access data to be detected is abnormal access data;
if the prediction probability is greater than or equal to a preset probability threshold, determining the access data to be detected as user access data;
and calculating the accuracy according to the abnormal access data and the user access data which are obtained through prediction, and the user access data and the abnormal access data in the verification access data.
Specifically, after the server obtains the trained access data detection model, verification access data can be obtained, the verification access data can comprise user access data and abnormal access data, the user access data comprises a verification network address sequence and a verification stay time sequence, the verification network address sequence is composed of a plurality of network addresses, a jump relation exists between every two adjacent network addresses in the verification network address sequence, the verification stay time sequence comprises the stay time of each network address when every two adjacent network addresses jump, the verification network address sequence is used as a hidden state sequence of the trained access data detection model, the verification stay time sequence is used as a visible state sequence of the trained access data detection model, and the abnormal access data is input into the trained access data detection model so as to detect the accuracy of the trained access data detection model by utilizing the verification access data, if the accuracy is smaller than the accuracy threshold, the trained access data detection model is poor in accuracy and robustness, the preset network is returned to be trained based on the sample access data until the accuracy is larger than or equal to the accuracy threshold, the trained access data detection model is obtained, if the accuracy is larger than or equal to the accuracy threshold, the trained access data detection model is good in accuracy and robustness, the trained access data detection model is output, the trained access data detection model is used for detecting the access data to be detected by utilizing the trained access data detection model, and whether the access data to be detected is abnormal access data is determined.
Therefore, in the embodiment of the disclosure, after the trained access data detection model is obtained, the accuracy of the trained access data detection model can be detected by using the verification access data, and under the condition that the accuracy is smaller than the accuracy threshold, the training of the preset network based on the sample access data is returned to be executed until the accuracy is greater than or equal to the accuracy threshold, so that the trained access data detection model is obtained, and the accuracy and the robustness of the trained access data detection model are further improved.
Fig. 5 is a logic diagram illustrating a training method for accessing a data detection model according to an embodiment of the present disclosure.
As shown in fig. 5, the training method of the visit data detection model may further include the following steps before acquiring the visit data to be detected.
And S510, obtaining sample access data of a plurality of network addresses accessed by historical users.
In the embodiment of the disclosure, the sample access data may include a sample network address sequence composed of a plurality of network addresses and a sample dwell time sequence, each two adjacent network addresses in the sample network address sequence have a jump relationship, and the sample dwell time sequence includes a dwell time of each network address when each two adjacent network addresses jump.
S520, training the preset network based on the sample access data to obtain an access data detection model.
In the embodiment of the present disclosure, the predetermined network may be a statistical analysis model;
wherein S520 may include:
taking the sample network address sequence as a hidden state sequence of a preset network;
taking the sample residence time sequence as a visible state sequence of a preset network;
and iteratively training the preset network by using the hidden state sequence and the visible state sequence until the currently trained preset network meets a preset iteration condition, and obtaining a trained access data detection model.
And S530, acquiring verification access data.
In the embodiment of the disclosure, the verification access data may include user access data and abnormal access data, the user access data includes a verification network address sequence composed of a plurality of network addresses and a verification dwell time sequence, each two adjacent network addresses in the verification network address sequence have a jump relationship, and the verification dwell time sequence includes a dwell time of each network address when each two adjacent network addresses jump.
And S540, detecting the accuracy of the trained access data detection model by using the verification access data.
And S550, judging whether the accuracy is smaller than an accuracy threshold, if so, returning to execute S520, and if not, ending.
The embodiment of the present disclosure further provides an access data detection apparatus for implementing the above access data detection method, which is described below with reference to fig. 6. In the disclosed embodiment, the access data detection device may be a server. The server may be a cloud server or a server cluster or other devices with storage and computing functions.
Fig. 6 shows a schematic structural diagram of an access data detection apparatus according to an embodiment of the present disclosure.
As shown in fig. 6, the access data detecting apparatus 600 may include: a to-be-detected access data acquisition module 610, a prediction probability determination module 620 and an abnormal access data determination module 630.
The to-be-detected access data acquiring module 610 may be configured to acquire to-be-detected access data, where the to-be-detected access data includes a retention time sequence of multiple network addresses;
the prediction probability determination module 620 may be configured to input the retention time sequence into the pre-trained access data detection model to obtain a prediction probability output by the pre-trained access data detection model, where the prediction probability is a sum of probabilities of a retention time sequence of a prediction retention time sequence of at least one network address sequence formed by a plurality of network addresses;
the abnormal access data determining module 630 may be configured to determine that the access data to be detected is abnormal access data if the prediction probability is smaller than a preset probability threshold.
In the embodiment of the disclosure, the access data to be detected can be obtained, the access data to be detected includes a dwell time sequence of a plurality of network addresses, the dwell time sequence is input into a pre-trained access data detection model, a prediction probability output by the pre-trained access data detection model is obtained, the prediction probability is a sum of probabilities of the dwell time sequence being a prediction dwell time sequence of at least one network address sequence composed of the plurality of network addresses, and if the prediction probability is smaller than a preset probability threshold, the access data to be detected is determined to be abnormal access data, therefore, the sum of probabilities of the dwell time sequence being the prediction dwell time sequence of the at least one network address sequence composed of the plurality of network addresses can be accurately predicted by using the dwell time sequence and the pre-trained access data detection model, and the sum of the probabilities is used as the prediction probability, the method and the device can realize the prediction of abnormal access data with higher accuracy and robustness so as to meet the detection requirement of complicated and diversified access data detection.
Optionally, the to-be-detected access data further includes a to-be-detected network address sequence formed by a plurality of network addresses, and each two adjacent network addresses in the to-be-detected network address sequence have a jump relationship.
Optionally, the prediction probability determining module 620 may be further configured to input the dwell time sequence and the network address sequence into the pre-trained access data detection model to obtain a prediction probability output by the pre-trained access data detection model, where the prediction probability is a probability that the predicted dwell time of the network address sequence is the dwell time.
Optionally, the apparatus further comprises: the system comprises a sample access data acquisition module and a model training module;
the sample access data acquisition module can be used for acquiring sample access data of a plurality of network addresses accessed by a historical user, the sample access data comprises a sample network address sequence and a sample dwell time sequence, the sample network address sequence is composed of a plurality of network addresses, each two adjacent network addresses in the sample network address sequence have a jump relation, and the sample dwell time sequence comprises the dwell time of each network address when each two adjacent network addresses jump;
the model training module can be used for training a preset network based on sample access data to obtain an access data detection model.
Optionally, the preset network is a statistical analysis model. The model training module can be also used for taking the sample network address sequence as a hidden state sequence of a preset network;
taking the sample residence time sequence as a visible state sequence of a preset network;
and iteratively training the preset network by using the hidden state sequence and the visible state sequence until the currently trained preset network meets a preset iteration condition, and obtaining a trained access data detection model.
Optionally, the apparatus may further include: the system comprises a verification access data acquisition module, a model accuracy detection module and an iterative training module;
the verification access data acquisition module can be used for acquiring verification access data, the verification access data comprises user access data and abnormal access data, the user access data comprises a verification network address sequence and a verification stay time sequence, the verification network address sequence is composed of a plurality of network addresses, a jump relation exists between every two adjacent network addresses in the verification network address sequence, and the verification stay time sequence comprises the stay time of each network address when every two adjacent network addresses jump;
the model accuracy detection module can be used for detecting the accuracy of the trained access data detection model by using the verification access data;
the iterative training module may be configured to, when the accuracy is less than the accuracy threshold, return to perform training on the preset network based on the sample access data until the accuracy is greater than or equal to the accuracy threshold, and obtain a trained access data detection model.
Optionally, the apparatus may further include: the user access data determining module and the model updating module;
the user access data determining module may be configured to determine that the access data to be detected is the user access data if the first prediction probability is greater than or equal to a preset probability threshold;
the model updating module can be used for updating the access data detection model which is trained in advance by using the retention time sequence and the network address sequence to be detected.
It should be noted that the access data detection apparatus 600 shown in fig. 6 may perform each step in the method embodiments shown in fig. 1 to fig. 5, and implement each process and effect in the method embodiments shown in fig. 1 to fig. 5, which are not described herein again.
Fig. 7 shows a schematic structural diagram of an access data detection device provided in an embodiment of the present disclosure. In an embodiment of the present disclosure, the access data detection device may be a server. The server may be a cloud server or a server cluster or other devices with storage and computing functions.
As shown in fig. 7, the access data detection device may include a processor 701 and a memory 702 storing computer program instructions.
Specifically, the processor 701 may include a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or may be configured to implement one or more Integrated circuits of the embodiments of the present Application.
The processor 701 performs the steps of the access data detection method provided by the embodiments of the present disclosure by reading and executing the computer program instructions stored in the memory 702.
In one example, the access data detection device may also include a transceiver 703 and a bus 704. As shown in fig. 7, the processor 701, the memory 702, and the transceiver 703 are connected via a bus 704 to complete communication therebetween.
The following is an embodiment of a computer-readable storage medium provided in an embodiment of the present disclosure, the computer-readable storage medium and the access data detection method in the foregoing embodiments belong to the same inventive concept, and details that are not described in detail in the embodiment of the computer-readable storage medium may refer to the embodiment of the access data detection method.
The present embodiments provide a storage medium containing computer-executable instructions which, when executed by a computer processor, are operable to perform a method of access data detection, the method comprising:
acquiring access data to be detected, wherein the access data to be detected comprises a retention time sequence of a plurality of network addresses;
inputting the residence time sequence into a pre-trained access data detection model to obtain a prediction probability output by the pre-trained access data detection model, wherein the prediction probability is the sum of the probability that the predicted residence time sequence of at least one network address sequence consisting of a plurality of network addresses is the residence time sequence;
and if the prediction probability is smaller than a preset probability threshold, determining that the access data to be detected is abnormal access data.
Of course, the storage medium provided by the embodiments of the present disclosure contains computer-executable instructions, and the computer-executable instructions are not limited to the above method operations, and may also perform related operations in the access data detection method provided by any embodiments of the present disclosure.
From the above description of the embodiments, it is obvious for a person skilled in the art that the present disclosure can be implemented by software and necessary general hardware, and certainly can be implemented by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present disclosure may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk, or an optical disk of a computer, and includes several instructions to enable a computer cloud platform (which may be a personal computer, a server, or a network cloud platform, etc.) to execute the Access data detection method provided in the embodiments of the present disclosure.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present disclosure and the technical principles employed. Those skilled in the art will appreciate that the present disclosure is not limited to the particular embodiments described herein, and that various obvious changes, adaptations, and substitutions are possible, without departing from the scope of the present disclosure. Therefore, although the present disclosure has been described in greater detail with reference to the above embodiments, the present disclosure is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present disclosure, the scope of which is determined by the scope of the appended claims.
Claims (10)
1. An access data detection method, comprising:
acquiring to-be-detected access data, wherein the to-be-detected access data comprises a retention time sequence of a plurality of network addresses;
inputting the stay time sequence into a pre-trained access data detection model to obtain a prediction probability output by the pre-trained access data detection model, wherein the prediction probability is the sum of the probability that the predicted stay time sequence of at least one network address sequence consisting of the plurality of network addresses is the stay time sequence;
and if the prediction probability is smaller than a preset probability threshold, determining that the access data to be detected is abnormal access data.
2. The method according to claim 1, wherein the access data to be detected further includes a network address sequence to be detected composed of the plurality of network addresses, and each two adjacent network addresses in the network address sequence to be detected have a jump relationship therebetween.
3. The method of claim 2, wherein inputting the dwell time sequence into a pre-trained visit data detection model to obtain a predicted probability output by the pre-trained visit data detection model comprises:
and inputting the residence time sequence and the network address sequence into the pre-trained access data detection model to obtain a prediction probability output by the pre-trained access data detection model, wherein the prediction probability is the probability that the predicted residence time of the network address sequence is the residence time.
4. The method of claim 2, wherein prior to said obtaining access data to be detected, the method further comprises:
obtaining sample access data of a plurality of network addresses accessed by a historical user, wherein the sample access data comprises a sample network address sequence and a sample staying time sequence, the sample network address sequence is composed of the plurality of network addresses, each two adjacent network addresses in the sample network address sequence have a jump relation, and the sample staying time sequence comprises the staying time of each network address when each two adjacent network addresses jump;
and training a preset network based on the sample access data to obtain the access data detection model.
5. The method of claim 4, wherein the predetermined network is a statistical analysis model;
training a preset network based on the sample access data to obtain the access data detection model, wherein the training comprises:
taking the sample network address sequence as a hidden state sequence of the preset network;
taking the sample residence time sequence as a visible state sequence of the preset network;
and iteratively training the preset network by using the hidden state sequence and the visible state sequence until the preset network which is currently trained meets a preset iteration condition, and obtaining the trained access data detection model.
6. The method of claim 4, wherein after the obtaining the trained visit data detection model, the method further comprises:
obtaining verification access data, wherein the verification access data comprises user access data and abnormal access data, the user access data comprises a verification network address sequence and a verification stay time sequence, the verification network address sequence is composed of a plurality of network addresses, each two adjacent network addresses in the verification network address sequence have a jump relation, and the verification stay time sequence comprises the stay time of each network address when each two adjacent network addresses jump;
detecting the accuracy of the trained access data detection model by using the verification access data;
and returning to execute training of a preset network based on the sample access data under the condition that the accuracy is smaller than an accuracy threshold value until the accuracy is larger than or equal to the accuracy threshold value, and obtaining the trained access data detection model.
7. The method of claim 2, wherein after obtaining the predicted probability of the pre-trained visit data detection model output, the method further comprises:
if the first prediction probability is greater than or equal to the preset probability threshold, determining the to-be-detected access data as user access data;
and updating the pre-trained access data detection model by using the retention time sequence and the network address sequence to be detected.
8. An access data detection apparatus, comprising:
the system comprises a to-be-detected access data acquisition module, a to-be-detected access data acquisition module and a to-be-detected access data acquisition module, wherein the to-be-detected access data acquisition module is used for acquiring to-be-detected access data which comprises a retention time sequence of a plurality of network addresses;
the prediction probability determination module is used for inputting the stay time sequence into a pre-trained access data detection model to obtain the prediction probability output by the pre-trained access data detection model, wherein the prediction probability is the sum of the probabilities of the stay time sequence and the predicted stay time sequence of at least one network address sequence formed by the plurality of network addresses;
and the abnormal access data determining module is used for determining the access data to be detected as abnormal access data if the prediction probability is smaller than a preset probability threshold.
9. An access data detection device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of accessing data as recited in any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method for access data detection according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110998200.0A CN113709159B (en) | 2021-08-27 | 2021-08-27 | Access data detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110998200.0A CN113709159B (en) | 2021-08-27 | 2021-08-27 | Access data detection method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113709159A true CN113709159A (en) | 2021-11-26 |
| CN113709159B CN113709159B (en) | 2023-05-05 |
Family
ID=78656250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110998200.0A Active CN113709159B (en) | 2021-08-27 | 2021-08-27 | Access data detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113709159B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107465651A (en) * | 2016-06-06 | 2017-12-12 | 腾讯科技（深圳）有限公司 | Network attack detecting method and device |
| CN107679626A (en) * | 2017-10-10 | 2018-02-09 | 上海优刻得信息科技有限公司 | Machine learning method, device, system, storage medium and equipment |
| CN109981533A (en) * | 2017-12-27 | 2019-07-05 | 中移（杭州）信息技术有限公司 | A kind of ddos attack detection method, device, electronic equipment and storage medium |
| CN111444931A (en) * | 2019-01-17 | 2020-07-24 | 北京京东尚科信息技术有限公司 | Method and device for detecting abnormal access data |
| CN111476610A (en) * | 2020-04-16 | 2020-07-31 | 腾讯科技（深圳）有限公司 | Information detection method and device and computer readable storage medium |
-
2021
- 2021-08-27 CN CN202110998200.0A patent/CN113709159B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107465651A (en) * | 2016-06-06 | 2017-12-12 | 腾讯科技（深圳）有限公司 | Network attack detecting method and device |
| CN107679626A (en) * | 2017-10-10 | 2018-02-09 | 上海优刻得信息科技有限公司 | Machine learning method, device, system, storage medium and equipment |
| CN109981533A (en) * | 2017-12-27 | 2019-07-05 | 中移（杭州）信息技术有限公司 | A kind of ddos attack detection method, device, electronic equipment and storage medium |
| CN111444931A (en) * | 2019-01-17 | 2020-07-24 | 北京京东尚科信息技术有限公司 | Method and device for detecting abnormal access data |
| CN111476610A (en) * | 2020-04-16 | 2020-07-31 | 腾讯科技（深圳）有限公司 | Information detection method and device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113709159B (en) | 2023-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105590055B (en) | Method and device for identifying user credible behaviors in network interaction system | |
| CN104836781B (en) | Distinguish the method and device for accessing user identity | |
| CN108650260B (en) | Malicious website identification method and device | |
| CN109327439B (en) | Risk identification method and device for service request data, storage medium and equipment | |
| CN112087452B (en) | Abnormal behavior detection method and device, electronic equipment and computer storage medium | |
| EP3750275B1 (en) | Method and apparatus for identity authentication, server and computer readable medium | |
| CN104956372A (en) | Determining coverage of dynamic security scans using runtime and static code analyses | |
| CN107426136B (en) | Network attack identification method and device | |
| CN114553523A (en) | Attack detection method and device based on attack detection model, medium and equipment | |
| CN111968625A (en) | Sensitive audio recognition model training method and recognition method fusing text information | |
| CN111641621A (en) | Internet of things security event identification method and device and computer equipment | |
| US20140230054A1 (en) | System and method for estimating typicality of names and textual data | |
| CN111641619B (en) | Method and device for constructing hacker portrait based on big data and computer equipment | |
| CN106998336B (en) | Method and device for detecting user in channel | |
| WO2022042194A1 (en) | Block detection method and apparatus for login device, server, and storage medium | |
| CN108234454B (en) | Identity authentication method, server and client device | |
| CN110958244A (en) | Method and device for detecting counterfeit domain name based on deep learning | |
| CN113709159B (en) | Access data detection method, device, equipment and storage medium | |
| CN116962009A (en) | Network attack detection method and device | |
| CN110704614A (en) | Information processing method and device for predicting user group type in application | |
| CN115455386A (en) | Operation behavior identification method and device | |
| CN112464218B (en) | Model training method and device, electronic equipment and storage medium | |
| CN115314236A (en) | System and method for detecting phishing domains in a Domain Name System (DNS) record set | |
| CN109214212B (en) | Information leakage prevention method and device | |
| CN114398094A (en) | Applet starting method, device, equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |