CN108200032A - A kind of data detection method, device and electronic equipment - Google Patents
A kind of data detection method, device and electronic equipment Download PDFInfo
- Publication number
- CN108200032A CN108200032A CN201711446332.2A CN201711446332A CN108200032A CN 108200032 A CN108200032 A CN 108200032A CN 201711446332 A CN201711446332 A CN 201711446332A CN 108200032 A CN108200032 A CN 108200032A
- Authority
- CN
- China
- Prior art keywords
- data
- detected
- business
- data flow
- identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
An embodiment of the present invention provides a kind of data detection method, device and electronic equipment, the method includes：Using sample business data flow as the input of initial machine learning model, initial machine learning model is trained, obtains trained machine learning model, and using trained machine learning model as Data Detection model；Using business data flow to be detected as the input of Data Detection model, obtain the Data Identification that Data Detection model is determined according to business data flow to be detected, according to Data Identification, business data flow to be detected is detected, determine in business data flow to be detected with the corresponding abnormal interaction data of Data Identification.By applying Data Detection model, the formation efficiency of the Data Identification of abnormal interaction data can be improved, so that the generation of Data Identification no longer significantly lags behind business data flow to be detected, so as to be detected in time to business data flow to be detected using the Data Identification.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of data detection method, device and electronic equipment.
Background technology
With the high speed development of computer technology and Internet technology, user can be by each Terminal Type, and e.g., hand is mechanical, electrical
The terminal devices such as brain, smart television, interact with server-side, so as to fulfill more and more functions, for example, net purchase, on line
Ballot, social interaction etc..Each terminal interacts realized function with server-side, is construed as server-side offer
A kind of data service.In each data service, the interaction data between a large amount of terminal and server-side, the interaction can be all generated
Data are real-time and constantly generate, so as to form business data flow.For example, a user by terminal in online purchase admission ticket,
It can then occur in the data service such as, terminal sends logging request, and server-side is verified, terminal selection different type or valency
Each generic operation, each generic operation such as the admission ticket of lattice, payment can generate corresponding interaction data；Many ends are had for server-side
End carries out aforesaid operations, so as to constantly generate interaction data, and forms business data flow.
In the interactive process of all kinds of business, it is possible to which there are various abnormal Operational Visits or data interactions.For example,
In above example, someone can make a big purchase admission ticket in large quantities by Malware, carry out admission ticket and hoard, to seek illegal profit.Institute
To be directed to this kind of situation, need to monitor business data flow in real time, so as to detect various abnormal interaction datas.
It in the prior art, can be by the exception that predefines out when being detected in real time to business data flow
Interaction data mark or feature, e.g., user account, IP address etc.；It is black using this using the mark or feature as blacklist
The business data flow that name single pair generates in real time is detected.However mark or feature in blacklist, all it is in abnormal interaction
After data generate, and after determining the interaction data for abnormal interaction data by the analysis of artificial or other modes, just carry
The mark or feature of the interaction data for the exception taken out, and the mark or feature are added in into blacklist.So generate the black name
Mark or feature efficiency in list is relatively low, and due to relatively low, the blacklist that generates mark in the blacklist or feature efficiency
In mark or feature always largely lag behind the business data flow that current time generates in real time, using in the blacklist
Mark or feature, it is difficult to timely detect the abnormal interaction data in the business data flow that current time generates in real time.
Invention content
The embodiment of the present invention is designed to provide a kind of data detection method, device and electronic equipment, to improve exception
Interaction data Data Identification formation efficiency, so as to when being detected to the business data flow generated in real time, Neng Gouji
When determine abnormal interaction data.Specific technical solution is as follows：
An embodiment of the present invention provides a kind of data detection method, including：
Using sample business data flow as the input of initial machine learning model, the initial machine learning model is carried out
Training, obtains trained machine learning model, and using the trained machine learning model as Data Detection mould
Type；
Using business data flow to be detected as the input of the Data Detection model, the Data Detection model root is obtained
According to the Data Identification that the business data flow to be detected is determined, the Data Identification is handed between terminal and server-side
The mark of abnormal interaction data caused by mutually, the business datum to be detected are same with the sample business data flow
The business datum of type；
According to the Data Identification, the business data flow to be detected is detected, is determined described to be detected
In business data flow with the corresponding abnormal interaction data of the Data Identification.
Optionally, the sample business data flow includes：
Real time traffic data stream caused by real-time, interactive is carried out between terminal and the server-side,
Alternatively, business data flow caused by the interaction completed between terminal and the server-side.
Optionally, the method further includes：
The Data Detection model to be realized by programming language, composition is corresponding with the Data Detection model,
And the data detection process being made up of the programming language；
Business data flow to be detected is inputted into the data detection process, obtains the data detection process according to
The Data Identification that business data flow to be detected is determined.
Optionally, it is described according to the Data Identification, the business data flow to be detected is detected, determines institute
State in business data flow to be detected with the corresponding interaction data of the Data Identification, including：
The data detection process is encapsulated as to the plug-in unit of preset data detecting system；
By the preset data detecting system, according to the Data Identification that the plug-in unit is determined, to described to be detected
Business data flow is detected, determine in the business data flow to be detected with the Data Identification is corresponding interacts number
According to.
Optionally, described using business data flow to be detected as the input of the Data Detection model, described in acquisition
After the Data Identification that Data Detection model is determined according to the business data flow to be detected, the method further includes：
The Data Identification is preserved in the buffer；
Correspondingly, it is described according to the Data Identification, the business data flow to be detected is detected, determines institute
State in business data flow to be detected with the corresponding interaction data of the Data Identification, including：
By preset data detecting system, the Data Identification in the caching is obtained, and pass through the preset data
Detecting system is detected the business data flow to be detected, determines described to be detected according to the Data Identification
In business data flow with the corresponding interaction data of the Data Identification.
Example in real time of the invention additionally provides a kind of data detection device, including：
Training module, for the input using sample business data flow as initial machine learning model, to the initial machine
Device learning model is trained, and obtains trained machine learning model, and by the trained machine learning model
As Data Detection model；
Feature selection module for the input using business data flow to be detected as the Data Detection model, obtains
The Data Identification that the Data Detection model is determined according to the business data flow to be detected, the Data Identification are terminal
The mark of the interaction data of exception caused by being interacted between server-side, the business datum to be detected and the sample
This business data flow is same type of business datum；
Detection module, for according to the Data Identification, being detected, determining to the business data flow to be detected
In the business data flow to be detected with the corresponding abnormal interaction data of the Data Identification.
Optionally, the sample business data flow in the training module includes：
Real time traffic data stream caused by real-time, interactive is carried out between terminal and the server-side,
Alternatively, business data flow caused by the interaction completed between terminal and the server-side.
Optionally, described device further includes：
Package module for the Data Detection model to be realized by programming language, is formed and is examined with the data
Survey model data detection process that is corresponding, and being made up of the programming language；
Import modul for business data flow to be detected to be inputted the data detection process, obtains the data inspection
The Data Identification that ranging sequence is determined according to the business data flow to be detected.
Optionally, the detection module, is specifically used for：
The data detection process is encapsulated as to the plug-in unit of preset data detecting system；Pass through the preset data detection system
System, according to the Data Identification that the plug-in unit is determined, is detected the business data flow to be detected, determines described treat
In the business data flow of detection with the corresponding interaction data of the Data Identification.
Optionally, described device further includes：
Preserving module, for the Data Identification to be preserved in the buffer；
Correspondingly, the detection module, is specifically used for：
By preset data detecting system, the Data Identification in the caching is obtained, and pass through the preset data
Detecting system is detected the business data flow to be detected, determines described to be detected according to the Data Identification
In business data flow with the corresponding interaction data of the Data Identification.
The embodiment of the present invention additionally provides a kind of electronic equipment, total including processor, communication interface, memory and communication
Line, wherein, processor, communication interface, memory completes mutual communication by communication bus；
Memory, for storing computer program；
Processor during for performing the program stored on memory, realizes any of the above-described data detection method.
At the another aspect that the present invention is implemented, a kind of computer readable storage medium is additionally provided, it is described computer-readable
Instruction is stored in storage medium, when run on a computer so that computer performs any of the above-described data inspection
Survey method.
At the another aspect that the present invention is implemented, the embodiment of the present invention additionally provides a kind of computer program production comprising instruction
Product, when run on a computer so that computer performs any of the above-described data detection method.
A kind of data detection method provided in an embodiment of the present invention, device and electronic equipment, can be by sample business datum
Stream input initial machine learning model, is trained initial machine learning model, obtains trained machine learning model,
And using trained machine learning model as Data Detection model；Business data flow input data to be detected is detected into mould
Type obtains the Data Identification that Data Detection model is determined according to business data flow to be detected；Further according to Data Identification, treat
The business data flow of detection is detected, determine in business data flow to be detected with Data Identification is corresponding interacts number
According to.Obtained Data Detection model is trained to initial machine learning model using sample business data flow, compared to biography
The artificial or other modes of system, Data Detection model can more quickly determine the exception in business data flow to be detected
Interaction data, and Data Identification possessed by the interaction data of output abnormality.The data of abnormal interaction data can be improved
The formation efficiency of mark so that the generation of Data Identification no longer significantly lags behind business data flow to be detected, so as to
To be timely detected using the Data Identification to business data flow to be detected, detection efficiency is improved.Certainly, implement this hair
Bright any product or method must be not necessarily required to reach all the above advantage simultaneously.
Description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described.
Fig. 1 is a kind of flow chart of data detection method provided in an embodiment of the present invention；
Fig. 2 is another flow chart of data detection method provided in an embodiment of the present invention；
Fig. 3 is the structure chart of data detection device provided in an embodiment of the present invention；
Fig. 4 is the structure chart of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is described.
Referring to Fig. 1, Fig. 1 is a kind of flow chart of data detection method provided in an embodiment of the present invention, including：
Step 101, using sample business data flow as the input of initial machine learning model, to initial machine learning model
It is trained, obtains trained machine learning model, and by trained machine learning model, as Data Detection mould
Type.
The embodiment of the present invention can be applied to each class of electronic devices of server-side, for example, server, Distributed Services cluster
Deng.
Terminal and server-side, can when being interacted for a data service, such as online ticketing, online prize drawing etc.
To generate a large amount of interaction data, for example, all kinds of requests that terminal is sent to server-side, each generic operation performed by terminal etc..
A large amount of terminal is interacted with server-side, can form the data flow for continuously containing a large amount of interaction datas, which can
To be known as the business data flow of the data service.
Sample business data flow refers to, for a kind of data service or same type of data service, can to perform
During the data service, the business data flow actually generated is as sample business data flow；Or software can also will be utilized to simulate
The virtual business data flow gone out is as sample business data flow.Wherein same type of data service refers to operation flow or step
Rapid essentially identical data service, e.g., the data services such as online ballot, online prize drawing, what terminal was interacted with server-side
Operation flow is basically identical, and generated interaction data is also closer to, it is possible to as same type of data service.
Initial machine learning model is without trained machine learning model in original state.The machine learning
Model can be the unsupervised machine learning model of autonomous learning.Specifically, machine learning model can there are many kinds of class, examples
Such as, can be the types such as cluster, entropy, decision tree, Bayes.According to the type of different data services, can select with being somebody's turn to do
The machine learning model that data flow caused by data service is adapted is as initial machine learning model.
After sample business data flow is inputted initial machine learning model, which can be according to the sample
This business data flow carries out autonomous machine learning, which is trained.Initial machine learning model
Can be internal by it, such as neural network algorithm internal algorithm, by calculating sample data stream, it is capable of determining that
Data characteristics possessed by interaction data in the sample data stream.
Using the data characteristics, machine learning model can be to other and the sample business data flow same-type that are inputted
Data flow be detected, so that it is determined that the friendship for going out the interaction data being consistent in inputted data flow with this feature or not being consistent
Mutual data.Meanwhile it can also utilize other business data flows during being detected to other business data flows
Continue autonomous machine learning, so as to be constantly modified to the data characteristics so that the data characteristics can be increasingly
Accurate hair mirrors the characteristics of interaction data in the business data flow of the type.Data characteristics can be abstract calculating knot
Fruit or specific one or more data or, terminal and rule or pattern, data possessed by server-side interactive process
Feature can be saved in inside machine learning model.Specifically, the training process of machine learning model and its internal meter
Calculation mode etc. belongs to the prior art, and details are not described herein.
After completing to the training of initial machine learning model, trained machine learning model be just provided with detection with
The ability of the identical data flow of sample data stream type, can be using trained machine learning model as Data Detection mould
Type.For detecting corresponding data flow.
Step 102, using business data flow to be detected as the input of Data Detection model, Data Detection model root is obtained
According to the Data Identification that business data flow to be detected is determined, produced by Data Identification interacts between terminal and server-side
Abnormal interaction data mark, business datum to be detected is same type of business datum with sample business data flow.
Business data flow to be detected can be that the interaction generated in real time between interactive terminal and server-side is occurring
The business data flow that data are formed.
In a kind of embodiment of the embodiment of the present invention, Data Detection model can be built in server-side, for example, passing through
Distributed server cluster or large server are realized in server-side, thus the business datum to be detected that server-side obtains in real time
Stream, can be directly inputted into Data Detection model.
In the another embodiment of the embodiment of the present invention, the cloud platform that Data Detection model can be on line is established,
Such as Tensor Flow or Spark Mlib etc. is used for the cloud platform of model generation, which can be distributed server
Cluster or system, but the cloud platform and server-side are mutual indepedent, to go out terminal and the third party device or system except server-side.
Server-side can the business data flow to be detected of acquisition be sent to the cloud platform in real time or the cloud platform can be right in real time
Whole interaction datas between terminal and server-side are monitored, so as to obtain business data flow to be detected and be input to data
Detection model.
Business data flow to be detected can with sample business data flow for data flow caused by same data service or
Person's business data flow to be detected can be data flow caused by same type of data service with sample business data flow.I.e.
Business data flow to be detected needs consistent in data type, data characteristics etc. with sample business data flow.
After business data flow input data detection model to be detected, Data Detection model can be according in itself
The data characteristics corresponding with business data flow to be detected that portion is preserved, examines the business data flow to be detected
Survey, pass through the matching of the data characteristics, it may be determined that go out in business data flow to be detected match with the data characteristics or with
The unmatched interaction data of the data characteristics.According to the different situations of different service types, the friendship to match with the data characteristics
Mutual data may be abnormal interaction data or may be abnormal interaction with interaction data that the data characteristics does not match that
Data, specifically, data detection model can be configured as needed in practical applications, so as to different for determining
Normal interaction data generates Data Identification corresponding with the interaction data of the exception.Mark of the Data Identification for abnormal interaction data
Know, the field or information that can be identified with abnormal interaction data can be contained in the Data Identification, for example, generating abnormal
The IP address of the terminal of interaction data, model etc..Business data flow to be detected so as to can directly be corresponded to by the mark
In abnormal interaction data.
Step 103, according to Data Identification, business data flow to be detected is detected, determines business to be detected
In data flow with the corresponding abnormal interaction data of Data Identification.
Data Detection model can determine interaction data abnormal in business data flow to be detected, but the Data Detection
Model directly can not carry out respective handling, but can export the mark of the interaction data of the exception to the interaction data of the exception in itself
Know, i.e. Data Identification.After Data Identification is determined by Data Detection model, it is possible to synchronous, according to the data mark
Know, business data flow to be detected is detected, is determined again from business data flow to be detected and the Data Identification pair
The interaction data answered, the abnormal interaction data in the business data flow which can be to be detected as this, and can
It is handled accordingly with the interaction data for being directed to the exception, for example, being blocked to the interaction data or in the interaction data
Corresponding terminal pops up corresponding verification page etc. when being interacted with server-side.
In another realization method of the embodiment of the present invention, the business datum to be detected of institute's input data detection model
Stream can also be the non real-time business data flow to be detected generated a data service mid-early stage, non real-time be treated by this
The business data flow of detection can equally produce the abnormal interaction data phase in the non real-time business data flow to be detected
Corresponding Data Identification.Since in a data service, abnormal interaction data generally can all continue for some time, for example,
When buying admission ticket by Malware, it generally can all continue regular hour or multiple carry out malice booking.So also may be used
By the abnormal corresponding Data Identification of interaction data in non real-time business data flow to be detected, to be produced in real time to current
Raw business data flow to be detected is detected, interaction data and number in the business data flow to be detected generated in real time
When matching according to mark, for example, in Data Identification in the IP address of terminal and the business data flow to be detected generated in real time
Certain interaction data IP address of terminal it is identical, then it is abnormal that can determine the interaction data to match with Data Identification
Interaction data.
In embodiments of the present invention, initial machine learning model is trained using sample business data flow obtained
Data Detection model, compared to traditional artificial or other modes, Data Detection model can more quickly be determined to be checked
Abnormal interaction data in the business data flow of survey, and Data Identification possessed by the interaction data of output abnormality.It can carry
The formation efficiency of the Data Identification of high abnormal interaction data so that the generation of Data Identification no longer significantly lags behind to be checked
The business data flow of survey so as to be timely detected using the Data Identification to business data flow to be detected, is improved
Detection efficiency.
With reference to the above embodiments, sample business data flow in data detection method provided in an embodiment of the present invention can be with
Including：
Real time traffic data stream caused by real-time, interactive is carried out between terminal and server-side, alternatively, terminal and server-side
Between business data flow caused by the interaction completed.
In the embodiment of the present invention, when being trained to initial machine learning model, current time can be generated in real time
Real time traffic data stream as sample business data flow, initial machine learning model is made to carry out autonomous unsupervised engineering
It practises.So as to not need to individually provide sample, and then can more fast and easily complete to initial machine learning model
Training.
It alternatively, can also be by business data flow caused by the interaction completed, as sample data stream.So as to
The training to initial machine learning model is completed in the case where not real-time business data flow generates.So as to when generation
It, can be timely by having completed trained machine learning model to generated business datum during real-time business data flow
Stream is handled, and output data mark is used for the detection of the real-time business data flow, and then can improve real-time and hold
Line efficiency.
With reference to above-described embodiment, referring to Fig. 2, data detection method provided in an embodiment of the present invention can also include：
Step 104, Data Detection model to be realized by programming language, composition is corresponding with Data Detection model,
And the data detection process being made up of programming language.
Initial machine learning model is completed to train, and using trained machine learning model as Data Detection model
When, it can be trained, and obtain inhomogeneity using different types of initial machine learning model for different types of service
The corresponding different Data Detection model of type of service of type.Wherein, partial data detection model can examine the data
It surveys model to be described by programming language, you can realize the function and work of the Data Detection model in the method by programming
With.For example, working as Data Detection model for cluster, entropy, decision tree, Bayes can pass through the program words such as such as JAVA when types
Speech according to Data Detection model, will build data detection process corresponding with the Data Detection model.
When being data detection process by Data Detection model conversation, the friendship for determining exception in data detection process
Parameter information in the data characteristics preserved inside the mutually individual parameters of data, such as Data Detection model etc., can be direct
It is obtained from Data Detection model.When Data Detection model is the Data Detection journey when the cloud platform with server-side independence is established
Sequence can obtain the individual parameters by the method remotely pulled, and when the parameter change of Data Detection model, data
Detecting program corresponding can also change the parameter, so as to the configuration of more convenient data detection process.
Step 105, business data flow input data to be detected is detected into program, obtains data detection process according to be checked
The Data Identification that the business data flow of survey is determined.
Constructed data detection process have the function of with Data Detection model it is identical, so as to by industry to be detected
Business data flow inputs the data detection process, and pass through the data detection process, and it is corresponding to obtain business data flow to be detected
Data Identification.
In embodiments of the present invention, it is corresponding data detection process by Data Detection model conversation by programming language,
Data detection process relative data detection model occupies less resource, more flexible can be run in server-side, data inspection
Ranging sequence can easily load on various types of server-sides, can improve data detection method provided in an embodiment of the present invention
Compatibility.
With reference to above-described embodiment, in data detection method provided in an embodiment of the present invention, step 103, according to data mark
Know, business data flow to be detected is detected, is determined corresponding with Data Identification in business data flow to be detected
Interaction data, including：
Data detection process is encapsulated as the plug-in unit of preset data detecting system by step 103a.
In practical applications, business data flow to be detected can in real time be examined by preset data detecting system
It surveys, and passes through the preset data detecting system and corresponding measure is taken to handle abnormal interaction data.Preset data
Detecting system can be the existing various data detection systems for being detected to interaction data.Preset data detecting system
Realization method belong to the prior art, details are not described herein.
It, can be by Data Detection journey when being detected by preset data detecting system to business data flow to be detected
Sequence is encapsulated as the plug-in unit of preset data detecting system.Specifically, data detection process to be encapsulated as to the method for plug-in unit, belong to existing
Details are not described herein for technology.
Step 103b, by preset data detecting system, according to the Data Identification that the plug-in unit is determined, to industry to be detected
Business data flow be detected, determine in business data flow to be detected with the corresponding interaction data of Data Identification.
Data detection process is encapsulated as after the plug-in unit of preset data detecting system, is utilizing preset data detecting system
When being detected to business data flow to be detected, preset data detecting system can obtain business data flow to be detected, example
Such as can business data flow to be detected be directly input to preset data detecting system or the preset data detecting system
Can by real-time monitor terminal and server-side in interactive process generated whole interaction datas, so as to fulfill to be checked
The monitoring of the business data flow of survey.
After preset data detecting system obtains business data flow to be detected, it can be sealed first by data detection process
Obtained plug-in unit is filled, which is analyzed, determines that the business data flow to be detected corresponds to
Data Identification, then preset data detecting system business data flow to be detected can be carried out according to the Data Identification
Detection, determines interaction number that is corresponding with the Data Identification in business data flow to be detected or matching with the Data Identification
According to as abnormal interaction data, and carrying out respective handling.
It in embodiments of the present invention, can be with by the way that data detection process to be encapsulated as to the plug-in unit of preset data detecting system
Further improve the compatibility of data detection process so that the data detection process can be applied to various existing preset datas
In detecting system.And due to being equipped with the plug-in unit in preset data detecting system so that preset data detecting system can be with
More effectively business data flow to be detected is detected, and the real-time of preset data detecting system can be improved.
It,, will be to be detected in step 102 in data detection method provided in an embodiment of the present invention with reference to above-described embodiment
Input of the business data flow as Data Detection model, obtain Data Detection model and determined according to business data flow to be detected
After the Data Identification gone out, this method further includes：
Step 100, Data Identification is preserved in the buffer.
Correspondingly, step 103, according to Data Identification, is detected business data flow to be detected, determines to be detected
Business data flow in the corresponding interaction data of Data Identification, can include：
By preset data detecting system, obtain the Data Identification in caching, and pass through preset data detecting system according to
Data Identification is detected business data flow to be detected, determine in business data flow to be detected with Data Identification phase
Corresponding interaction data.
Data Detection model can independently of server-side cloud platform generate or structure, when the Data Detection model according to
Business data flow to be detected, after producing the corresponding Data Identification of business data flow to be detected, in order to cause
Server-side can efficiently call the Data Identification, and the Data Identification can be first stored in caching, which can be
Tell caching.So as to further improve the real-time that server-side is detected the business data flow to be detected.
Specifically, server-side can be detected business data flow to be detected by preset data detecting system,
The preset data detecting system can convenient access cache, so as to timely call caching in Data Identification, and
Since Data Detection model can generate Data Identification in real time, and preserve to the Data Identification in caching or update caching, institute
Data Identification in caching can also be called in real time with preset data detecting system, preset data detection system can be improved
The real-time of system guarantees timely to detect the abnormal interaction data corresponding to Data Identification.
Referring to Fig. 3, Fig. 3 is the structure chart of data detection device provided in an embodiment of the present invention, including：
Training module 301, for the input using sample business data flow as initial machine learning model, to described initial
Machine learning model is trained, and obtains trained machine learning model, and by the trained machine learning mould
Type is as Data Detection model；
Feature selection module 302 for the input using business data flow to be detected as the Data Detection model, obtains
The Data Identification that the Data Detection model is taken to be determined according to the business data flow to be detected, the Data Identification is eventually
Hold server-side between interact caused by exception interaction data mark, the business datum to be detected with it is described
Sample business data flow is same type of business datum；
Detection module 303, for according to the Data Identification, being detected to the business data flow to be detected, really
Make in the business data flow to be detected with the corresponding abnormal interaction data of the Data Identification.
In embodiments of the present invention, initial machine learning model is trained using sample business data flow obtained
Data Detection model, compared to traditional artificial or other modes, Data Detection model can more quickly be determined to be checked
Abnormal interaction data in the business data flow of survey, and Data Identification possessed by the interaction data of output abnormality.It can carry
The formation efficiency of the Data Identification of high abnormal interaction data so that the generation of Data Identification no longer significantly lags behind to be checked
The business data flow of survey so as to be timely detected using the Data Identification to business data flow to be detected, is improved
Detection efficiency.
Optionally, in data detection device provided in an embodiment of the present invention, the sample in the training module 301
Business data flow includes：
Between terminal and the server-side carry out real-time, interactive caused by real time traffic data stream or,
Business data flow caused by the interaction completed between terminal and the server-side.
Optionally, in data detection device provided in an embodiment of the present invention, described device further includes：
Package module for the Data Detection model to be realized by programming language, is formed and is examined with the data
Survey model is corresponding, and passes through the data detection process that programming language is formed；
Import modul for business data flow to be detected to be inputted the data detection process, obtains the data inspection
The Data Identification that ranging sequence is determined according to the business data flow to be detected.
Optionally, in data detection device provided in an embodiment of the present invention, the detection module 303 is specifically used for：
The data detection process is encapsulated as to the plug-in unit of preset data detecting system；Pass through the preset data detection system
System, according to the Data Identification that the plug-in unit is determined, is detected the business data flow to be detected, determines described treat
In the business data flow of detection with the corresponding interaction data of the Data Identification.
Optionally, in data detection device provided in an embodiment of the present invention, described device further includes：
Preserving module, for the Data Identification to be preserved in the buffer；
Correspondingly, the detection module 303, is specifically used for：
By the preset data detecting system, the Data Identification in the caching is obtained, and is passed through described default
Data detection system is detected the business data flow to be detected, determines described to be checked according to the Data Identification
In the business data flow of survey with the corresponding interaction data of the Data Identification.
The embodiment of the present invention additionally provides a kind of electronic equipment, as shown in figure 4, including processor 401, communication interface 402,
Memory 403 and communication bus 404, wherein, processor 401, communication interface 402, memory 403 is complete by communication bus 404
Into mutual communication,
Memory 403, for storing computer program；
Processor 401 during for performing the program stored on memory 403, realizes following steps：
Using sample business data flow as the input of initial machine learning model, the initial machine learning model is carried out
Training, obtains trained machine learning model, and using the trained machine learning model as Data Detection mould
Type；
Using business data flow to be detected as the input of the Data Detection model, the Data Detection model root is obtained
According to the Data Identification that the business data flow to be detected is determined, the Data Identification is handed between terminal and server-side
The mark of abnormal interaction data caused by mutually, the business datum to be detected are same with the sample business data flow
The business datum of type；
According to the Data Identification, the business data flow to be detected is detected, is determined described to be detected
In business data flow with the corresponding abnormal interaction data of the Data Identification.
The communication bus that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral Pomponent
Interconnect, abbreviation PCI) bus or expanding the industrial standard structure (Extended Industry Standard
Architecture, abbreviation EISA) bus etc..The communication bus can be divided into address bus, data/address bus, controlling bus etc..
For ease of representing, only represented in figure with a thick line, it is not intended that an only bus or a type of bus.
Communication interface is for the communication between above-mentioned electronic equipment and other equipment.
Memory can include random access memory (Random Access Memory, abbreviation RAM), can also include
Nonvolatile memory (non-volatile memory), for example, at least a magnetic disk storage.Optionally, memory may be used also
To be at least one storage device for being located remotely from aforementioned processor.
Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit,
Abbreviation CPU), network processing unit (Ne twork Processor, abbreviation NP) etc.；It can also be digital signal processor
(Digital Signal Processing, abbreviation DSP), application-specific integrated circuit (Applica tion Specific
Integrated Circuit, abbreviation ASIC), field programmable gate array (Field-Programmable Gate Array,
Abbreviation FPGA) either other programmable logic device, discrete gate or transistor logic, discrete hardware components.
In another embodiment provided by the invention, a kind of computer readable storage medium is additionally provided, which can
It reads to be stored with instruction in storage medium, when run on a computer so that computer performs any institute in above-described embodiment
The data detection method stated.
In another embodiment provided by the invention, a kind of computer program product for including instruction is additionally provided, when it
When running on computers so that computer performs any data detection method in above-described embodiment.
In the above-described embodiments, can come wholly or partly by software, hardware, firmware or its arbitrary combination real
It is existing.When implemented in software, it can entirely or partly realize in the form of a computer program product.The computer program
Product includes one or more computer instructions.When loading on computers and performing the computer program instructions, all or
It partly generates according to the flow or function described in the embodiment of the present invention.The computer can be all-purpose computer, special meter
Calculation machine, computer network or other programmable devices.The computer instruction can be stored in computer readable storage medium
In or from a computer readable storage medium to another computer readable storage medium transmit, for example, the computer
Instruction can pass through wired (such as coaxial cable, optical fiber, number from a web-site, computer, server or data center
User's line (DSL)) or wireless (such as infrared, wireless, microwave etc.) mode to another web-site, computer, server or
Data center is transmitted.The computer readable storage medium can be any usable medium that computer can access or
It is the data storage devices such as server, the data center integrated comprising one or more usable mediums.The usable medium can be with
It is magnetic medium, (for example, floppy disk, hard disk, tape), optical medium (for example, DVD) or semiconductor medium (such as solid state disk
Solid State Disk (SSD)) etc..
It should be noted that herein, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any this practical relationship or sequence.Moreover, term " comprising ", "comprising" or its any other variant are intended to
Non-exclusive inclusion, so that process, method, article or equipment including a series of elements not only will including those
Element, but also including other elements that are not explicitly listed or further include as this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
Also there are other identical elements in process, method, article or equipment including the element.
Each embodiment in this specification is described using relevant mode, identical similar portion between each embodiment
Point just to refer each other, and the highlights of each of the examples are difference from other examples.Especially for device reality
For applying example, since it is substantially similar to embodiment of the method, so description is fairly simple, related part is referring to embodiment of the method
Part explanation.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all
Any modification, equivalent replacement, improvement and so within the spirit and principles in the present invention, are all contained in protection scope of the present invention
It is interior.
Claims (11)
1. a kind of data detection method, which is characterized in that including：
Using sample business data flow as the input of initial machine learning model, the initial machine learning model is instructed
Practice, obtain trained machine learning model, and using the trained machine learning model as Data Detection model；
Using business data flow to be detected as the input of the Data Detection model, the Data Detection model is obtained according to institute
The Data Identification that business data flow to be detected is determined is stated, the Data Identification interacts institute between terminal and server-side
The mark of the abnormal interaction data of generation, the business datum to be detected are same type with the sample business data flow
Business datum；
According to the Data Identification, the business data flow to be detected is detected, determines the business to be detected
In data flow with the corresponding abnormal interaction data of the Data Identification.
2. data detection method according to claim 1, which is characterized in that the sample business data flow includes：
Real time traffic data stream caused by real-time, interactive is carried out between terminal and the server-side,
Alternatively, business data flow caused by the interaction completed between terminal and the server-side.
3. data detection method according to claim 1, which is characterized in that the method further includes：
The Data Detection model by programming language is realized, is formed corresponding with the Data Detection model and logical
Cross the data detection process that the programming language is formed；
Business data flow to be detected is inputted into the data detection process, obtains the data detection process according to described to be checked
The Data Identification that the business data flow of survey is determined.
4. data detection method according to claim 3, which is characterized in that it is described according to the Data Identification, to described
Business data flow to be detected is detected, and is determined corresponding with the Data Identification in the business data flow to be detected
Interaction data, including：
The data detection process is encapsulated as to the plug-in unit of preset data detecting system；
By the preset data detecting system, according to the Data Identification that the plug-in unit is determined, to the business to be detected
Data flow is detected, determine in the business data flow to be detected with the corresponding interaction data of the Data Identification.
5. data detection method according to claim 1, which is characterized in that make business data flow to be detected described
For the input of the Data Detection model, obtain the Data Detection model and determined according to the business data flow to be detected
Data Identification after, the method further includes：
The Data Identification is preserved in the buffer；
Correspondingly, it is described according to the Data Identification, the business data flow to be detected is detected, determines described treat
In the business data flow of detection with the corresponding interaction data of the Data Identification, including：
By preset data detecting system, the Data Identification in the caching is obtained, and detect by the preset data
System is detected the business data flow to be detected, determines the business to be detected according to the Data Identification
In data flow with the corresponding interaction data of the Data Identification.
6. a kind of data detection device, which is characterized in that including：
Training module, for the input using sample business data flow as initial machine learning model, to the initial machine
Practise model to be trained, obtain trained machine learning model, and using the trained machine learning model as
Data Detection model；
Feature selection module, for the input using business data flow to be detected as the Data Detection model, described in acquisition
The Data Identification that Data Detection model is determined according to the business data flow to be detected, the Data Identification are terminal and clothes
The mark of the interaction data of exception, the business datum to be detected and the sample industry caused by being interacted between business end
Business data flow is same type of business datum；
Detection module, for according to the Data Identification, being detected, determining described to the business data flow to be detected
In business data flow to be detected with the corresponding abnormal interaction data of the Data Identification.
7. data detection device according to claim 6, which is characterized in that the sample business in the training module
Data flow includes：
Real time traffic data stream caused by real-time, interactive is carried out between terminal and the server-side,
Alternatively, business data flow caused by the interaction completed between terminal and the server-side.
8. data detection device according to claim 6, which is characterized in that described device further includes：
Package module for the Data Detection model to be realized by programming language, is formed and the Data Detection mould
Type is corresponding, and the data detection process being made up of the programming language；
Import modul for business data flow to be detected to be inputted the data detection process, obtains the Data Detection journey
The Data Identification that sequence is determined according to the business data flow to be detected.
9. data detection device according to claim 8, which is characterized in that the detection module is specifically used for：
The data detection process is encapsulated as to the plug-in unit of preset data detecting system；By the preset data detecting system,
According to the Data Identification that the plug-in unit is determined, the business data flow to be detected is detected, is determined described to be checked
In the business data flow of survey with the corresponding interaction data of the Data Identification.
10. data detection device according to claim 6, which is characterized in that described device further includes：
Preserving module, for the Data Identification to be preserved in the buffer；
Correspondingly, the detection module, is specifically used for：
By preset data detecting system, the Data Identification in the caching is obtained, and detect by the preset data
System is detected the business data flow to be detected, determines the business to be detected according to the Data Identification
In data flow with the corresponding interaction data of the Data Identification.
11. a kind of electronic equipment, which is characterized in that including processor, communication interface, memory and communication bus, wherein, processing
Device, communication interface, memory complete mutual communication by communication bus；
Memory, for storing computer program；
Processor during for performing the program stored on memory, realizes any method and steps of claim 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711446332.2A CN108200032A (en) | 2017-12-27 | 2017-12-27 | A kind of data detection method, device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711446332.2A CN108200032A (en) | 2017-12-27 | 2017-12-27 | A kind of data detection method, device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108200032A true CN108200032A (en) | 2018-06-22 |
Family
ID=62584608
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711446332.2A Pending CN108200032A (en) | 2017-12-27 | 2017-12-27 | A kind of data detection method, device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108200032A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109919744A (en) * | 2018-11-23 | 2019-06-21 | 阿里巴巴集团控股有限公司 | Detection method neural network based and device |
| CN110837718A (en) * | 2019-11-07 | 2020-02-25 | 交控科技股份有限公司 | Turnout fault detection method and device, electronic equipment and storage medium |
| CN112235230A (en) * | 2019-07-15 | 2021-01-15 | 北京观成科技有限公司 | Malicious traffic identification method and system |
| CN112835780A (en) * | 2019-11-25 | 2021-05-25 | 杭州海康威视系统技术有限公司 | Service detection method and device |
| CN113497797A (en) * | 2020-04-08 | 2021-10-12 | 中国移动通信集团广东有限公司 | Method and device for detecting abnormality of ICMP tunnel transmission data |
| CN114465962A (en) * | 2019-09-16 | 2022-05-10 | 华为技术有限公司 | Data stream type identification method and related equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101060444A (en) * | 2007-05-23 | 2007-10-24 | 西安交大捷普网络科技有限公司 | Bayesian statistical model based network anomaly detection method |
| US20160105462A1 (en) * | 2008-12-16 | 2016-04-14 | At&T Intellectual Property I, L.P. | Systems and Methods for Rule-Based Anomaly Detection on IP Network Flow |
| CN106657141A (en) * | 2017-01-19 | 2017-05-10 | 西安电子科技大学 | Android malware real-time detection method based on network flow analysis |
| CN106817270A (en) * | 2015-12-01 | 2017-06-09 | 精硕科技（北京）股份有限公司 | Network traffics acquisition method, system and server |
| CN107465643A (en) * | 2016-06-02 | 2017-12-12 | 国家计算机网络与信息安全管理中心 | A kind of net flow assorted method of deep learning |
-
2017
- 2017-12-27 CN CN201711446332.2A patent/CN108200032A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101060444A (en) * | 2007-05-23 | 2007-10-24 | 西安交大捷普网络科技有限公司 | Bayesian statistical model based network anomaly detection method |
| US20160105462A1 (en) * | 2008-12-16 | 2016-04-14 | At&T Intellectual Property I, L.P. | Systems and Methods for Rule-Based Anomaly Detection on IP Network Flow |
| CN106817270A (en) * | 2015-12-01 | 2017-06-09 | 精硕科技（北京）股份有限公司 | Network traffics acquisition method, system and server |
| CN107465643A (en) * | 2016-06-02 | 2017-12-12 | 国家计算机网络与信息安全管理中心 | A kind of net flow assorted method of deep learning |
| CN106657141A (en) * | 2017-01-19 | 2017-05-10 | 西安电子科技大学 | Android malware real-time detection method based on network flow analysis |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109919744B (en) * | 2018-11-23 | 2023-01-10 | 创新先进技术有限公司 | Neural network-based detection method and device |
| CN109919744A (en) * | 2018-11-23 | 2019-06-21 | 阿里巴巴集团控股有限公司 | Detection method neural network based and device |
| CN112235230A (en) * | 2019-07-15 | 2021-01-15 | 北京观成科技有限公司 | Malicious traffic identification method and system |
| CN112235230B (en) * | 2019-07-15 | 2023-05-02 | 北京观成科技有限公司 | Malicious traffic identification method and system |
| US11838215B2 (en) | 2019-09-16 | 2023-12-05 | Huawei Technologies Co., Ltd. | Data stream classification method and related device |
| CN114465962A (en) * | 2019-09-16 | 2022-05-10 | 华为技术有限公司 | Data stream type identification method and related equipment |
| CN114465962B (en) * | 2019-09-16 | 2024-01-05 | 华为技术有限公司 | Data stream type identification method and related equipment |
| CN110837718A (en) * | 2019-11-07 | 2020-02-25 | 交控科技股份有限公司 | Turnout fault detection method and device, electronic equipment and storage medium |
| CN110837718B (en) * | 2019-11-07 | 2023-12-26 | 交控科技股份有限公司 | Switch fault detection method and device, electronic equipment and storage medium |
| CN112835780A (en) * | 2019-11-25 | 2021-05-25 | 杭州海康威视系统技术有限公司 | Service detection method and device |
| CN112835780B (en) * | 2019-11-25 | 2024-02-02 | 杭州海康威视系统技术有限公司 | Service detection method and device |
| CN113497797A (en) * | 2020-04-08 | 2021-10-12 | 中国移动通信集团广东有限公司 | Method and device for detecting abnormality of ICMP tunnel transmission data |
| CN113497797B (en) * | 2020-04-08 | 2023-04-28 | 中国移动通信集团广东有限公司 | Abnormality detection method and device for ICMP tunnel transmission data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108200032A (en) | A kind of data detection method, device and electronic equipment | |
| US10733088B1 (en) | Methods, systems, and computer readable media for testing a network node or a related application programming interface using source code metadata | |
| US10880197B2 (en) | Methods, systems, and computer readable media for testing a network node using source code for programming a packet forwarding plane of the network node | |
| US11218510B2 (en) | Advanced cybersecurity threat mitigation using software supply chain analysis | |
| Huong et al. | Federated learning-based explainable anomaly detection for industrial control systems | |
| US20170168885A1 (en) | System and Method for Testing Internet of Things Network | |
| Shang et al. | Modbus/TCP communication anomaly detection based on PSO-SVM | |
| CN111586071B (en) | Encryption attack detection method and device based on recurrent neural network model | |
| CN114331761B (en) | Equipment parameter analysis and adjustment method and system for special transformer acquisition terminal | |
| CN110166276A (en) | A kind of localization method, device, terminal device and the medium of frame synchronization exception | |
| CN110177079A (en) | The calling system and call method of intelligent contract | |
| CN106529283B (en) | A kind of software-oriented defines network-based control device safety quantitative analysis method | |
| CN109274692A (en) | A kind of identification block chain network is done evil the method and device of node | |
| CN107678852A (en) | Method, system, equipment and the storage medium calculated in real time based on flow data | |
| US20170123765A1 (en) | Composable application session parameters | |
| Pinto et al. | Enabling data-driven anomaly detection by design in cyber-physical production systems | |
| TWI703846B (en) | URL abnormal location method, device, server and storage medium | |
| Gupta et al. | Ddos attack detection through digital twin technique in metaverse | |
| CN108234441A (en) | Determine method, apparatus, electronic equipment and the storage medium of forgery access request | |
| CN110874638A (en) | Behavior analysis-oriented meta-knowledge federation method, device, electronic equipment and system | |
| CN103780592B (en) | Method and apparatus for determining being stolen of user account | |
| CN112839055B (en) | Network application identification method and device for TLS encrypted traffic and electronic equipment | |
| CN106789948A (en) | A kind of network web page method for detecting abnormality | |
| Demmese et al. | Machine learning based fileless malware traffic classification using image visualization | |
| US20200213203A1 (en) | Dynamic network health monitoring using predictive functions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180622 |