CN111935140A - Abnormal message identification method and device - Google Patents
Abnormal message identification method and device Download PDFInfo
- Publication number
- CN111935140A CN111935140A CN202010793851.1A CN202010793851A CN111935140A CN 111935140 A CN111935140 A CN 111935140A CN 202010793851 A CN202010793851 A CN 202010793851A CN 111935140 A CN111935140 A CN 111935140A
- Authority
- CN
- China
- Prior art keywords
- data
- message
- processing server
- abnormal
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses an abnormal message identification method and device, belonging to the technical field of artificial intelligence, wherein the method comprises the following steps: acquiring message data and determining a processing server corresponding to the message data; adding the message data into a message data set of the whole day of the day corresponding to the processing server; and identifying whether the message data are abnormal messages according to the daily message data set and a trained classification model corresponding to the processing server, wherein the classification model is obtained by taking a historical daily message data set corresponding to the processing server as training data and adopting a preset classification algorithm for training. The invention realizes the beneficial effect of accurately and rapidly identifying the abnormal message.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an abnormal message identification method and device.
Background
With the rapid development of internet innovation and the remarkable increase of service processing capacity, each service is processed by a plurality of processing servers, each processing server is responsible for processing one part of the service, and the processing servers frequently interact with each other through messages during service processing so as to complete specific services. At present, the data volume of interactive messages between each processing server is huge, and how to perform different knowledge aiming at the messages with huge data volume is a problem to be solved urgently in the prior art.
Disclosure of Invention
The present invention provides a method and an apparatus for identifying an abnormal packet, in order to solve the technical problems in the background art.
In order to achieve the above object, according to an aspect of the present invention, there is provided an abnormal packet identification method, including:
acquiring message data and determining a processing server corresponding to the message data;
adding the message data into a message data set of the whole day of the day corresponding to the processing server;
and identifying whether the message data are abnormal messages according to the daily message data set and a trained classification model corresponding to the processing server, wherein the classification model is obtained by taking a historical daily message data set corresponding to the processing server as training data and adopting a preset classification algorithm for training.
Optionally, the message data set of the whole day is divided into a plurality of data subsets according to a plurality of preset time intervals, and each data subset corresponds to one data item.
Optionally, the data item includes a time characteristic value and a data column, the time characteristic value is determined by a time interval of the corresponding data subset, and the data column is determined by all message data included in the corresponding data subset.
Optionally, the identifying the abnormal packet of the processing server according to the data set of the packet on the whole day of the day and the trained classification model corresponding to the processing server includes:
and identifying the abnormal messages of the processing server according to the data items of the data set of the messages on the whole day of the current day and the trained classification model corresponding to the processing server.
Optionally, the multiple time intervals are consecutive, and the size of each time interval is equal to a preset value.
Optionally, the method for identifying an abnormal packet further includes:
acquiring training data corresponding to each processing server, wherein the training data is a historical full-day message data set;
and training a classification model corresponding to each processing server according to the training data and the preset classification algorithm.
Optionally, the training of the classification model corresponding to each processing server according to the training data and the classification algorithm includes:
obtaining a value of a loss function according to the training data and a preset loss function;
normalizing the value of the loss function according to a Softmax classifier;
and optimizing the loss function by using a gradient descent algorithm to train a model.
Optionally, the message data includes: at least one of message receiving/sending time, port information, encoding type, message duration, message size, message protocol, routing information, message status, and message type.
In order to achieve the above object, according to another aspect of the present invention, there is provided an abnormal packet identifying apparatus, including:
the message data acquisition unit is used for acquiring message data and determining a processing server corresponding to the message data;
the data writing unit is used for adding the message data into a message data set of the whole day of the day corresponding to the processing server;
and the abnormal message identification unit is used for identifying whether the message data is an abnormal message according to the daily message data set of the current day and a trained classification model corresponding to the processing server, wherein the classification model is obtained by taking a historical daily message data set corresponding to the processing server as training data and adopting a preset classification algorithm for training.
In order to achieve the above object, according to another aspect of the present invention, there is also provided a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the above exception message identification method when executing the computer program.
In order to achieve the above object, according to another aspect of the present invention, there is also provided a computer-readable storage medium storing a computer program which, when executed in a computer processor, implements the steps in the above-mentioned abnormal packet identification method.
The invention has the beneficial effects that: the invention trains the classification model corresponding to each processing server through the historical full-day message data set, and then identifies the abnormal message in the current message data according to the trained classification model, thereby realizing the beneficial effect of accurately and rapidly identifying the abnormal message.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts. In the drawings:
FIG. 1 is a first flowchart of an abnormal message identification method according to an embodiment of the present invention;
FIG. 2 is a second flowchart of an abnormal message identification method according to an embodiment of the present invention;
FIG. 3 is a flow chart of model training according to an embodiment of the present invention;
fig. 4 is a first structural block diagram of an abnormal packet identification apparatus according to an embodiment of the present invention;
fig. 5 is a second structural block diagram of the abnormal packet identification apparatus according to the embodiment of the present invention;
FIG. 6 is a schematic diagram of a computer device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It should be noted that the terms "comprises" and "comprising," and any variations thereof, in the description and claims of the present invention and the above-described drawings, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 is a first flowchart of an abnormal packet identification method according to an embodiment of the present invention, and as shown in fig. 1, the abnormal packet identification method according to the embodiment includes steps S101 to S103.
Step S101, obtaining message data and determining a processing server corresponding to the message data.
In an optional embodiment of the present invention, each processing server performs message interaction through a gateway, and the gateway is configured to relay a message of each processing server. The step of obtaining the message data may specifically be that the gateway obtains the message data of each message interacted with each processing server in real time. Further, the gateway determines a processing server corresponding to each message data according to the IP address in the message data.
In an optional embodiment of the present invention, the IP addresses of the processing servers are recorded in the message data, and each processing server corresponds to a fixed IP address, and in this step, the processing server corresponding to the message data can be determined by using the IP address recorded in the message data.
In an optional embodiment of the present invention, the message data is a feature information set of the message, and one message corresponds to one group of message data. In an optional embodiment of the present invention, the message data may include: at least one of message receiving/sending time, port information, encoding type, message duration, message size, message protocol, routing information, message status, and message type. In a specific embodiment of the present invention, the message data specifically includes: message receiving/sending time, port information, coding type, message duration, message size, message protocol, routing information, message state, message type, load condition, receiving and sending speed and other information. In the embodiment of the present invention, each information in the message data is in a numerical form, for example, the message status represents different statuses by using different preset numerical values.
And step S102, adding the message data into a message data set of the whole day of the day corresponding to the processing server.
In an optional embodiment of the present invention, each processing server corresponds to one entire-day packet data set each day, and the entire-day packet data set is used to store all packet data of the corresponding processing server for one entire day. In an optional embodiment of the present invention, the full-day packet data set is used to store all packet data from 0 point to 24:00 point of a day, and each packet data is distinguished in the full-day packet data set according to the packet receiving/transmitting time corresponding to each packet data, for example, 8: 45, message data A, 8: 46 message data B, 8: 47 message data C.
In an optional embodiment of the present invention, the whole-day packet data set (including the historical whole-day packet data set) includes a plurality of data items, each packet data item corresponds to one data item, the data item is a one-dimensional array, the one-dimensional array represents the packet data corresponding to the time point, and the time point is the packet receiving/sending time corresponding to the packet data. Adding the message data into the message data set of the whole day corresponding to the processing server, and generating the data item corresponding to the message data.
The step may specifically be that the message data is added to the message data set of the whole day corresponding to the processing server on the same day according to the message receiving/sending time of the message data.
Step S103, identifying whether the message data is an abnormal message according to the whole-day message data set of the current day and a trained classification model corresponding to the processing server, wherein the classification model is obtained by taking a historical whole-day message data set corresponding to the processing server as training data and training by adopting a preset classification algorithm.
In an optional embodiment of the present invention, in this step, an abnormal packet of the processing server is identified according to a data item of the packet data set of the whole day of the day and a trained classification model corresponding to the processing server. Specifically, it may be determined whether the packet data is an abnormal packet by inputting the data item corresponding to the packet data into the trained classification model corresponding to the processing server.
In alternative embodiments of the present invention, the classification model may be a curve model or a numerical distribution model.
In another optional embodiment of the present invention, the whole-day message data set (including the historical whole-day message data set used for model training) is divided into a plurality of data subsets according to a plurality of preset time intervals, for example, each hour is a time interval, and the whole-day message data set is divided into 24 data subsets according to 24 hours a day. In the embodiment of the present invention, each data subset corresponds to a data item, the data item includes a time characteristic value and a data column, and the data column may be a column vector. The time characteristic value is determined by a time interval corresponding to the corresponding data subset, and specifically may be a time value determined by the time interval, or may be a sequence number corresponding to the time interval. The data column is determined by all the message data included in the corresponding data subset, and specifically, the data column may be obtained by summing each corresponding item of all the message data included in the data subset. In the embodiment of the present invention, the data column includes the same items as the items included in the message data.
In an embodiment of the present invention, the plurality of time intervals are consecutive, and the size of each time interval is equal to a preset value. Optionally, the preset value ranges from greater than or equal to 1 millisecond to less than or equal to 1 second. In a preferred embodiment of the present invention, the preset value is 1 millisecond, so that 24 hours a day includes 86400000 milliseconds, the message data set of the whole day is divided into 86400000 data subsets, each data subset records data of a corresponding millisecond, for example, for the message data of the message sent in the first millisecond of the day, the message data is recorded in the first data subset in the message data set of the whole day. Thus, the whole-day message data set contains 86400000 data items, the time characteristic value of each data item is a time value, and the unit can be millisecond, when only one message data is contained in the data subset, the data column of the data item corresponding to the data subset is the contained message data; and when the data subset only comprises two or more message data, summing corresponding items of each message data to obtain a data column. In the embodiment of the present invention, initially, a data column corresponding to each data subset in the message data set of the entire day is represented by 0.
Fig. 2 is a second flowchart of an abnormal packet identification method according to an embodiment of the present invention, as shown in fig. 2, in an alternative embodiment of the present invention, the training process of the classification model in step S103 includes step S201 and step S202.
Step S201, obtaining training data corresponding to each processing server, wherein the training data is a historical whole-day message data set.
In an embodiment of the invention, each training data comprises a plurality of data items.
Step S202, training out a classification model corresponding to each processing server according to the training data and the preset classification algorithm.
In the embodiment of the invention, the model training is carried out by adopting a preset classification algorithm according to the data items of the training data, and the classification models corresponding to the processing servers are obtained.
Fig. 3 is a flowchart of model training according to an embodiment of the present invention, and as shown in fig. 3, in an alternative embodiment of the present invention, the step S202 of training the classification model corresponding to each processing server according to the training data and the classification algorithm specifically includes steps S301 to S303.
Step S301, obtaining a loss function value according to the training data and a preset loss function.
Step S302, normalization processing is carried out on the value of the loss function according to a Softmax classifier.
Step S303, optimizing the loss function by using a gradient descent algorithm, and training a model.
In an optional embodiment of the present invention, the loss function includes a loss equation and a regularization penalty term, the loss equation includes an actual value in the training data and a pre-estimated value obtained by a one-dimensional pre-estimated value function equation, and the regularization penalty term is determined by a one-dimensional random value W in the one-dimensional pre-estimated value function equation.
In another optional embodiment of the present invention, the process of model training may specifically include the following steps:
A. forward propagation: and constructing a one-dimensional prediction function equation of y-W x _ data + b, wherein y is the value of a prediction tag, W can be a one-dimensional random value between-1 and 1, W can also be a random function, b is initialized to a constant value (can take the value of 0 or 1 and the like), x _ data is the time characteristic value of the data item, and y is the data in the data column of the predicted data item.
B. Obtaining a preset loss equation:
wherein the content of the first and second substances,
for the estimated value, Y, of the output of the above-mentioned one-dimensional predictor function equationiThe loss equation squares the difference between the predicted and actual values for the actual values, i.e., the values in the data columns of the data items in the training data, and averages the sum of the squares of all differences.
C. The value of the loss function is equal to the value of the constructed loss equation plus the value of the regularization penalty term. The regularization penalty term is formulated as: the squares of W in the one-dimensional estimation function equation are summed.
D. Constructing a Softmax classifier: normalizing the value of the loss function: the formula is to operate the value of the loss function first, and the formula is as follows: e.g. of the typexThe method comprises the steps of calculating a loss function, calculating a loss value formula of the loss function, wherein x is a value for calculating the loss function, adding values after the loss function exp is operated to be denominators, forming a group of one-dimensional functions by taking the value operated by each loss function exp as a numerator, and calculating the loss value formula of the one-dimensional functions to be y-log P, wherein P is each value of the one-dimensional functions.
E. This loss function is optimized using the Adam gradient descent algorithm, training the model.
F. And continuously carrying out model training through training data, and finally fitting a classification model, wherein the classification model can be a curve model or a numerical distribution model.
According to the embodiment, the classification models corresponding to the processing servers are trained through the historical full-day message data set, and then the abnormal messages in the current message data are identified according to the trained classification models, so that the abnormal messages are accurately and quickly identified, the daily routing inspection work efficiency is improved, and the safe and stable operation of the processing servers can be better.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
Based on the same inventive concept, an embodiment of the present invention further provides an abnormal packet identification apparatus, which can be used to implement the abnormal packet identification method described in the foregoing embodiment, as described in the following embodiment. Because the principle of solving the problem of the abnormal message identification apparatus is similar to that of the abnormal message identification method, the embodiment of the abnormal message identification apparatus can refer to the embodiment of the abnormal message identification method, and repeated parts are not described again. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 4 is a first structural block diagram of an abnormal packet identification apparatus according to an embodiment of the present invention, and as shown in fig. 4, the abnormal packet identification apparatus according to the embodiment of the present invention includes:
the message data acquisition unit 1 is used for acquiring message data and determining a processing server corresponding to the message data;
the data writing unit 2 is used for adding the message data into a message data set of the whole day corresponding to the processing server;
and the abnormal message identification unit 3 is configured to identify whether the message data is an abnormal message according to the daily message data set of the current day and a trained classification model corresponding to the processing server, where the classification model is obtained by using a historical daily message data set corresponding to the processing server as training data and training the training data by using a preset classification algorithm.
In an optional embodiment of the present invention, the whole-day packet data set is divided into a plurality of data subsets according to a plurality of preset time intervals; each data subset corresponds to a data item, the data item comprises a time characteristic value and a data column, the time characteristic value is determined by the time interval of the corresponding data subset, and the data column is determined by all message data contained in the corresponding data subset.
In an optional embodiment of the present invention, the abnormal packet identifying unit is specifically configured to identify the abnormal packet of the processing server according to a data item of the data set of the daily packet and a trained classification model corresponding to the processing server.
In an alternative embodiment of the present invention, the plurality of time intervals are consecutive, and the size of each time interval is equal to a preset value.
In an optional embodiment of the present invention, the message data includes: at least one of message receiving/sending time, port information, encoding type, message duration, message size, message protocol, routing information, message status, and message type.
Fig. 5 is a second structural block diagram of the abnormal packet identification apparatus according to the embodiment of the present invention, and as shown in fig. 5, the abnormal packet identification apparatus according to the embodiment of the present invention further includes:
a training data obtaining unit 4, configured to obtain training data corresponding to each processing server, where the training data is a historical full-day packet data set;
and the model training unit 5 is used for training the classification model corresponding to each processing server according to the training data and the preset classification algorithm.
In one embodiment of the present invention, the model training unit 5 includes:
the loss function module is used for obtaining a value of a loss function according to the training data and a preset loss function;
the classifier module is used for carrying out normalization processing on the value of the loss function according to a Softmax classifier;
and the loss function optimization module is used for optimizing the loss function by using a gradient descent algorithm to train the model.
To achieve the above object, according to another aspect of the present application, there is also provided a computer apparatus. As shown in fig. 6, the computer device comprises a memory, a processor, a communication interface and a communication bus, wherein a computer program that can be run on the processor is stored in the memory, and the steps of the method of the above embodiment are realized when the processor executes the computer program.
The processor may be a Central Processing Unit (CPU). The Processor may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or a combination thereof.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and units, such as the corresponding program units in the above-described method embodiments of the present invention. The processor executes various functional applications of the processor and the processing of the work data by executing the non-transitory software programs, instructions and modules stored in the memory, that is, the method in the above method embodiment is realized.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor, and the like. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and such remote memory may be coupled to the processor via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more units are stored in the memory and when executed by the processor perform the method of the above embodiments.
The specific details of the computer device may be understood by referring to the corresponding related descriptions and effects in the above embodiments, and are not described herein again.
In order to achieve the above object, according to another aspect of the present application, there is also provided a computer-readable storage medium storing a computer program which, when executed in a computer processor, implements the steps in the above-mentioned abnormal packet identification method. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD) or a Solid State Drive (SSD), etc.; the storage medium may also comprise a combination of memories of the kind described above.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (13)
1. An abnormal message identification method is characterized by comprising the following steps:
acquiring message data and determining a processing server corresponding to the message data;
adding the message data into a message data set of the whole day of the day corresponding to the processing server;
and identifying whether the message data are abnormal messages according to the daily message data set and a trained classification model corresponding to the processing server, wherein the classification model is obtained by taking a historical daily message data set corresponding to the processing server as training data and adopting a preset classification algorithm for training.
2. The abnormal message identification method of claim 1, wherein the daily message data set is divided into a plurality of data subsets according to a plurality of preset time intervals, and each data subset corresponds to one data item.
3. The abnormal packet identification method according to claim 2, wherein the data item comprises a time characteristic value and a data column, the time characteristic value is determined by a time interval of the corresponding data subset, and the data column is determined by all packet data included in the corresponding data subset.
4. The method according to claim 2, wherein the identifying the abnormal packet of the processing server according to the data set of the packet of the whole day of the day and the trained classification model corresponding to the processing server comprises:
and identifying the abnormal messages of the processing server according to the data items of the data set of the messages on the whole day of the current day and the trained classification model corresponding to the processing server.
5. The method according to claim 2, wherein the plurality of time intervals are consecutive, and the size of each time interval is equal to a predetermined value.
6. The method according to claim 1, further comprising:
acquiring training data corresponding to each processing server, wherein the training data is a historical full-day message data set;
and training a classification model corresponding to each processing server according to the training data and the preset classification algorithm.
7. The abnormal message identification method of claim 6, wherein the historical daily message data set is divided into a plurality of data subsets according to a plurality of preset time intervals, and each data subset corresponds to one data item.
8. The abnormal packet identification method according to claim 7, wherein the data item comprises a time characteristic value and a data column, the time characteristic value is determined by a time interval of the corresponding data subset, and the data column is determined by all packet data included in the corresponding data subset.
9. The method according to claim 6, wherein training the classification model corresponding to each processing server according to the training data and the classification algorithm includes:
obtaining a value of a loss function according to the training data and a preset loss function;
normalizing the value of the loss function according to a Softmax classifier;
and optimizing the loss function by using a gradient descent algorithm to train a model.
10. The abnormal packet identification method according to claim 1, wherein the packet data includes: at least one of message receiving/sending time, port information, encoding type, message duration, message size, message protocol, routing information, message status, and message type.
11. An abnormal message recognition apparatus, comprising:
the message data acquisition unit is used for acquiring message data and determining a processing server corresponding to the message data;
the data writing unit is used for adding the message data into a message data set of the whole day of the day corresponding to the processing server;
and the abnormal message identification unit is used for identifying whether the message data is an abnormal message according to the daily message data set of the current day and a trained classification model corresponding to the processing server, wherein the classification model is obtained by taking a historical daily message data set corresponding to the processing server as training data and adopting a preset classification algorithm for training.
12. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 10 when executing the computer program.
13. A computer-readable storage medium, in which a computer program is stored which, when executed in a computer processor, implements the method of any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010793851.1A CN111935140B (en) | 2020-08-10 | 2020-08-10 | Abnormal message identification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010793851.1A CN111935140B (en) | 2020-08-10 | 2020-08-10 | Abnormal message identification method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111935140A true CN111935140A (en) | 2020-11-13 |
| CN111935140B CN111935140B (en) | 2022-10-28 |
Family
ID=73307045
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010793851.1A Active CN111935140B (en) | 2020-08-10 | 2020-08-10 | Abnormal message identification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111935140B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112688950A (en) * | 2020-12-26 | 2021-04-20 | 中国农业银行股份有限公司 | Message classification method and device |
| CN112907351A (en) * | 2021-02-05 | 2021-06-04 | 中国工商银行股份有限公司 | Financial message abnormity identification method and device |
| CN112995155A (en) * | 2021-02-09 | 2021-06-18 | 中国工商银行股份有限公司 | Financial abnormal message identification method and device |
| CN113114679A (en) * | 2021-04-13 | 2021-07-13 | 中国工商银行股份有限公司 | Message identification method and device, electronic equipment and medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103514708A (en) * | 2013-10-13 | 2014-01-15 | 林兴志 | Logistics transportation intelligent short message information alarm device based on compass and GIS |
| CN106161066A (en) * | 2015-04-13 | 2016-11-23 | 中国移动通信集团福建有限公司 | A kind of log collection method and server |
| US20170289815A1 (en) * | 2016-03-31 | 2017-10-05 | Lenovo (Beijing) Limited | Malicious text message identification |
| CN108595643A (en) * | 2018-04-26 | 2018-09-28 | 重庆邮电大学 | Text character extraction and sorting technique based on more class node convolution loop networks |
| US20180288579A1 (en) * | 2017-03-29 | 2018-10-04 | Beijing Xiaomi Mobile Software Co., Ltd. | Short message identification method and device, and storage medium |
| CN109246027A (en) * | 2018-09-19 | 2019-01-18 | 腾讯科技（深圳）有限公司 | A kind of method, apparatus and terminal device of network operation |
| US20190109810A1 (en) * | 2011-05-12 | 2019-04-11 | Jeffrey Alan Rapaport | Social-topical adaptive networking (stan) system allowing for group based contextual transaction offers and acceptances and hot topic watchdogging |
| CN109688030A (en) * | 2019-02-26 | 2019-04-26 | 百度在线网络技术（北京）有限公司 | Message detecting method, device, equipment and storage medium |
| CN111061152A (en) * | 2019-12-23 | 2020-04-24 | 深圳供电局有限公司 | Attack recognition method based on deep neural network and intelligent energy power control device |
-
2020
- 2020-08-10 CN CN202010793851.1A patent/CN111935140B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190109810A1 (en) * | 2011-05-12 | 2019-04-11 | Jeffrey Alan Rapaport | Social-topical adaptive networking (stan) system allowing for group based contextual transaction offers and acceptances and hot topic watchdogging |
| CN103514708A (en) * | 2013-10-13 | 2014-01-15 | 林兴志 | Logistics transportation intelligent short message information alarm device based on compass and GIS |
| CN106161066A (en) * | 2015-04-13 | 2016-11-23 | 中国移动通信集团福建有限公司 | A kind of log collection method and server |
| US20170289815A1 (en) * | 2016-03-31 | 2017-10-05 | Lenovo (Beijing) Limited | Malicious text message identification |
| US20180288579A1 (en) * | 2017-03-29 | 2018-10-04 | Beijing Xiaomi Mobile Software Co., Ltd. | Short message identification method and device, and storage medium |
| CN108595643A (en) * | 2018-04-26 | 2018-09-28 | 重庆邮电大学 | Text character extraction and sorting technique based on more class node convolution loop networks |
| CN109246027A (en) * | 2018-09-19 | 2019-01-18 | 腾讯科技（深圳）有限公司 | A kind of method, apparatus and terminal device of network operation |
| CN109688030A (en) * | 2019-02-26 | 2019-04-26 | 百度在线网络技术（北京）有限公司 | Message detecting method, device, equipment and storage medium |
| CN111061152A (en) * | 2019-12-23 | 2020-04-24 | 深圳供电局有限公司 | Attack recognition method based on deep neural network and intelligent energy power control device |
Non-Patent Citations (2)
| Title |
|---|
| STEPHEN R.MOUNCE，RICHARD B.MOUNCE，JOBY B.BOXALL: "Novelty detection for time series data analysis in water distribution systems using support vector machines", 《JOURNAL OF HYDROINFORMATICS》 * |
| 宋若宁: "海量数据环境下的网络流量异常检测的研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112688950A (en) * | 2020-12-26 | 2021-04-20 | 中国农业银行股份有限公司 | Message classification method and device |
| CN112907351A (en) * | 2021-02-05 | 2021-06-04 | 中国工商银行股份有限公司 | Financial message abnormity identification method and device |
| CN112995155A (en) * | 2021-02-09 | 2021-06-18 | 中国工商银行股份有限公司 | Financial abnormal message identification method and device |
| CN113114679A (en) * | 2021-04-13 | 2021-07-13 | 中国工商银行股份有限公司 | Message identification method and device, electronic equipment and medium |
| CN113114679B (en) * | 2021-04-13 | 2023-03-24 | 中国工商银行股份有限公司 | Message identification method and device, electronic equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111935140B (en) | 2022-10-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111935140B (en) | Abnormal message identification method and device | |
| US10572828B2 (en) | Transfer learning and domain adaptation using distributable data models | |
| US10140572B2 (en) | Memory bandwidth management for deep learning applications | |
| US20210182254A1 (en) | Distributable model with biases contained within distributed data | |
| CN109586950B (en) | Network scene recognition method, network management device, network scene recognition system and storage medium | |
| CN117157628A (en) | System and method related to applied anomaly detection and contact center computing environments | |
| CN110263869B (en) | Method and device for predicting duration of Spark task | |
| CN111611351B (en) | Control method and device for online customer service session and electronic equipment | |
| CN114818446B (en) | Power service decomposition method and system facing 5G cloud edge terminal cooperation | |
| CN115562824A (en) | Computing resource cooperative scheduling system, method, device and storage medium | |
| CN111104954A (en) | Object classification method and device | |
| CN110796366A (en) | Quality difference cell identification method and device | |
| CN114610475A (en) | Training method of intelligent resource arrangement model | |
| CN107846402B (en) | BGP stability abnormity detection method and device and electronic equipment | |
| CN110084406B (en) | Load prediction method and device based on self-encoder and meta-learning strategy | |
| CN111445027B (en) | Training method and device for machine learning model | |
| US11321637B2 (en) | Transfer learning and domain adaptation using distributable data models | |
| CN109446020B (en) | Dynamic evaluation method and device of cloud storage system | |
| CN113407491A (en) | Data processing method and device | |
| CN113850390A (en) | Method, device, equipment and medium for sharing data in federal learning system | |
| WO2019113501A1 (en) | Transfer learning and domain adaptation using distributable data models | |
| CN111757115A (en) | Video stream processing method and device | |
| Oktug et al. | A prediction module for smart city IoT platforms | |
| Laalaoui et al. | Artificial intelligence applications in information and communication technologies | |
| WO2019071055A1 (en) | Improving a distributable model with distributed data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |