CN108965001B - Method and device for evaluating vehicle message data model - Google Patents
Method and device for evaluating vehicle message data model Download PDFInfo
- Publication number
- CN108965001B CN108965001B CN201810762222.5A CN201810762222A CN108965001B CN 108965001 B CN108965001 B CN 108965001B CN 201810762222 A CN201810762222 A CN 201810762222A CN 108965001 B CN108965001 B CN 108965001B
- Authority
- CN
- China
- Prior art keywords
- data
- message
- test
- abnormal
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000013499 data model Methods 0.000 title claims abstract description 18
- 238000012360 testing method Methods 0.000 claims abstract description 71
- 230000002159 abnormal effect Effects 0.000 claims abstract description 47
- 238000001514 detection method Methods 0.000 claims abstract description 31
- 238000012549 training Methods 0.000 claims abstract description 29
- 238000011156 evaluation Methods 0.000 claims abstract description 27
- 238000004088 simulation Methods 0.000 claims abstract description 11
- 238000012795 verification Methods 0.000 claims abstract description 10
- 230000015654 memory Effects 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 24
- 230000004913 activation Effects 0.000 claims description 12
- 230000008859 change Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 4
- 239000013598 vector Substances 0.000 claims description 4
- 206010000117 Abnormal behaviour Diseases 0.000 claims description 3
- 238000007477 logistic regression Methods 0.000 claims description 2
- 238000003745 diagnosis Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 8
- 238000013528 artificial neural network Methods 0.000 description 6
- 230000002441 reversible effect Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 4
- 230000000875 corresponding effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000005336 cracking Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000008034 disappearance Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000007787 long-term memory Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 230000006403 short-term memory Effects 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/40—Bus networks
- H04L2012/40208—Bus networks characterized by the use of a particular bus standard
- H04L2012/40215—Controller Area Network CAN
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/40—Bus networks
- H04L2012/40267—Bus for use in transportation systems
- H04L2012/40273—Bus for use in transportation systems the transportation system being a vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Small-Scale Networks (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses an evaluation method and a device of a vehicle message data model, wherein the method comprises the following steps: collecting vehicle message data with preset time length, and dividing the vehicle message data into different data sets; for data in each data set, setting a part of data as training data, setting a part of data as test data, and setting the rest data as verification data, wherein part of data in the test data is modified into simulation abnormal data; inputting training data into a long-time memory network to establish a message prediction model; testing the test data according to the message prediction model to obtain the detection rate and the false alarm rate of the simulated abnormal data; and evaluating the message prediction model according to the detection rate and the false alarm rate. Compared with the prior art, the method and the device can reduce the workload in the vehicle abnormal data detection process.
Description
Technical Field
The invention relates to the field of automobiles, in particular to an evaluation method and device of a vehicle message data model.
Background
With the development of technologies such as mobile internet, big data and cloud computing, the development speed of the automobile industry to intellectualization and networking is rapidly increased. Along with the continuous improvement of the automobile intelligent degree, the automobile network safety problem is gradually shown. In recent years, car hacker attack events occur frequently, and the car recall events cause great economic loss to car enterprises; personal privacy disclosure, threats existing after the automobile is controlled and the like also bring serious personal safety problems to automobile users.
A Controller Area Network (CAN) protocol is a serial communication protocol widely used in automobiles nowadays. Information between Electronic Control Units (ECUs) is transmitted in the form of data frames through a CAN bus. An ECU broadcasts a data message with a specific ID to the CAN network, and the ECU connected with the CAN selects whether to receive and respond the message according to the ID, thereby controlling the automobile to execute corresponding actions. The current CAN bus is generally lack of safety, and some hackers CAN send abnormal messages to the CAN by physically contacting an automobile communication channel, so that the monitoring and control of the automobile CAN be quickly and conveniently realized.
The traditional CAN network protection method mainly comprises the methods of data encryption and gateway authentication, CAN message time interval analysis, automobile safety architecture analysis based on a probability model and the like. The learning of the CAN message data needs to be analyzed through reverse decomposition. In practice, however, each vehicle has a different message data standard, and therefore, a lot of workload is brought by reversely analyzing data.
Disclosure of Invention
In view of the above, the present invention provides a method and an apparatus for evaluating a vehicle message data model, which solves at least one of the above-mentioned problems.
According to one aspect of the invention, there is provided a method of evaluating a vehicle message data model, the method comprising: collecting vehicle message data with preset time length, and dividing the vehicle message data into different data sets; for data in each data set, setting a part of data as training data, setting a part of data as test data, and setting the rest data as verification data, wherein part of data in the test data is modified into simulation abnormal data; inputting training data into a Long Short-Term Memory (LSTM) network to establish a message prediction model; testing the test data according to the message prediction model to obtain the detection rate and the false alarm rate of the simulated abnormal data; and evaluating the message prediction model according to the detection rate and the false alarm rate.
According to another aspect of the present invention, there is provided an apparatus for evaluating a vehicle message data model, the apparatus comprising: the system comprises a collecting unit, a processing unit and a processing unit, wherein the collecting unit is used for collecting vehicle message data with preset duration and dividing the vehicle message data into different data sets; the data setting unit is used for setting a part of data as training data, a part of data as test data and the rest of data as verification data for the data in each data set, wherein the part of data in the test data is modified into simulation abnormal data; the model establishing unit is used for inputting training data into a long-time memory network so as to establish a message prediction model; the test unit is used for testing the test data according to the message prediction model so as to obtain the detection rate and the false alarm rate of the simulated abnormal data; and the evaluation unit is used for evaluating the message prediction model according to the detection rate and the false alarm rate.
According to still another aspect of the present invention, there is provided a computer-readable storage medium storing a computer program for executing the above method.
Compared with the method for detecting the abnormal data by reversely analyzing the vehicle message data in the prior art, the method does not need a reverse analysis link, thereby reducing the workload of detecting the abnormal data of the vehicle.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent from the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
FIG. 1 is a flow chart of a vehicle message data model evaluation method according to an embodiment of the invention;
FIG. 2 is a flow chart of a vehicle message data model evaluation method based on a CAN network protocol according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a long term memory network unit according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of anomaly detection based on long and short term memory network according to an embodiment of the present invention;
FIG. 5 is a block diagram of a vehicle message data model evaluation device according to an embodiment of the present invention;
fig. 6 is a block diagram showing a detailed structure of a vehicle message data model evaluation apparatus according to an embodiment of the present invention.
Detailed Description
The present invention will be described below based on examples, but the present invention is not limited to only these examples.
Fig. 1 is a flowchart of a vehicle message data model evaluation method according to an embodiment of the invention, as shown in fig. 1, the method comprising:
step 101, collecting vehicle message data with preset duration, and dividing the vehicle message data into different data sets;
102, regarding data in each data set, setting a part of data as training data, setting a part of data as test data, and setting the rest data as verification data, wherein part of data in the test data is modified into simulated abnormal data;
step 103, inputting training data into a long-time memory network (LSTM) to establish a message prediction model;
step 104, testing the test data according to the message prediction model to obtain the detection rate and the false alarm rate of the simulated abnormal data; and
and 105, evaluating a message prediction model according to the detection rate and the false alarm rate.
Compared with the method for detecting the abnormal data by reversely analyzing the vehicle message data in the prior art, the method and the device for detecting the abnormal data have the advantage that a reverse analysis link is not needed, so that the workload of detecting the abnormal data of the vehicle is reduced.
Specifically, the step 101 of dividing the vehicle message data into different data sets includes: deleting and selecting the vehicle message data according to a preset rule; and dividing the deleted vehicle message data into different data sets according to the ID. The predetermined rule may be to determine whether to delete the corresponding data according to whether the data stream changes or not.
In step 104, testing the test data according to the message prediction model to obtain a detection rate and a false alarm rate of the simulated abnormal data specifically includes: testing the simulation abnormal data in the test data according to the message prediction model; drawing an ROC (receiver operating characteristic) Curve according to the test result and calculating AUC (Area Under the Curve); and obtaining the detection rate and the false alarm rate of the simulated abnormal data according to the ROC curve and the AUC.
Then, in step 105, setting an abnormal threshold value for simulating abnormal data according to the detection rate and the false alarm rate; and evaluating the message prediction model according to the anomaly threshold.
The LSTM network CAN effectively solve the problem of gradient disappearance of a recurrent neural network and realize long-distance data information memory, and because the CAN message original data is used as input, the content of the message information does not need to be understood, so that a reverse cracking link in the traditional CAN anomaly detection is omitted.
Fig. 2 is a flowchart of a vehicle message data model evaluation method based on a CAN network protocol according to an embodiment of the present invention, the method comprising the steps of:
s1, collecting the message data of the automobile CAN bus;
s2, observing the change of the message data stream in the ID, deleting the ID without change or with unobvious change, and dividing the rest data into different data sets according to the ID;
s3, independently dividing the data in each ID data set into a training set, a test set and a verification set;
s4, extracting partial data from the test set and modifying the data into simulated abnormal data;
s5, establishing a message prediction model based on the LSTM network: CAN message data in a training set is used as network input, because the message data are high-dimensional binary vectors, the binary input is projected into a real-value state space through a linear embedding layer, and a sequence { x is input1,x2,x3,……xNIn which x isiIs a 64-bit vector; in the training of each time step, the network uses the message data xiFor input, with the following message sequence yi＝xi+1As a training target; testing test set data by adopting different network layer numbers, hidden units, activation functions and other parameters through tests, and selecting the optimal parameters;
s6, applying a loss function of logistic regression as a loss function of model evaluation;
s7: calculating and evaluating the detection rate and the false alarm rate of each abnormal behavior through an ROC curve and an AUC;
s8: training a network for data in each ID independently, and testing and evaluating;
s9: setting an evaluation threshold value and evaluating the detection method.
Wherein the collecting data in S1 includes: the On-Board Diagnostic (OBD) interface is connected with the CAN bus of the automobile and connected with the message transceiver, so that the data packet broadcasted On the CAN bus CAN be acquired. The data packets can be observed and collected in real time by using the vessel Spy software.
In S3, a data set is truncated to have 70% of the data amount as a training set, 10% as a verification set, and the remaining 20% as a test set.
In S4, since it is difficult to obtain a large amount of actual attack data in practical application, it is necessary to simulate an abnormal situation by modifying the obtained message data; meanwhile, because the data information used in the method does not relate to time information, simulation is realized only by modifying the sequence information of the data, and the applied simulation data comprises the actual physical scene. The test set is divided into five equal parts, one part is normal data, and the other four parts are respectively modified into the following attack forms. The abnormal conditions simulated by the embodiment of the invention comprise:
1. interleaving the subsequences: constructed by interleaving two normal subsequences from different time points. This situation simulates the situation where two senders race to send commands on the bus at the same rate, and most attacks are of this type.
2. Loss: a normal subsequence has several words deleted in the middle. This simulates an attack in which the ECU is suppressed and then quickly replaced by a simulator that sends normal data packets.
3. Discontinuity: the sub-sequence is constructed by extracting the first half from one time segment and the second half from another time point. This jump simulates an attack where the attacked ECU suddenly switches from sending normal traffic to legitimate traffic under certain circumstances.
4. Data heterogeneity: most IDs do not use all 64 bits. In unusual cases, for example, two of the normally unused bits change from 0 to 1. This situation simulates the use of hidden command and control channels embedded in normal traffic.
In the network structure of S5, the bit sequence is first converted by two non-recursive hidden layers, each layer containing 128 cells, and the activation function selects the tanh activation function. The output of the linear layer is used as the input of a two-layer long-and-short time memory network, each layer comprises 512 units, and the tanh function is used as an activation function. The final linear output layer contains 64 cells, normalizing the output values between 0 and 1 using the sigmoid function as the activation function.
Fig. 3 is a schematic diagram of the structure of an LSTM network unit according to an embodiment of the present invention, and as shown in fig. 3, the first step of the LSTM network is to determine what information of the previous state can enter a new unit, and this determination is controlled by a "forgetting gate" through a sigmoid function, and is based on the output h at the previous timet-1And the current input xtGenerating a f of 0 to 1tValue, determining whether to let the previous timeLearned information Ct-1By, or partially by, passing, wherein,
ft＝σ(Wf×[ht-1,xt]+bf) (1)
the second step generates new information to be updated, which is determined by sigmoid via "input gate" and then new candidate value is generated by tanh
Is added to the new cell state in which,
it＝σ(Wi×[ht-1,xt]+bi) (2)
combining the first step and the second step, updating the state of the previous unit: first by associating the old cell state with ftMultiplying and discarding the unwanted information
The addition results in the following candidate values,
determining the output of the model in the last step, firstly obtaining an initial output through a sigmoid function, and then obtaining C through tanhtThe value is controlled between-1 and 1, and then multiplied by the output obtained by sigmoid to obtain model output,
Ot＝σ(Wo[ht-1,xt]+bo) (5)
ht＝Ot×tanh(Ct) (6)
wherein h ist-1Represents the last time output, xtRepresents the current input, W represents the weighting factor, b represents the bias, σ represents the sigmoid activation function, and tanh represents the tanh activation function.
The following example is given below in conjunction with the schematic diagram of the LSTM neural network-based anomaly detection shown in fig. 4.
The method comprises the following steps: the transceiver is connected through the OBD interface, and the Vehicle Spy software is used for collecting data of the CAN bus message data of a certain automobile for 19 hours.
Step two: the number of the collected data comprises 20 IDs, the change of the message data stream is observed, 3 IDs which have no change or have no obvious change are deleted, and the rest data are divided into 17 data sets according to the IDs.
Step three: the data in each ID dataset was partitioned separately, with 13 hours of data being the training set, 4 hours of data for the test set, and 2 hours of data for the validation set.
Step four: dividing the 4-hour test set into five equal parts, wherein one part is normal data, and the rest data are respectively modified into: interleaving subsequences, loss, discontinuity and data heterogeneity.
Step five: as shown in fig. 4, a training set sequence in an ID dataset is input to the network, and the training data consists of 20 word-long subsequences, presented in 128 batches. The loss function is calculated using a logarithmic loss function, which is a binary logarithmic loss between the output and the next data word in the sequence; training is stopped if the training data fails to improve after 5 consecutive iterations to verify performance.
Step six: and (3) drawing an ROC curve for each analog abnormal data and calculating AUC, and testing the detection rate and the false alarm rate of the model for each abnormal behavior.
Step seven: a network is trained separately for each of the remaining IDs and tested and evaluated separately.
Step eight: and setting an anomaly threshold value for each anomaly according to the AUC data, and evaluating the detection method.
Compared with the prior art, the embodiment of the invention CAN detect the sequence data abnormity on the automobile CAN bus by using the LSTM neural network, the LSTM neural network method has the advantage of no need of knowing the knowledge of a specific protocol, and the LSTM neural network method has effective performance in detecting a series of abnormal conditions corresponding to known attacks on the CAN bus.
Based on similar inventive concepts, the embodiment of the invention further provides an evaluation device of the vehicle message data model, and the method can be preferably applied.
Fig. 5 is a block diagram of a vehicle message data model evaluation apparatus according to an embodiment of the present invention, as shown in fig. 5, the apparatus including: an acquisition unit 501, a data setting unit 502, a model establishing unit 503, a testing unit 504 and an evaluation unit 505, wherein:
the acquisition unit 501 is configured to acquire vehicle message data of a predetermined duration, and divide the vehicle message data into different data sets;
a data setting unit 502, configured to set, for data in each data set, a part of the data as training data, a part of the data as test data, and the rest of the data as verification data, where part of the data in the test data is modified into simulated abnormal data;
a model establishing unit 503, configured to input training data into an LSTM network to establish a message prediction model;
the testing unit 504 is configured to test the test data according to the message prediction model to obtain a detection rate and a false alarm rate of the simulated abnormal data; and
and an evaluation unit 505, configured to evaluate the message prediction model according to the detection rate and the false alarm rate.
Compared with the method for detecting abnormal data by reversely analyzing the vehicle message data in the prior art, the method and the device for detecting abnormal data in the vehicle message data have the advantages that a reverse analysis link is not needed, so that the workload of detecting the abnormal data of the vehicle is reduced.
Specifically, fig. 6 shows a specific structural block diagram of the evaluation device of the vehicle message data model, and the above units are described in detail below with reference to fig. 6.
As shown in fig. 6, the acquisition unit 501 includes: the collection module 5011, the deletion module 5012, and the classification module 5013, wherein: the acquisition module 5011 is used for acquiring vehicle message data with a preset time length; a deleting module 5012, configured to delete the vehicle message data according to a predetermined rule; the classification module 5013 is configured to classify the deleted vehicle message data into different data sets according to the ID. The predetermined rule may be to determine whether to delete the corresponding data according to whether the data stream changes or not.
The test unit 504 includes: a test module 5041, a curve plotting module 5042, and a test result obtaining module 5043, wherein: the test module 5041 is used for testing the simulation abnormal data in the test data according to the message prediction model; a curve drawing module 5042, configured to draw an ROC curve according to the test result and calculate AUC; and the test result obtaining module 5043 is used for obtaining the detection rate and the false alarm rate of the simulated abnormal data according to ROC and AUC. The simulated anomaly data here may be: interleaving subsequences, loss, discontinuity, data anomalies, etc.
The above-described evaluation unit 505 includes: an anomaly threshold setting module 5051 and an evaluation module 5052, wherein: an abnormal threshold setting module 5051, configured to set an abnormal threshold of the analog abnormal data according to the detection rate and the false alarm rate; an evaluation module 5052 is used for evaluating the message prediction model according to the anomaly threshold.
Because the principle of solving the problems of the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and repeated details are not repeated.
An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program for executing the above method is stored.
In summary, the LSTM network effectively solves the problem of gradient disappearance of the recurrent neural network, realizes the memory of long-distance data information, and simultaneously, adopts the CAN message original data as input, and does not need to understand the content of the message information, so that the reverse cracking link in the conventional CAN anomaly detection is omitted, and the embodiment of the invention CAN be conveniently transplanted and applied to any automobile using the CAN network as a protocol.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (2)
1. The vehicle message data model evaluation method based on the CAN network protocol comprises the following steps:
s1, collecting the message data of the automobile CAN bus;
s2, observing the change of the message data stream in the ID, deleting the ID without change or with unobvious change, and dividing the rest data into different data sets according to the ID;
s3, independently dividing the data in each ID data set into a training set, a test set and a verification set;
s4, extracting partial data from the test set and modifying the data into simulated abnormal data;
s5, establishing a message prediction model based on the LSTM network: taking CAN message data in a training set as network input, wherein the message data is a high-dimensional binary vector, projecting the binary input into a real-valued state space through a linear embedding layer, and inputting a sequence { x1, x2, x3, … … xN }, wherein xi is a 64-bit vector; in the training of each time step, the network takes message data xi as input, and takes a subsequent message sequence yi ═ xi +1 as a training target; testing test set data by adopting different network layer numbers, hidden units, activation functions and other parameters through tests, and selecting the optimal parameters;
s6, applying a loss function of logistic regression as a loss function of model evaluation;
s7: calculating and evaluating the detection rate and the false alarm rate of each abnormal behavior through an ROC curve and an AUC;
s8: training a network for data in each ID independently, and testing and evaluating;
s9: setting an evaluation threshold value, and evaluating the detection method;
wherein the collecting data in S1 includes: the vehicle-mounted diagnosis system interface is connected with a CAN bus of an automobile and connected with a message transceiver to obtain a data packet broadcasted on the CAN bus; the data packet can be observed and collected in real time by using the vessel Spy software;
in S3, sequentially intercepting 70% of data volume in a data set as a training set, 10% as a verification set and the rest 20% as a test set;
in S4, an abnormal situation is simulated by modifying the acquired message data; the simulation is realized only by modifying the sequence information of the data, and the applied simulation data contains the actual physical scene; dividing the test set into five equal parts, wherein one part is normal data, and the other four parts are respectively modified into the following attack forms; the abnormal conditions include:
interleaving the subsequences: constructed by interleaving two normal subsequences from different time points;
loss: a plurality of words are deleted in the middle of a normal subsequence;
discontinuity: the sub-sequence is constructed by extracting the first half from one time segment and the second half from another time point;
data heterogeneity: most IDs do not use all 64 bits;
in the network structure of S5, a bit sequence is firstly converted by two non-recursive hidden layers, each layer comprises 128 units, and an activation function selects a tanh activation function; the output of the linear layer is used as the input of a two-layer long-and-short time memory network, each layer comprises 512 units, and a tanh function is used as an activation function; the final linear output layer contains 64 cells, normalizing the output values between 0 and 1 using the sigmoid function as the activation function.
2. An apparatus for evaluating a vehicle message data model, wherein the method of claim 1 is applied, comprising: acquisition unit, data set up unit, model establishment unit, test unit and evaluation unit, wherein:
the system comprises a collecting unit, a processing unit and a processing unit, wherein the collecting unit is used for collecting vehicle message data with preset duration and dividing the vehicle message data into different data sets;
the data setting unit is used for setting a part of data as training data, a part of data as test data and the rest of data as verification data for the data in each data set, wherein the part of data in the test data is modified into simulation abnormal data;
the model establishing unit is used for inputting the training data into the LSTM network to establish a message prediction model;
the test unit is used for testing the test data according to the message prediction model so as to obtain the detection rate and the false alarm rate of the simulated abnormal data; and
the evaluation unit is used for evaluating the message prediction model according to the detection rate and the false alarm rate;
establishing a message prediction model based on an LSTM network through a model establishing unit according to vehicle message data acquired by an acquisition unit, and then testing the vehicle message data by a testing unit according to the model to obtain abnormal data detection rate and false alarm rate in the vehicle message data;
the above-mentioned collection unit includes: collection module, select module and classification module delete, wherein: the acquisition module is used for acquiring vehicle message data with preset time; the deleting module is used for deleting the vehicle message data according to a preset rule; the classification module is used for classifying the deleted vehicle message data into different data sets according to the ID; the predetermined rule is to determine whether to delete the corresponding data according to whether the data stream changes or not;
the test unit includes: the test module, the curve drawing module and the test result obtaining module are provided, wherein: the test module is used for testing the simulation abnormal data in the test data according to the message prediction model; the curve drawing module is used for drawing an ROC curve according to the test result and calculating AUC; the test result acquisition module is used for acquiring the detection rate and the false alarm rate of the simulated abnormal data according to the ROC and the AUC; the simulated anomaly data here are: interleaving subsequences, lost, discontinuous, data alien;
the above evaluation unit includes: an anomaly threshold setting module and an evaluation module, wherein: the abnormal threshold setting module is used for setting an abnormal threshold of the analog abnormal data according to the detection rate and the false alarm rate; and the evaluation module is used for evaluating the message prediction model according to the abnormal threshold value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810762222.5A CN108965001B (en) | 2018-07-12 | 2018-07-12 | Method and device for evaluating vehicle message data model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810762222.5A CN108965001B (en) | 2018-07-12 | 2018-07-12 | Method and device for evaluating vehicle message data model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108965001A CN108965001A (en) | 2018-12-07 |
| CN108965001B true CN108965001B (en) | 2020-08-25 |
Family
ID=64483004
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810762222.5A Expired - Fee Related CN108965001B (en) | 2018-07-12 | 2018-07-12 | Method and device for evaluating vehicle message data model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108965001B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111835695B (en) * | 2019-04-23 | 2021-06-25 | 华东师范大学 | Vehicle-mounted CAN bus intrusion detection method based on deep learning |
| CN110505134B (en) * | 2019-07-04 | 2021-10-01 | 国家计算机网络与信息安全管理中心 | Internet of vehicles CAN bus data detection method and device |
| CN110445758A (en) * | 2019-07-05 | 2019-11-12 | 华东师范大学 | The vehicle-mounted CAN bus network abnormal deviation data examination method and system of network are generated based on confrontation |
| CN110826054B (en) * | 2019-11-05 | 2022-07-15 | 哈尔滨工业大学 | Vehicle-mounted CAN bus intrusion detection method based on message data field characteristics |
| CN112787984B (en) * | 2019-11-11 | 2023-11-14 | 厦门雅迅网络股份有限公司 | Vehicle-mounted network anomaly detection method and system based on correlation analysis |
| CN111770069B (en) * | 2020-06-17 | 2022-02-15 | 北京航空航天大学 | Vehicle-mounted network simulation data set generation method based on intrusion attack |
| CN111866017B (en) * | 2020-07-29 | 2022-09-16 | 北京天融信网络安全技术有限公司 | Method and device for detecting abnormal frame interval of CAN bus |
| CN111880983B (en) * | 2020-08-04 | 2021-12-14 | 北京天融信网络安全技术有限公司 | CAN bus abnormality detection method and device |
| CN112327794B (en) * | 2020-10-15 | 2023-05-09 | 无锡沃尔福汽车技术有限公司 | OBD robustness test evaluation method and device |
| CN112905213B (en) * | 2021-03-26 | 2023-08-08 | 中国重汽集团济南动力有限公司 | Method and system for realizing ECU (electronic control Unit) refreshing parameter optimization based on convolutional neural network |
| CN113660137B (en) * | 2021-08-13 | 2024-03-22 | 杭州安恒信息技术股份有限公司 | Vehicle-mounted network fault detection method and device, readable storage medium and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7475426B2 (en) * | 2001-11-30 | 2009-01-06 | Lancope, Inc. | Flow-based detection of network intrusions |
| CN106411956A (en) * | 2016-12-02 | 2017-02-15 | 北京奇虎科技有限公司 | Method and device for analyzing automobile bus safety |
| CN106487630A (en) * | 2016-12-02 | 2017-03-08 | 北京奇虎科技有限公司 | A kind of method and apparatus that vehicle safety is detected based on test case |
| CN107241251A (en) * | 2017-06-16 | 2017-10-10 | 龙海特尔福汽车电子研究所有限公司 | The software implementation method of multichannel CAN message real-time reception |
| CN107454117A (en) * | 2017-09-30 | 2017-12-08 | 中国联合网络通信集团有限公司 | The intrusion detection method and system of a kind of car networking |
-
2018
- 2018-07-12 CN CN201810762222.5A patent/CN108965001B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7475426B2 (en) * | 2001-11-30 | 2009-01-06 | Lancope, Inc. | Flow-based detection of network intrusions |
| CN106411956A (en) * | 2016-12-02 | 2017-02-15 | 北京奇虎科技有限公司 | Method and device for analyzing automobile bus safety |
| CN106487630A (en) * | 2016-12-02 | 2017-03-08 | 北京奇虎科技有限公司 | A kind of method and apparatus that vehicle safety is detected based on test case |
| CN107241251A (en) * | 2017-06-16 | 2017-10-10 | 龙海特尔福汽车电子研究所有限公司 | The software implementation method of multichannel CAN message real-time reception |
| CN107454117A (en) * | 2017-09-30 | 2017-12-08 | 中国联合网络通信集团有限公司 | The intrusion detection method and system of a kind of car networking |
Non-Patent Citations (2)
| Title |
|---|
| "Evaluating performance of long short-term memory recurrent neural networks on intrusion detection data";Ralf C. Staudemeyer，Christian W. Omlin;《Proceedings of the South African Institute of Computer Scientists and Information Technologists Conference》;20131009;第219页右栏第11行-第222页左栏第30行，图2，表2 * |
| "入侵检测系统在车联网中的应用研究";梁俊威;《CNKI中国优秀硕士论文工程科技Ⅱ辑》;20170715;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108965001A (en) | 2018-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108965001B (en) | Method and device for evaluating vehicle message data model | |
| US11689549B2 (en) | Continuous learning for intrusion detection | |
| CN112839034B (en) | Network intrusion detection method based on CNN-GRU hierarchical neural network | |
| CN109922032B (en) | Method, device, equipment and storage medium for determining risk of logging in account | |
| CN106776842B (en) | Multimedia data detection method and device | |
| CN111783442A (en) | Intrusion detection method, device, server and storage medium | |
| Desta et al. | ID sequence analysis for intrusion detection in the CAN bus using long short term memory networks | |
| CN113079167B (en) | Internet of vehicles intrusion detection method and system based on deep reinforcement learning | |
| CN109656818B (en) | Fault prediction method for software intensive system | |
| CN114330487A (en) | Wireless network security situation assessment method based on BIPMU | |
| CN113553624A (en) | WGAN-GP privacy protection system and method based on improved PATE | |
| CN111898129B (en) | Malicious code sample screener and method based on Two-Head anomaly detection model | |
| CN113704082A (en) | Model evaluation method and device, electronic equipment and storage medium | |
| CN116015932A (en) | Intrusion detection network model generation method and data flow intrusion detection method | |
| CN113780382A (en) | AE and PMU-based high-efficiency network security situation assessment method | |
| CN117319223A (en) | Digital twinning technology-based pilot cockpit visualization method and system | |
| Kang et al. | A transfer learning based abnormal can bus message detection system | |
| CN115758337A (en) | Back door real-time monitoring method based on timing diagram convolutional network, electronic equipment and medium | |
| CN115314239A (en) | Analysis method and related equipment for hidden malicious behaviors based on multi-model fusion | |
| CN112329908A (en) | Image generation method for neural network model test | |
| CN108964969B (en) | High-speed railway signal system flow prediction method based on hybrid neural network and AR model | |
| Chen et al. | Adversarial sample detection via channel pruning | |
| CN116016298B (en) | 5G communication protocol anomaly detection method based on hidden semi-Markov model | |
| CN117891566B (en) | Reliability evaluation method, device, equipment, medium and product of intelligent software | |
| CN116886448B (en) | DDoS attack alarm studying and judging method and device based on semi-supervised learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200825 |