CN117009785A - Security monitoring method, device, server and system - Google Patents
Security monitoring method, device, server and system Download PDFInfo
- Publication number
- CN117009785A CN117009785A CN202310912771.7A CN202310912771A CN117009785A CN 117009785 A CN117009785 A CN 117009785A CN 202310912771 A CN202310912771 A CN 202310912771A CN 117009785 A CN117009785 A CN 117009785A
- Authority
- CN
- China
- Prior art keywords
- safety
- data
- monitoring
- safety monitoring
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 185
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000012549 training Methods 0.000 claims abstract description 65
- 238000013528 artificial neural network Methods 0.000 claims abstract description 55
- 238000005315 distribution function Methods 0.000 claims abstract description 21
- 238000012806 monitoring device Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 5
- 230000007613 environmental effect Effects 0.000 abstract description 11
- 238000013527 convolutional neural network Methods 0.000 description 22
- 230000000875 corresponding effect Effects 0.000 description 21
- 230000006870 function Effects 0.000 description 21
- 239000011159 matrix material Substances 0.000 description 10
- 210000002569 neuron Anatomy 0.000 description 9
- 239000013598 vector Substances 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000013507 mapping Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000013210 evaluation model Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000014509 gene expression Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000000844 transformation Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- CURLTUGMZLYLDI-UHFFFAOYSA-N Carbon dioxide Chemical compound O=C=O CURLTUGMZLYLDI-UHFFFAOYSA-N 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- UGFAIRIUMAVXCW-UHFFFAOYSA-N Carbon monoxide Chemical compound [O+]#[C-] UGFAIRIUMAVXCW-UHFFFAOYSA-N 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 229910002092 carbon dioxide Inorganic materials 0.000 description 1
- 239000001569 carbon dioxide Substances 0.000 description 1
- 229910002091 carbon monoxide Inorganic materials 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000007786 learning performance Effects 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/15—Correlation function computation including computation of convolution operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Computational Mathematics (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Alarm Systems (AREA)
Abstract
The embodiment of the invention provides a safety monitoring method, a safety monitoring device, a safety monitoring server and a safety monitoring system, wherein the safety monitoring method comprises the following steps: receiving safety monitoring data acquired by a data acquisition terminal; inputting the safety monitoring data into a multi-element Gaussian distribution function to extract characteristic data, and inputting the extracted characteristic data into a trained safety monitoring model, wherein the safety monitoring model is obtained by training a deep neural network by adopting the characteristic data; and after the positions of the clustering centers output by the safety monitoring model are corresponding to the corresponding safety levels, sending the corresponding safety levels to the monitoring terminal, so that a user obtains the safety levels from the monitoring terminal. The invention realizes the purpose of monitoring the safety state of the household living environment and improves the living safety of the household environment; the comprehensive safety of the whole is evaluated by integrating a plurality of environmental factors, and a personalized intelligent monitoring scheme conforming to personal environmental habits is provided.
Description
Technical Field
The embodiment of the invention relates to the technical field of the Internet of things, in particular to a safety monitoring method, a safety monitoring device, a safety monitoring server and a safety monitoring system.
Background
At present, for monitoring the living environment of a household, a household safety monitoring device such as a temperature sensor and/or a safety alarm device such as a gas concentration alarm and the like are/is used for monitoring the safety state of the living environment in real time, and when the alarm device sends out an alarm signal, the alarm signal is processed in time according to the alarm signal.
However, after the safety alarm device sends out an alarm signal, serious safety accidents occur due to the fact that the safety alarm device cannot process the alarm signals in time. Moreover, safety monitoring can only monitor and alarm different environmental factors, and can not integrate multiple environmental factors to evaluate the overall safety, and can not provide a personalized monitoring scheme conforming to personal environmental habits. Therefore, it is needed to provide a safer monitoring method conforming to personal environment habit so as to ensure the living safety of the home environment.
Disclosure of Invention
The embodiment of the invention provides a safety monitoring method, a safety monitoring device, a safety monitoring server and a safety monitoring system, provides an intelligent monitoring scheme for predicting the safety state of a household living environment, improves the living safety of the household environment and has a personalized prediction function.
In a first aspect, an embodiment of the present invention provides a security monitoring method, including:
receiving safety monitoring data acquired by a data acquisition terminal;
inputting the safety monitoring data into a multi-element Gaussian distribution function to extract characteristic data, and inputting the extracted characteristic data into a trained safety monitoring model, wherein the safety monitoring model is obtained by training a deep neural network by adopting the characteristic data;
and after the positions of the clustering centers output by the safety monitoring model are corresponding to the corresponding safety levels, sending the corresponding safety levels to the monitoring terminal, so that a user obtains the safety levels from the monitoring terminal.
Optionally, the training process of the safety monitoring model includes:
inputting the safety monitoring data into a multi-element Gaussian distribution function to perform feature extraction so as to obtain a feature data training set; using the first data set in the characteristic data training set, realizing unsupervised pre-training of the deep neural network through an automatic encoder, and taking parameters of the deep neural network obtained by pre-training as initialization parameters; and on the basis of the initialization parameters, training and fine-tuning parameters of the deep neural network by utilizing a second data set in the characteristic data training set.
Optionally, the training fine adjustment for parameters of the deep neural network includes: the parameters of the deep neural network are subjected to training fine adjustment through a forward propagation stage and a backward propagation stage.
Optionally, the deep neural network is a deep neural network with a multi-layer convolutional network structure, and comprises a convolutional layer and a downsampling layer.
Optionally, before or during the step of inputting the safety monitoring data into the multivariate gaussian distribution function to extract the feature data, the safety monitoring method of the embodiment of the present invention further includes:
receiving prompt information sent by a data acquisition terminal, wherein the prompt information comprises a prompt type and a processing guide;
and transmitting the prompt information to a monitoring terminal, so that a user obtains the prompt type and the processing guide from the monitoring terminal, and manages the target environment according to the prompt type and the processing guide.
Optionally, after the cluster center positions output by the safety monitoring model correspond to the corresponding safety levels and are sent to the monitoring terminal, the method further comprises the steps of:
and sending the security level to a third party management platform, so that a manager of the third party management platform obtains the security level from the third party management platform to manage the security state of the target environment.
In a second aspect, an embodiment of the present invention provides a safety monitoring device, including:
the receiving module is used for receiving the safety monitoring data acquired by the data acquisition terminal;
the prediction module is used for inputting the safety monitoring data into a multi-element Gaussian distribution function to extract characteristic data, and inputting the extracted characteristic data into a trained safety monitoring model, wherein the safety monitoring model is obtained by training a deep neural network by adopting the characteristic data;
and the sending module is used for sending the cluster center positions output by the safety monitoring model to the monitoring terminal after corresponding to the corresponding safety levels, so that a user obtains the safety levels from the monitoring terminal.
In a third aspect, an embodiment of the present invention provides a server, including: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executes computer-executable instructions stored by the memory to cause the at least one processor to perform the security monitoring method as provided in the first aspect above.
In a fourth aspect, an embodiment of the present invention provides a security monitoring system, including: at least one data acquisition terminal, a server and at least one monitoring terminal;
the data acquisition terminal is used for acquiring safety monitoring data of the target environment;
the server is used for receiving the safety monitoring data acquired by the data acquisition terminal and executing the safety monitoring method provided by the first aspect;
and the monitoring terminal is used for receiving the security level of the target environment from the server and outputting the security level to a user.
In a fifth aspect, an embodiment of the present invention provides a computer readable storage medium having stored therein computer executable instructions which, when executed by a processor, implement a security monitoring method as provided in the first aspect above.
The safety monitoring method, the safety monitoring device, the safety monitoring server and the safety monitoring system provided by the embodiment of the invention receive the safety monitoring data acquired by the data acquisition terminal; inputting the safety monitoring data into a multi-element Gaussian distribution function to extract characteristic data, and inputting the extracted characteristic data into a trained safety monitoring model, wherein the safety monitoring model is obtained by training a deep neural network by adopting the characteristic data; and after the positions of the clustering centers output by the safety monitoring model are corresponding to the corresponding safety levels, sending the corresponding safety levels to the monitoring terminal, so that a user obtains the safety levels from the monitoring terminal. According to the embodiment of the invention, the Gaussian distribution theory and the deep neural network are applied to safety monitoring data analysis, the deep neural network for learning the key characteristics of the safety time sequence data in an unsupervised learning mode is provided, a safety state evaluation model is built based on the multi-element Gaussian distribution theory and the deep neural network, and the data characteristics obtained based on the multi-element Gaussian distribution theory are used as the input of the evaluation model, so that the aim of monitoring the safety state of the household living environment is fulfilled, and the living safety of the household environment is improved; the comprehensive safety of the whole is evaluated by integrating a plurality of environmental factors, and a personalized intelligent monitoring scheme conforming to personal environmental habits is provided.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it will be obvious that the drawings in the following description are some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort to a person skilled in the art.
FIG. 1 is a schematic diagram of a security monitoring system according to an embodiment of the present invention;
FIG. 2 is a flowchart of a security monitoring method according to an embodiment of the present invention
FIG. 3 is a flowchart further illustrating step S202 in FIG. 2 according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a convolutional neural network according to an embodiment of the present invention;
FIG. 5 is a second flowchart of a security monitoring method according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of a safety monitoring device according to an embodiment of the present invention;
fig. 7 is a schematic hardware structure of a server according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The monitoring of the living environment of the family mostly depends on the household safety monitoring devices such as a temperature sensor and/or a gas concentration alarm of the safety alarm device, etc., so as to monitor the safety state of the living environment in real time, and when the alarm device sends out an alarm signal, the alarm device processes the alarm signal in time. However, after the safety alarm device sends out an alarm signal, serious safety accidents occur due to the fact that the safety alarm device cannot process the alarm signals in time. Meanwhile, the safety monitoring in the prior art can only monitor and alarm different kinds of environmental factors, can not integrate multiple environmental factors to evaluate the overall safety, and can not provide a personalized monitoring scheme conforming to personal environmental habits.
In order to solve the technical problems, the embodiment of the invention receives the safety monitoring data sent by the data acquisition terminal; inputting the safety monitoring data into a trained safety monitoring model to obtain a corresponding safety level; the security level is sent to the monitoring terminal, so that a manager manages the security state of the target environment according to the security level displayed by the monitoring terminal, an intelligent monitoring scheme is provided for predicting the security state of the household living environment, and living security of the household environment is improved.
Fig. 1 is a schematic structural diagram of a security monitoring system according to an embodiment of the present invention. As shown in fig. 1, in an embodiment of the present invention, the security monitoring system includes at least one data acquisition terminal 10, a server 20, and at least one monitoring terminal 30. Wherein, the data acquisition terminal 10 is used for acquiring safety monitoring data; the server 20 is configured to receive the security monitoring data sent by the data acquisition terminal 10 at preset time intervals, input the security monitoring data into a trained security monitoring model, obtain a corresponding security level, and send the security level to the monitoring terminal 30; the monitoring terminal 30 is configured to display the security level, so that a user or a manager can manage the security state of the target environment according to the security level displayed by the monitoring terminal 30.
Fig. 2 is a flowchart of a security monitoring method according to an embodiment of the present invention. The execution body of the embodiment may be the server in fig. 1. As shown in fig. 2, the security monitoring method includes the steps of:
s201: and receiving the safety monitoring data acquired by the data acquisition terminal.
In an embodiment of the invention, the data acquisition terminal includes, by way of example, a power meter, a gas flow meter, and/or a video monitor, among others. The power meter is specifically a power load management terminal meter, and can automatically collect electric meter data such as electric quantity, voltage, current, power factor, voltage qualification rate and/or the like of a user electric energy meter. The terminal can also control the power meter to remotely control power outage and power transmission of a user. By way of example, the gas flow meter may be a gas consumption sensor mounted on a household gas line, which is capable of detecting the instantaneous consumption value, cumulative value, gas leakage, carbon monoxide concentration, carbon dioxide concentration, number of times and/or time period of the household gas, etc. By way of example, the video monitor is an intelligent camera installed in a household, and by presetting a face image of a household member in the video monitor, when a stranger is recognized as occurring in a target environment, behavior and sound data of the stranger in the environment can be captured.
The safety monitoring data are exemplified as real-time monitoring data of a target environment, such as data of gas concentration, electricity consumption of household appliances, water consumption and the like, collected by the data acquisition terminal, and dangerous situations possibly occurring in the living environment are monitored through the data. Optionally, the safety monitoring data may further include a video of the photographed target environment, and by presetting a face image of a family member in the video monitor, behavior and sound data of a stranger in the environment can be captured when the stranger is recognized as occurring in the target environment.
In the embodiment of the invention, the safety monitoring data sent by one or more data acquisition terminals can be received according to the preset time interval. For example, the preset time interval may be set every 1 minute or 2 minutes or 3 minutes, etc. Optionally, the preset time intervals of sending the safety monitoring data by different data acquisition terminals can be the same intervals, or can be adjusted according to actual data types and transmission data amounts, so that the safety state of the target environment can be predicted in time according to the safety monitoring data received by the preset time intervals, the transmission types of the data acquired by the data acquisition terminals can be matched, and the overhead of data transmission is reduced.
S202: and inputting the safety monitoring data into a multi-element Gaussian distribution function to extract characteristic data, and inputting the extracted characteristic data into a trained safety monitoring model, wherein the safety monitoring model is obtained by training a deep neural network by adopting the characteristic data.
In the embodiment of the invention, the security level of the target environment can be predicted through the security monitoring model. Specifically, the safety monitoring data are input into a trained safety monitoring model, and corresponding safety levels are obtained. Illustratively, the security levels may be classified into level 1, level 2, and level 3. Wherein, level 1 indicates that the current target environment has potential safety hazard and needs to be treated safely in time; the level 2 indicates that the safety state of the current target environment is general, and has some non-serious problems, so that the safety treatment can be carried out subsequently; the level 3 indicates that the security state of the current target environment is good, and no problem exists.
In the embodiment of the invention, the trained safety monitoring model is a model obtained by preprocessing safety monitoring data through a Gaussian function to extract multidimensional characteristic data and inputting the extracted multidimensional characteristic data into a deep neural network for training. Specifically, as shown in fig. 3, the deep neural network training process of the safety monitoring model includes the following steps:
in step S2021, the safety monitoring data is input into a multi-element gaussian distribution function to perform feature extraction, so as to obtain a feature data training set.
In the embodiment of the present invention, an exemplary formula of the multivariate gaussian distribution function is shown as formula (1):
wherein p is a feature data training set for training a neural network, n represents n dimensions, Σ is a covariance matrix of n×n, μ is a mean vector of n dimensions, and T represents a transpose of the matrix.
In step S2021, the raw data is preprocessed to obtain a feature data training set, where the raw data is safety monitoring data of a plurality of monitoring targets collected by the data collecting terminal in a preset period of time, and the preprocessing is used to screen and format the raw data, so that the obtained feature data training set is a data set meeting the training requirement of the deep neural network.
Step S2022, implementing unsupervised pre-training of the deep neural network by using the first data set in the feature data training set through the automatic encoder, and taking the parameters of the deep neural network obtained by pre-training as the initialization parameters.
An automatic encoder is utilized to realize the unsupervised pre-training of the deep neural network, and the deep neural network is assumed to be a four-layer neural network comprising an H1 layer, an H2 layer, an H3 layer and an H4 layer. The parameters from the input layer to the H1 layer are first trained using an automatic encoder. After training, we remove the decoding layer, then take the output data of H1 layer as the input data of H2 layer, and then automatically encode. After training, the H2 layer decoder is removed. For higher-layer networks, parameters between two layers are trained sequentially until reaching the last hidden layer, and then multi-classification is performed by using softmax. After the unsupervised pre-training is finished, the obtained parameters are used as initialization parameters, and then the parameters of the neural network model can be finely adjusted by adopting a supervised learning mode.
In step S2023, based on the initialization parameters, training and fine-tuning parameters of the deep neural network are performed by using the second data set in the feature data training set.
The deep neural network may be a deep neural network of a multi-layer convolutional network structure.
In the embodiment of the invention, a deep neural network based on a multi-layer convolution network structure is adopted, a feature data training set is input into the deep neural network, and deep features of multiple metadata are trained; the feature mapping method can be adopted, namely, a group of optimal parameters are obtained through continuous iteration, and the key parameters such as the number of layers, the number of network nodes, the learning rate and the like of the optimized network model are used for efficiently and accurately extracting the deep key features of the environmental data.
The convolutional neural network in the embodiment of the invention can learn the weight parameters of the training network in a gradient error mode, and comprises the following components: a convolutional layer and a downsampling layer. Each neuron of the network in the receiving domain has a relatively constant spatial relationship with a particular small region in the input data. Convolutional neural networks are essentially a neural network that contains multiple layers of perceptrons, each layer of the network consisting of multiple two-dimensional planes. Convolutional neural networks have many advantageous features, such as invariance to translation, rotation, scaling, etc., transformations. Convolutional neural networks also have some constraints, including, in particular, feature learning, feature mapping, and downsampling. Wherein, feature learning, namely extracting partial area from the output of the upper layer as input by each neuron of the convolutional neural network, so that the convolutional neural network learns local features; feature mapping, which is to make each calculation layer of the network in the convolutional neural network consist of a plurality of feature mappings, and neurons in a small area share the same weight matrix under a certain constraint; downsampling is the local averaging and downsampling of the data achieved by the downsampling layer after the convolutional layer operation, with the aim of reducing the sensitivity of the output of the feature map to some linear transformations.
In the embodiment of the invention, the attribute of weight sharing of the convolutional neural network enables the same-layer neurons of the same small area in the network to share a weight matrix, so that the method is equivalent to reducing the number of network parameters in effect, greatly reducing the calculated amount of the network and improving the training speed of the network. The convolutional neural network is also used for essentially completing the mapping from the input space to the output space of the original data, and can well learn a great number of mapping relations between the input space and the output space under the condition that an accurate mathematical logic expression between the input space and the output space is not needed.
The process of training and fine-tuning parameters of the deep neural network mainly comprises a forward propagation stage and a backward propagation stage. In the embodiment of the present invention, the operation of the forward propagation phase and the backward propagation phase is specifically described as follows:
specifically, in the forward propagation stage, a multi-dimensional gaussian distribution function is firstly utilized to extract a multi-dimensional feature construction training set from a safety monitoring data set, and a data sample (X, y) is selected from the training set p ) Wherein X is a characteristic vector composed of mean value, standard deviation, skewness, kurtosis and the like extracted from safety monitoring data acquired by a certain data acquisition terminal through a Gaussian distribution function, and is transmitted to a network as input of the network, and y is ideal output p The similarity index corresponding to the type of the monitored data is obtained, then the data is gradually transmitted from the input layer to the output layer through gradual learning change, and finally the corresponding actual output O of the network is calculated p The sigmoid function calculation formula adopted by each layer of the network is shown as formula (2) and formula (3):
here, z represents an input variable, i.e., X in the sample;
O p =F n (...F 2 (XW 1 )W 2 )...)W n (3)
here, F represents formula (2), and W is a weight.
Specifically, the back propagation phase is also called error back propagation, W is the moment of the weight parameter of each layerMatrix, calculating similarity index y of actual output and ideal safety category through back propagation p And adjusting the weight parameter matrix of the network with the aim of minimizing the error, calculating the actual output and the ideal output y p The error of (2) is shown in formula (4):
in the embodiment of the present invention, a structural diagram specifically describing a convolutional neural network is shown in fig. 4, and includes a plurality of convolutional layers and a downsampling layer. The method comprises the steps of selecting a data sample X from multidimensional features extracted after safety monitoring data are input into a multi-element Gaussian distribution function, taking the data X as input of a convolutional neural network, and using H i Representing the feature matrix of the ith layer of the convolutional neural network. H in convolutional layer i The calculation process of (2) can be as shown in the formula (5):
wherein Wi represents a weight matrix of a convolution kernel of an ith layer of the convolution neural network,convolution operation is carried out on a convolution kernel representing a convolution neural network and 0 th layer input data or an intermediate feature matrix, and the obtained output data and an offset value b of automatic super-parameter setting of an ith layer i After summation, the characteristic data of the ith layer is calculated by a nonlinear activation function f (). After the convolutional layer is learned, the downsampling layer performs downsampling operation on the characteristics learned by the convolutional layer according to a certain downsampling rule. The downsampling layer has the main function of ensuring the invariable characteristic of the characteristic scale learned by the network to a certain extent and reducing the sensitivity of the position of the characteristic scale. Assume that the expression of the downsampling layer is as shown in formula (6):
H i =subsampling(H i-1 ) (6)
after the convolutional neural network is operated alternately by the convolutional layer and the downsampling layer, the convolutional neural network is connected with a fully-connected network according to the similarity output result mapped by the multidimensional data features. And then classifying the learned features to obtain an input-based probability distribution as shown in a formula (7):
Y(i)=P(L=L i |H 0 ;(W,b)) (7)
wherein L is i The cluster center value of the ith security level is represented, W represents the weight matrix of the convolutional neural network, and b represents the bias value. And training the constraint of the convolutional neural network by taking the minimum cost function L (W, b) of the network, namely taking the maximum distance between the characteristic points among the classes of each security class and the minimum distance between the characteristic points in the class as a target. Input data H 0 The error between the network output and the expected value is calculated in a certain way after forward propagation. The cost function can be regarded as an error function. There are many ways to calculate the error function, and common error functions include a Mean Square Error (MSE) function, a Negative Log Likelihood (NLL) function, and a difference Fang Hanshu Jw constructed by using the relationship between the input signal and the output signal, and the expressions are shown in the formula (8), the formula (9), and the formula (10), respectively:
Jw(xz)=||x-z|| 2 (10)
convolutional neural networks also have the phenomenon of overfitting, so the cost function is added with an L2 norm in the objective function to limit the magnitude of the network weight to prevent the overfitting phenomenon, and the overfitting degree is controlled by adding a weight attenuation coefficient lambda (weight, attenuation), and an exemplary calculation formula for calculating the overfitting parameter according to lambda is shown in a formula (11):
here, E (W, b) is the total error of the model, which is the sum of the loss function and regularization term;
l (W, b) is a cost function, which is the average of the sample errors;
lambda is a regularization parameter that controls the trade-off between the loss function and regularization term. A larger λ means more regularization and less overfitting, but also means more variance and less variance;
w is a weight vector of the model, which contains coefficients of a linear function;
W T is a transpose of the weight vector, which means that the rows and columns of the matrix are flipped;
W T w is the dot product of the weight vector and its transpose, which measures the square norm or length of the vector.
The formula can be interpreted to minimize the error of the model by finding the best value of W and b that makes the loss function as small as possible, while also penalizing large values of W to prevent overfitting.
At the end of the convolutional neural network training process, in order to make the convolutional neural network have better learning performance, the gradient descent method is used for back propagation of errors, and the weights and bias parameters (W and b) of the fine-tuning convolutional neural network are trained layer by layer from back to front. In the process, the intensity of error back propagation is controlled by controlling the magnitude of the learning rate parameter (eta), and the specific formulas for calculating the weight and the bias parameter are shown in the formula (12) and the formula (13):
here the number of the elements is the number,representing the partial derivative of the loss function with respect to the i-th parameter.
Illustratively, by employing an automatic encoder, a low-dimensional representation of the input space is generated. The output of each layer is a map of neurons arranged in a grid and having weight vectors similar to the input data. Neurons are organized by similarity, so nearby neurons have similar weight vectors. To achieve extraction of low-level features and high-level features as much as possible for accurate classification using the extracted spatial information and global information.
In the embodiment of the invention, the safety monitoring data is input into a multi-element Gaussian distribution function to extract a characteristic data training set, after the data characteristics are unsupervised and learned from the unlabeled data through the pre-training of an automatic encoder, the neural network mapped by the self-organizing characteristics is used for multidimensional training in the training of deep learning, and the local average of the data is realized during downsampling. Specifically, since the classification of the collected raw data is very unbalanced, a machine learning algorithm can be adopted to obtain experience from a large number of data sets through calculation, so as to determine whether certain data are normal or not. However, because the number of few class samples in the unbalanced data set is too small, the trained model is more prone to the defect of the majority class sample set, so that a part of data can be selected from the majority set and recombined with the minority set to form a new data set, local average is realized, and the problem of unbalanced data distribution is solved. For the application scene of the embodiment of the invention, the safety state is good and is generally the majority, and the potential safety hazard is the minority.
In embodiments of the present invention, the sensitivity of the output of the feature map to some linear transformations is reduced by locally averaging the data at the time of downsampling. Specifically, the convolutional neural network model is based on local connection among neurons and hierarchical organization class conversion, neurons with the same parameters can be applied to different positions of a previous layer of neural network, a translational unchanged neural network structure form is obtained, and a safety monitoring model for multidimensional health monitoring is obtained.
S203: and after the positions of the clustering centers output by the safety monitoring model are corresponding to the corresponding safety levels, sending the corresponding safety levels to the monitoring terminal, so that a user can acquire the safety levels from the monitoring terminal, and the safety state of the target environment can be managed.
In the embodiment of the invention, the monitoring terminal is a remote control terminal of a family member to which the target environment belongs, and the monitoring terminal can be mobile terminal equipment such as a mobile phone. The security level is sent to the mobile phone of the family member, so that the family member can conduct security management on the target environment according to the predicted security level.
From the above embodiments, it can be seen that, the safety monitoring data is input into the multivariate gaussian distribution function to extract the feature data, and the extracted feature data is input into the trained safety monitoring model, so as to obtain the corresponding safety level of each cluster center position output by the safety monitoring model. The method is characterized in that a Gaussian distribution theory and a deep neural network are applied to safety monitoring data analysis, the deep neural network for learning key characteristics of safety time sequence data in an unsupervised learning mode is provided, a safety state evaluation model is built based on a multi-element Gaussian distribution theory, the data characteristics learned by the deep neural network are used as input of the evaluation model, the purpose of monitoring the safety state of a household living environment is achieved, and living safety of the household environment is improved.
In one possible implementation, after the security level is sent to the monitoring terminal, the security level is sent to a third party management platform, so that a third party manager manages the target environment according to the security level.
In the embodiment of the invention, the third party manager is an operator of the community service station, and the operator of the community service station is generally closer to the monitored environment, so that potential safety hazards possibly occurring in the monitored environment can be timely processed. For example, the third party management platform may serve the community terminal where the target environment is located. By sending the security level to the third party management platform, the third party management personnel can timely process potential safety hazards possibly occurring in the target environment, and living safety of the target environment is guaranteed.
Fig. 5 is a flowchart of a second safety monitoring method according to an embodiment of the present invention. On the basis of the embodiment shown in fig. 2, as shown in fig. 5, before or during the step S202 of inputting the safety monitoring data into the multivariate gaussian distribution function to extract the feature data, the safety monitoring method provided by the embodiment of the present invention further includes the following steps:
s401: and receiving prompt information sent by the data acquisition terminal, wherein the prompt information comprises a prompt type and a processing guide.
In the embodiment of the invention, the data acquisition terminal has the functions of data acquisition and pre-judgment, namely, different data acquisition terminals can perform preliminary judgment on acquired data. The processor in the data acquisition terminal may comprise a Loongson 1C101 chip. Specifically, through setting up the judgement model that corresponds to the emergency in collector or the equipment that is correlated with the sensor respectively, for example when the emergency such as fire alarm, gas leakage and personnel's unusual appears, can confirm the result of emergency according to the judgement model that corresponds to produce corresponding suggestion type, make the server of safety monitoring system in time handle the potential safety hazard that probably takes place according to the processing guideline that the suggestion type corresponds. Illustratively, the prompt type includes at least one of a fire alarm, a gas leak, and a personnel anomaly, respectively. The security monitoring method provided in the embodiment of fig. 2 predicts the security level of the target environment according to the collected metadata, the purpose of the security monitoring method provided in the embodiment of fig. 5 is to timely process the emergency state of the situation, the accuracy of the monitoring method provided in the embodiment of fig. 2 is higher, and the real-time performance of the monitoring method provided in the embodiment of fig. 5 is higher.
S402: and transmitting the prompt information to a monitoring terminal, so that a user acquires the prompt type and the processing guide from the monitoring terminal, and the target environment is managed according to the prompt type and the processing guide.
According to the embodiment, the prompt information of the emergency sent by the data acquisition terminal is received and transmitted to the monitoring terminal, so that a user can manage the target environment according to the prompt type and the processing guide displayed by the monitoring terminal, namely, when the emergency occurs, the user can perform safe processing on the target environment in time directly according to the prompt information and the processing guide, and living safety of the home environment is guaranteed.
Fig. 6 is a schematic structural diagram of a safety monitoring device according to an embodiment of the present invention. As shown in fig. 6, the safety monitoring device includes: a receiving module 501, a predicting module 502, and a transmitting module 503;
the receiving module 501 is configured to receive the safety monitoring data collected by the data collecting terminal.
The prediction module 502 is configured to input the safety monitoring data into a multivariate gaussian distribution function to extract feature data, and input the extracted feature data into a trained safety monitoring model, where the safety monitoring model is obtained by training a deep neural network using the feature data.
And the sending module 503 is configured to send the cluster center positions output by the security monitoring model to the monitoring terminal after corresponding to the corresponding security levels, so that the user obtains the security levels from the monitoring terminal.
In one possible implementation, the safety monitoring device further includes a model training module, the model training module including:
the preprocessing sub-module is used for inputting the safety monitoring data into the multi-element Gaussian distribution function to perform characteristic extraction so as to obtain a characteristic data training set;
the pre-training sub-module is provided with a first data set in the characteristic data training set, and is used for realizing the unsupervised pre-training of the deep neural network through an automatic encoder, and taking parameters of the deep neural network obtained by the pre-training as initialization parameters;
and the parameter tuning sub-module is used for training and fine tuning parameters of the deep neural network by utilizing the second data set in the characteristic data training set on the basis of the initialization parameters.
In one possible implementation, the tuning submodule is further configured to perform training fine tuning on parameters of the deep neural network through a forward propagation phase and a backward propagation phase.
In one possible implementation, the deep neural network is a deep neural network of a multi-layer convolutional network structure, including a convolutional layer and a downsampling layer.
In one possible implementation manner, the safety monitoring device further comprises a transmission module, wherein the transmission module is used for receiving prompt information sent by the data acquisition terminal, and the prompt information comprises a prompt type and a processing guide; and transmitting the prompt information to a monitoring terminal, so that a user manages a target environment according to the prompt type displayed by the monitoring terminal and the processing guide.
In a possible implementation manner, the sending module 503 is further configured to send the security level to a third party management platform, so that an administrator of the third party management platform obtains the security level from the third party management platform to manage the target environment.
In this embodiment, the method described in the foregoing embodiments may be adopted for the safety monitoring device, and the technical scheme and the technical effects thereof are similar and are not described herein.
Fig. 7 is a schematic hardware structure of a server according to an embodiment of the present invention. The server of the embodiment of the invention can specifically comprise a Loongson 3C6000L four-way server. As shown in fig. 7, the server of the present embodiment includes: a processor 601 and a memory 602; wherein the method comprises the steps of
A memory 602 for storing computer-executable instructions;
a processor 601 for executing computer-executable instructions stored in a memory to implement the steps performed by the server in the above embodiments. Reference may be made in particular to the relevant description of the embodiments of the method described above.
Alternatively, the memory 602 may be separate or integrated with the processor 601.
When the memory 602 is provided separately, the server further comprises a bus 603 for connecting said memory 602 and the processor 601.
The embodiment of the invention also provides a computer readable storage medium, wherein computer execution instructions are stored in the computer readable storage medium, and when a processor executes the computer execution instructions, the safety monitoring method is realized.
Embodiments of the present invention also provide a computer program product comprising a computer program which, when executed by a processor, implements a security monitoring method as described above.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each module may exist alone physically, or two or more modules may be integrated in one unit. The integrated units of the modules can be realized in a form of hardware or a form of hardware and software functional units.
The integrated modules, which are implemented in the form of software functional modules, may be stored in a computer readable storage medium. The software functional modules described above are stored in a storage medium and include instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or processor to perform some of the steps of the methods described in the various embodiments of the invention.
It should be understood that the above processor (i.e., CPU) may be a processor of a loongson serial chip, may be a loongson serial No. 3 general purpose processor, and may also be a loongson serial No. 2 processor or a loongson serial No. 1 processor. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor for execution, or in a combination of hardware and software modules in a processor for execution.
The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile memory NVM, such as at least one magnetic disk memory, and may also be a U-disk, a removable hard disk, a read-only memory, a magnetic disk or optical disk, etc.
The bus may be an industry standard architecture (Industry Standard Architecture, ISA) bus, an external device interconnect (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, the buses in the drawings of the present invention are not limited to only one bus or one type of appearance bus.
The storage medium may be implemented by any type of volatile or non-volatile memory device or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an application specific integrated circuit (Application Specific Integrated Circuits, ASIC for short). It is also possible that the processor and the storage medium reside as discrete components in an electronic device or a master device.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by hardware associated with program instructions. The foregoing program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (10)
1. A method of security monitoring, the method comprising:
receiving safety monitoring data acquired by a data acquisition terminal;
inputting the safety monitoring data into a multi-element Gaussian distribution function to extract characteristic data, and inputting the extracted characteristic data into a trained safety monitoring model, wherein the safety monitoring model is obtained by training a deep neural network by adopting the characteristic data;
and after the positions of the clustering centers output by the safety monitoring model are corresponding to the corresponding safety levels, sending the corresponding safety levels to the monitoring terminal, so that a user obtains the safety levels from the monitoring terminal.
2. The method of claim 1, wherein the training process of the safety monitoring model comprises:
inputting the safety monitoring data into a multi-element Gaussian distribution function to perform feature extraction so as to obtain a feature data training set;
using the first data set in the characteristic data training set, realizing unsupervised pre-training of the deep neural network through an automatic encoder, and taking parameters of the deep neural network obtained by pre-training as initialization parameters;
and on the basis of the initialization parameters, training and fine-tuning parameters of the deep neural network by utilizing a second data set in the characteristic data training set.
3. The method of claim 2, wherein the training fine-tuning of parameters of the deep neural network comprises:
the parameters of the deep neural network are subjected to training fine adjustment through a forward propagation stage and a backward propagation stage.
4. A method according to any one of claims 1 to 3, wherein the deep neural network is a deep neural network of a multi-layer convolutional network structure comprising a convolutional layer and a downsampling layer.
5. The method of claim 1, further comprising, prior to or during said inputting the safety monitoring data into the multivariate gaussian distribution function to extract the feature data:
receiving prompt information sent by a data acquisition terminal, wherein the prompt information comprises a prompt type and a processing guide;
and transmitting the prompt information to a monitoring terminal, so that a user acquires the prompt type and the processing guide from the monitoring terminal.
6. A method according to any one of claims 1 to 3, further comprising, after each cluster center position output by the security monitoring model corresponds to a corresponding security level and is transmitted to a monitoring terminal:
and sending the security level to a third party management platform, so that a manager of the third party management platform obtains the security level from the third party management platform to manage the target environment.
7. A safety monitoring device, comprising:
the receiving module is used for receiving the safety monitoring data acquired by the data acquisition terminal;
the prediction module is used for inputting the safety monitoring data into a multi-element Gaussian distribution function to extract characteristic data, and inputting the extracted characteristic data into a trained safety monitoring model, wherein the safety monitoring model is obtained by training a deep neural network by adopting the characteristic data;
and the sending module is used for sending the cluster center positions output by the safety monitoring model to the monitoring terminal after corresponding to the corresponding safety levels, so that a user obtains the safety levels from the monitoring terminal.
8. A server, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing computer-executable instructions stored in the memory causes the at least one processor to perform the security monitoring method of any one of claims 1 to 6.
9. A safety monitoring system, comprising: at least one data acquisition terminal, a server and at least one monitoring terminal;
the data acquisition terminal is used for acquiring safety monitoring data of the target environment;
the server is used for receiving the safety monitoring data acquired by the data acquisition terminal and executing the safety monitoring method according to any one of claims 1 to 6;
and the monitoring terminal is used for receiving the security level of the target environment from the server and outputting the security level to a user.
10. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor implement the security monitoring method of any of claims 1 to 6.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310912771.7A CN117009785A (en) | 2023-07-24 | 2023-07-24 | Security monitoring method, device, server and system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310912771.7A CN117009785A (en) | 2023-07-24 | 2023-07-24 | Security monitoring method, device, server and system |
Publications (1)
Publication Number | Publication Date |
---|---|
CN117009785A true CN117009785A (en) | 2023-11-07 |
Family
ID=88575642
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310912771.7A Pending CN117009785A (en) | 2023-07-24 | 2023-07-24 | Security monitoring method, device, server and system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117009785A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117272029A (en) * | 2023-11-20 | 2023-12-22 | 北京世纪慈海科技有限公司 | Old man safety monitoring method and device based on big data processing |
-
2023
- 2023-07-24 CN CN202310912771.7A patent/CN117009785A/en active Pending
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117272029A (en) * | 2023-11-20 | 2023-12-22 | 北京世纪慈海科技有限公司 | Old man safety monitoring method and device based on big data processing |
CN117272029B (en) * | 2023-11-20 | 2024-03-01 | 北京世纪慈海科技有限公司 | Old man safety monitoring method and device based on big data processing |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110929622B (en) | Video classification method, model training method, device, equipment and storage medium | |
CN107516110B (en) | Medical question-answer semantic clustering method based on integrated convolutional coding | |
CN111967343B (en) | Detection method based on fusion of simple neural network and extreme gradient lifting model | |
CN109490814B (en) | Metering automation terminal fault diagnosis method based on deep learning and support vector data description | |
CN109345260A (en) | A kind of fraud detection model training method and device and fraud detection method and device | |
CN113723007B (en) | Equipment residual life prediction method based on DRSN and sparrow search optimization | |
CN111353153A (en) | GEP-CNN-based power grid malicious data injection detection method | |
CN108133225A (en) | A kind of icing flashover fault early warning method based on support vector machines | |
CN109886496B (en) | Agricultural yield prediction method based on meteorological information | |
He et al. | MTAD‐TF: Multivariate Time Series Anomaly Detection Using the Combination of Temporal Pattern and Feature Pattern | |
CN117009785A (en) | Security monitoring method, device, server and system | |
CN114048468A (en) | Intrusion detection method, intrusion detection model training method, device and medium | |
CN112699998A (en) | Time series prediction method and device, electronic equipment and readable storage medium | |
CN116993537A (en) | Power load abnormality detection method and system based on serial GRU (generic routing unit) self-encoder | |
CN116340796A (en) | Time sequence data analysis method, device, equipment and storage medium | |
CN116702090A (en) | Multi-mode data fusion and uncertain estimation water level prediction method and system | |
CN113935413A (en) | Distribution network wave recording file waveform identification method based on convolutional neural network | |
Qin et al. | Remaining useful life prediction using temporal deep degradation network for complex machinery with attention-based feature extraction | |
CN112819256A (en) | Convolution time sequence room price prediction method based on attention mechanism | |
CN115952928B (en) | Short-term power load prediction method, device, equipment and storage medium | |
CN117407770A (en) | High-voltage switch cabinet fault mode classification and prediction method based on neural network | |
CN117391466A (en) | Novel early warning method and system for contradictory dispute cases | |
CN116776209A (en) | Method, system, equipment and medium for identifying operation state of gateway metering device | |
Kong et al. | A novel ConvLSTM with multifeature fusion for financial intelligent trading | |
Singh et al. | LSTM based Stacked Autoencoder Approach for Time Series Forecasting |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |