CN113971513A - Data storage and optimization method of enterprise security risk management platform - Google Patents
Data storage and optimization method of enterprise security risk management platform Download PDFInfo
- Publication number
- CN113971513A CN113971513A CN202111231028.2A CN202111231028A CN113971513A CN 113971513 A CN113971513 A CN 113971513A CN 202111231028 A CN202111231028 A CN 202111231028A CN 113971513 A CN113971513 A CN 113971513A
- Authority
- CN
- China
- Prior art keywords
- data
- safety production
- production data
- enterprise safety
- enterprise
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000007726 management method Methods 0.000 title claims abstract description 19
- 238000013500 data storage Methods 0.000 title claims abstract description 11
- 238000005457 optimization Methods 0.000 title claims abstract description 10
- 238000004519 manufacturing process Methods 0.000 claims abstract description 92
- 230000002159 abnormal effect Effects 0.000 claims abstract description 68
- 238000009826 distribution Methods 0.000 claims abstract description 51
- 238000001514 detection method Methods 0.000 claims abstract description 8
- 230000000295 complement effect Effects 0.000 claims description 15
- 238000011084 recovery Methods 0.000 claims description 11
- 230000005856 abnormality Effects 0.000 claims description 9
- 230000002596 correlated effect Effects 0.000 claims description 8
- 230000006735 deficit Effects 0.000 claims 1
- 238000007906 compression Methods 0.000 abstract description 10
- 230000006835 compression Effects 0.000 abstract description 9
- 238000003860 storage Methods 0.000 abstract 1
- 230000000875 corresponding effect Effects 0.000 description 17
- 238000005070 sampling Methods 0.000 description 16
- 230000006837 decompression Effects 0.000 description 13
- 238000012549 training Methods 0.000 description 11
- 230000008901 benefit Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000013144 data compression Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000005314 correlation function Methods 0.000 description 1
- 238000005336 cracking Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0635—Risk analysis of enterprise or organisation activities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/906—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- Evolutionary Biology (AREA)
- Game Theory and Decision Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Development Economics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Educational Administration (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention provides a data storage and optimization method of an enterprise security risk management platform. The method comprises the following steps: carrying out anomaly detection on the enterprise safety production data to obtain the anomaly degree of the enterprise safety production data; selecting a variational self-encoder according to the abnormal degree of the enterprise safety production data to compress the enterprise safety production data and store the data to an enterprise safety risk management platform; and the standard deviation parameter of the reference distribution of the variational self-encoder is determined according to the abnormal degree grade of the enterprise safety production data and a preset secret key. The invention optimizes the storage of the data of the enterprise security risk management platform, improves the compression ratio of the data, and can effectively prevent the data of the enterprise security risk management platform from being cracked.
Description
Technical Field
The invention relates to the technical field of security risk management and big data, in particular to a data storage and optimization method of an enterprise security risk management platform.
Background
Enterprises can generate massive safe production data in production and monitoring links. In the prior art, data storage and optimization are usually realized by removing redundant data, and the data storage requirement under a safe production scene cannot be met. Particularly, the importance degree of the safety production data of the enterprises is different, the prior art is difficult to perform data compression in a targeted manner, and the data compression is low.
Disclosure of Invention
In order to solve the technical problem, the invention provides a data storage and optimization method for an enterprise security risk management platform, which comprises the following steps:
carrying out anomaly detection on the enterprise safety production data to obtain the anomaly degree of the enterprise safety production data;
selecting a variational self-encoder according to the abnormal degree of the enterprise safety production data, and compressing the enterprise safety production data by using the encoding end of the selected variational self-encoder and storing the compressed variational self-encoder to an enterprise safety risk management platform; and the standard deviation parameter of the reference distribution of the variational self-encoder is determined according to the abnormal degree grade of the enterprise safety production data and a preset secret key.
Further, the mean parameter of the variational self-coder reference distribution is determined according to a standard deviation parameter.
Further, the method further comprises: and grading the abnormal degrees, wherein the enterprise safety production data of each abnormal degree grade corresponds to a reference distribution and a variation self-encoder.
Further, the standard deviation parameter is positively correlated with the abnormal degree grade of the safety production data of the enterprise.
Further, the step of determining the standard deviation parameter of the variational self-encoder reference distribution according to the abnormal degree level of the enterprise safety production data and a preset key comprises the following steps: and determining a first parameter according to the abnormal degree grade of the enterprise safety production data, wherein the first parameter is positively correlated with the abnormal degree grade of the enterprise safety production data, and the product of the first parameter and a preset key is used as a standard deviation parameter of the reference distribution of the variational self-encoder.
Further, the performing anomaly detection on the enterprise safety production data to obtain the anomaly degree of the enterprise safety production data includes: and clustering the enterprise safety production data, wherein the noise data points are abnormal data of the enterprise safety production, and determining the abnormal degree of the enterprise safety production data according to the distance between the noise data points and the clustering center.
Further, the determining the abnormal degree of the enterprise safety production data according to the distance between the noise data point and the clustering center comprises: and normalizing the distance between the noise data point and the clustering center, wherein the normalized distance is the abnormal degree of the enterprise safety production data.
Further, the method further comprises: according to the reference distribution, the variational self-encoder is trained by utilizing enterprise safety production data: determining the error digit according to the abnormal degree grade of the enterprise safety production data; performing final complement for the enterprise safety production data at least twice, wherein one time is a random complement, one time is a full zero complement, and the complement digit is equal to the error digit; and respectively inputting the complemented enterprise safety production data into the variational self-encoder to obtain recovery data.
Further, the classifying the degree of abnormality specifically includes: the degree of abnormality is classified into five stages.
Further, the loss of the variational self-encoder includes: the difference between the input data and the recovered data after random number padding, the difference between the input data and the recovered data after all zero number padding, and the difference between the reference distribution and the output distribution.
The invention has the beneficial effects that:
the invention expresses the enterprise safety production data by adopting the hidden variable distribution of the variational self-encoder, and has extremely high compression ratio. The enterprise safety production data with different abnormal degrees correspond to different variational self-encoders, and the recovery precision, the cracking difficulty and the abnormal degree of the variational self-encoders are positively correlated by means of setting reference distribution, error digits and the like, so that the data safety is adaptive to the important degree of the variational self-encoders, and the safety performance of the data is comprehensively improved. And adding error bits and setting a corresponding loss function to ensure that the original data can be correctly recovered, wherein the higher the abnormal degree of the safety production data is, the higher the recovery precision is. The reference distribution is determined according to the abnormal degree of the enterprise safety production data, the higher the abnormal degree is, the larger the parameter value of the reference distribution is, the more difficult the enterprise safety production data is to be decoded, decrypted and decompressed, and the safety performance of the data is improved. Because the variational self-encoder has a generating function, if a sampling position is wrong or a decoder selects a wrong position, information inconsistent with original data is recovered, and the variational self-encoder can play a role in safety protection of stored data.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention mainly aims to realize the compression and decompression of safe production data of an enterprise safety risk management platform. In order to achieve the purpose, the invention designs a data storage and optimization method of an enterprise security risk management platform.
Example 1:
the present embodiment provides a data storage and optimization method for an enterprise security risk management platform, and fig. 1 is a flowchart of the method of the present invention. Under the scenario of an enterprise security risk management platform, data is mainly safe production data, and the data exception degree generally represents the importance degree of the data.
And step 1, carrying out abnormity detection on the enterprise safety production data to obtain the abnormity degree of the enterprise safety production data. The input of the step is as follows: the enterprise safety production data is output as follows: and the abnormal degree corresponding to the data. The purpose of this step is: detecting abnormal data of safety production and evaluating the abnormal degree; the advantages that can be brought are: the method is beneficial to determining different compression encryption degrees according to the abnormal degree of the data subsequently, and improves the data security.
Data anomaly detection is a well-known technique, and an implementer can realize the data anomaly detection by adopting a plurality of modes such as a binary neural network, a self-encoder, an SVM and the like. The embodiment adopts unsupervisedThe clustering algorithm carries out anomaly detection, clusters the enterprise safety production data, noise data points are abnormal data of the enterprise safety production, and the abnormal degree of the enterprise safety production data is determined according to the distance between the noise data points and a clustering center. And normalizing the distance between the noise data point and the clustering center, wherein the normalized distance is the abnormal degree of the enterprise safety production data. The clustering algorithm adopted in this embodiment is specifically a DBSCAN clustering algorithm, density clustering is performed in a high-dimensional space where data is located, clustering radii and quantity thresholds are set by an implementer, clustering radii corresponding to different data are different, quantity thresholds corresponding to different data quantities are different, but since the task is actually a two-classification task, only one clustering set is required after clustering; the data judged as the noise point is regarded as abnormal data. Let d be the minimum Euclidean distance between the nth noisy data point and the point in the cluster setnWith dn/dmaxRepresenting the degree of abnormality of the nth noise data point, dmaxFor the maximum value of the minimum euclidean distances from all the noise data points to the points in the cluster set, the distance metric is used to describe the degree of abnormality in this embodiment, and the implementer may select other metric manners as needed, which is not limited herein. In addition, the abnormal degree analysis of the enterprise safety production data can be directly carried out by utilizing the neural network classification model, and the abnormal degree is output. The network structure adopts a structure of an encoder-full connection layer, the loss function adopts a mean square error loss function, and the training set adopts enterprise safety production data.
And 2, selecting a variational self-encoder according to the abnormal degree of the enterprise safety production data, and compressing the enterprise safety production data by using the encoding end of the selected variational self-encoder and storing the compressed variational self-encoder to an enterprise safety risk management platform. The standard deviation parameter of the reference distribution of the variational self-encoder is determined according to the abnormal degree grade of the enterprise safety production data and a preset secret key, the standard deviation parameter is positively correlated with the abnormal degree grade of the enterprise safety production data, and the mean parameter of the reference distribution of the variational self-encoder is determined according to the standard deviation parameter. The input of step 2 is: the enterprise safety production data and the corresponding abnormal degree are output as follows: the compressed data. The purpose of this step is: confirm the compression mode according to enterprise safety production data abnormal degree, the benefit that can bring: data is compressed efficiently. Preferably, the determination mode of the reference distribution standard deviation parameter is as follows: and determining a first parameter according to the abnormal degree grade of the enterprise safety production data, wherein the first parameter is positively correlated with the abnormal degree grade of the enterprise safety production data, and the product of the first parameter and a preset key is used as a standard deviation parameter of the reference distribution of the variational self-encoder.
Training is performed before compression using the variational self-encoder. Before training, it is first determined that several variational autocoders need to be trained. The invention determines the number of variational self-encoders according to the number of the abnormal degree grades. Specifically, the abnormal degree is classified, and the enterprise safety production data of each abnormal degree grade corresponds to a reference distribution and a variation self-encoder. The present embodiment divides the degree of abnormality into five levels. Because the value range of the abnormal degree is [0,1], the abnormal degree value is in the interval of [0,0.2) as the first stage, the abnormal degree value is in the interval of [0.2,0.4) as the second stage, the abnormal degree value is in the interval of [0.4,0.6) as the third stage, the abnormal degree value is in the interval of [0.6,0.8) as the fourth stage, and the abnormal degree value is in the interval of [0.8,1.0] as the fifth stage; the implementer can determine different grading modes according to needs, the grading aims to determine a subsequent compression process, the grading determines that the modification is not allowed before the overall compression, and if the modification exists, the subsequent network needs to be trained again. The number of the variational self-encoders to be trained is determined according to the classification, and if the embodiment is divided into five stages, five variational self-encoders are required to be trained.
Different from the existing variational self-encoder, in the training process, the implicit variable distribution generated by encoding is not aligned to the standard normal distribution, but the reference distribution is determined according to the abnormal degree grade of the corresponding safety production data. The standard deviation parameter of the reference distribution is positively correlated with the abnormal degree grade of the enterprise safety production data. Specifically, a large prime number p is determined according to the abnormal grade of the safe production data corresponding to the variational self-encoderc(first parameter), where c is the rank, the larger the large prime number,and the implementer sets a large prime number q as a decompression key (preset key), a reference normal distribution N- ([ mu ]) is obtainedc,(pc*q)2) In which μcAs a mean parameter, (p)c*q)2Is a variance parameter, μcIs a large prime number pcThe sum of each bit.
In addition, in order to improve the decompression precision of the variational self-encoder, the invention also determines the error digit, and the error digit is positively correlated with the abnormal degree grade of the enterprise safety production data. One embodiment of the number of error bits: k is a radical ofc＝[pc/μc]Wherein k iscIs the error bit number, here [ ·]Represents rounding down; the number of error bits is used to complement the input data at the end of training, for example, if the data is a binary number 10 and the number of error bits is 2, the input data is changed to 10 XX. Another embodiment of the number of error bits: k is a radical ofc＝[log(pc)]Preferably, log (p)c) With a base of 10.
Training the variational self-encoder by utilizing enterprise safety production data: determining the error digit according to the abnormal degree grade of the enterprise safety production data; performing final complement for the enterprise safety production data at least twice, wherein one time is a random complement, one time is a full zero complement, and the complement digit is equal to the error digit; and respectively inputting the complemented enterprise safety production data into the variational self-encoder to obtain recovery data.
The training process of the variational self-encoder is as follows: the safety production data with the same abnormal degree grade is used as a training data set, and the training of the variational self-encoder is unsupervised training, so that manual marking is not needed; determining the error digit k according to the abnormal degree grade c of the safety production datacComplementing numbers at the tail of the training set, wherein the complemented numbers can be randomly generated without constraint; trained loss function L:
L＝ω1Lr+ω2LR+ω3Lp
wherein L isrFor global reconstruction of losses, omega1Loss of corresponding weight for the overall reconstruction, LRReconstruction of the loss, omega, for the original data2Is a prime numberAccording to the weight value, L, corresponding to the reconstruction losspFor implicit variable distribution loss, omega3The weight corresponding to the hidden variable distribution loss is obtained;
specifically, the overall reconstruction loss:
Lr＝||x′-x||2
wherein x is input data with random number complementing at the tail, and x' is recovery data;
original data reconstruction loss:
LR＝||y′-y||2
wherein y is input data with the tail complement number set to be 0, and y' is data with the tail of the corresponding recovery data set to be zero;
Lp＝DKL[N(μ,σ2)||N(μc,(pc*q)2)]
wherein, N (mu, sigma)2) For implicit variable distribution, N (. mu.)c,(pc*q)2) For reference to normal distribution, DKL() Represents the KL divergence; among the above weights, ω2Greater than omega1That is, the accuracy of original data reconstruction is ensured, and preferably, the three weights are respectively set to be 0.4,0.6 and 1; in addition, the variational self-encoder needs a heavy parameter skill to ensure that the network can be trained, and the basic distribution corresponding to the abnormal levels of different safety service data is not consistent, wherein the basic distribution is N (0, p)c 2) Then, the transformation of the heavy parameter is as Z ═ mu + epsilon ^ sigma/pcWhere Z is the sample point on the latent variable distribution and ε is the sample point on the base distribution.
After the plurality of variational self-coders are trained, in the using process, an implementer carries out tail random complement on the safety production data according to the data abnormal grade of the safety production data, then the safety production data are input into the corresponding variational self-coders, the encoder outputs hidden variable distribution, each dimensionality of the hidden variable distribution can be represented by a mean value and a variance value, the hidden variable distribution is compressed data, and compared with the input data, the high-speed data compression effect is achieved.
The compression mode of the invention has the advantages that: the input data is represented by the implicit variable distribution of a variational self-encoder, and the high compression ratio is achieved; because the variational self-encoder has a generating function, if a sampling position is wrong or a decoder selects a mistake, information inconsistent with original data is recovered, and the variational self-encoder can play a role in safety protection of stored data; adding error bits and setting corresponding loss functions to ensure that the original data can be correctly recovered; the importance degrees of data corresponding to the abnormal levels of different safety service data are different, a positive correlation function is set, the more serious the abnormal level is, the more error bits are, and the probability that the original data is retained is relatively higher even if the data recovery is wrong; changing a KL divergence measurement object in a loss function of the variational self-encoder, ensuring that the variance of the object is not 0, having a generating function and simultaneously recovering data according to the distribution of the object; the large prime number is used as the variance, and a large prime number key is added, so that safer data security protection can be achieved, and the prior based on the large prime number decomposition problem is achieved.
Example 2:
the embodiment provides a data storage and optimization method of an enterprise security risk management platform. This embodiment further provides a decompression method based on embodiment 1.
The advantages that can be brought to enterprise safety production data decompression: the original data can be accurately recovered when the secret key is held, otherwise, the error data can be recovered. The process inputs are: the compressed data is output as: and recovering the original data of the safe production. The decompression process comprises the following steps: the decompression end obtains a first parameter according to the carried preset secret key and the standard deviation parameter of the compressed data; then determining whether the first parameter and the average parameter of the compressed data meet a preset condition, if so, generating basic distribution according to the first parameter, sampling on the basic distribution to obtain a sampling value, and decompressing and recovering the sampling value by using a decoding end to obtain recovered data; and determining the number of error bits, and deleting the number of error bits from the recovered data to obtain decompressed data. Wherein the preset conditions include: the sum of the data bits of the first parameter is equal to the mean parameter. The preset conditions further include: the error of the sum of the data bits of the first parameter and the average parameter is within a tolerable error range. Preferably, the sampled value is not zero.
Specifically, the compressed data is the mean value mu of the distribution of the hidden variablescSum variance (p)c*q)2Since the dimensions are independent, the description is made here in a single dimension. If the decompressor carries the preset key q, p can be recovered according to the variance datac(first parameter), otherwise, it is a large prime decomposition problem. In order to eliminate the influence of the precision of the variational self-coding network, when a first parameter (hidden variable standard deviation/preset key) is recovered, a prime number closest to a division result is taken as the first parameter.
Recovery of pcThe decompressor can then perform mean value verification, i.e. based on recovered pcEach bit sum and mean mucAnd comparing, if the preset condition is met, confirming that the decompression operation of the decompressor is valid. Recovery of pcThen, based on the distribution N (0, p)c 2) Sampling, limiting the sampling point to non-zero integer, and obtaining corresponding N (mu) through repeated parameter transformationc,(pc*q)2) Is limited to the distribution N (0, p)c 2) If the sampling points are non-zero integers (i.e., epsilon should be a non-zero integer), then N (mu) is determinedc,(pc*q)2) And (3) decompressing the data only when the sampling points are pre-stored discrete sampling points (outputting the corresponding sampling points in the distribution when epsilon is a non-zero integer), otherwise, locking the data and being incapable of being read.
It should be noted that the sampling value set on the basic distribution of the present invention cannot be zero, which is very critical to the implementation of the present invention, and has the following effects: the decompression end must recover the first parameter to obtain the correct value, thereby avoiding the decompression reading of the user not carrying the preset key. If the decompressor does not hold the predetermined key, directly at N (μ)c,(pc*q)2) And (4) sampling is distributed, at the moment, the sampling points are difficult to be matched with pre-stored discrete sampling points, and data cannot be read. After the data can be decompressed, the sampling point data is sent to a decoder, and the recovery data is output. Then, according to pcOr pcAnd mucAnd determining the error bit number to further obtain the decompressed data. It is to be noted thatIn order to prevent malicious reading of abnormal data of the security service, the method and the device set the limitation of decompression times, for example, set the decompression times to be twice, lock the data when decompression fails in two times, and generate early warning information to report to the management terminal.
The above embodiments are merely preferred embodiments of the present invention, which should not be construed as limiting the present invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A data storage and optimization method for an enterprise security risk management platform is characterized by comprising the following steps:
carrying out anomaly detection on the enterprise safety production data to obtain the anomaly degree of the enterprise safety production data;
selecting a variational self-encoder according to the abnormal degree of the enterprise safety production data, and compressing the enterprise safety production data by using the encoding end of the selected variational self-encoder and storing the compressed variational self-encoder to an enterprise safety risk management platform; and the standard deviation parameter of the reference distribution of the variational self-encoder is determined according to the abnormal degree grade of the enterprise safety production data and a preset secret key.
2. The method of claim 1, wherein the mean parameter of the variational self-coder reference distribution is determined from a standard deviation parameter.
3. The method of claim 1, further comprising: and grading the abnormal degrees, wherein the enterprise safety production data of each abnormal degree grade corresponds to a reference distribution and a variation self-encoder.
4. The method of claim 1, wherein the standard deviation parameter positively correlates to a level of anomaly in the enterprise safety production data.
5. The method of claim 4, wherein the determining the standard deviation parameter of the variational self-encoder reference distribution according to the enterprise safe production data anomaly level and the preset key comprises: and determining a first parameter according to the abnormal degree grade of the enterprise safety production data, wherein the first parameter is positively correlated with the abnormal degree grade of the enterprise safety production data, and the product of the first parameter and a preset key is used as a standard deviation parameter of the reference distribution of the variational self-encoder.
6. The method of claim 1, wherein the detecting the abnormality of the enterprise safety production data, and obtaining the degree of abnormality of the enterprise safety production data comprises: and clustering the enterprise safety production data, wherein the noise data points are abnormal data of the enterprise safety production, and determining the abnormal degree of the enterprise safety production data according to the distance between the noise data points and the clustering center.
7. The method of claim 5, wherein determining the degree of anomaly of the enterprise safe production data based on the distance of the noise data point from the cluster center comprises: and normalizing the distance between the noise data point and the clustering center, wherein the normalized distance is the abnormal degree of the enterprise safety production data.
8. The method of claim 1, wherein the method further comprises: according to the reference distribution, the variational self-encoder is trained by utilizing enterprise safety production data: determining the error digit according to the abnormal degree grade of the enterprise safety production data; performing final complement for the enterprise safety production data at least twice, wherein one time is a random complement, one time is a full zero complement, and the complement digit is equal to the error digit; and respectively inputting the complemented enterprise safety production data into the variational self-encoder to obtain recovery data.
9. The method of claim 3, wherein said ranking said degree of abnormality is specifically: the degree of abnormality is classified into five stages.
10. The method of claim 8, wherein the impairment of the variational self-encoder comprises: the difference between the input data and the recovered data after random number padding, the difference between the input data and the recovered data after all zero number padding, and the difference between the reference distribution and the output distribution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111231028.2A CN113971513B (en) | 2021-10-22 | 2021-10-22 | Data storage and optimization method of enterprise security risk management platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111231028.2A CN113971513B (en) | 2021-10-22 | 2021-10-22 | Data storage and optimization method of enterprise security risk management platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113971513A true CN113971513A (en) | 2022-01-25 |
| CN113971513B CN113971513B (en) | 2024-08-06 |
Family
ID=79587803
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111231028.2A Active CN113971513B (en) | 2021-10-22 | 2021-10-22 | Data storage and optimization method of enterprise security risk management platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113971513B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109815223A (en) * | 2019-01-21 | 2019-05-28 | 北京科技大学 | A kind of complementing method and complementing device for industry monitoring shortage of data |
| CN111708739A (en) * | 2020-05-21 | 2020-09-25 | 北京奇艺世纪科技有限公司 | Method and device for detecting abnormality of time series data, electronic device and storage medium |
| CN112381790A (en) * | 2020-11-13 | 2021-02-19 | 天津大学 | Abnormal image detection method based on depth self-coding |
| CN112529767A (en) * | 2020-12-01 | 2021-03-19 | 平安科技（深圳）有限公司 | Image data processing method, image data processing device, computer equipment and storage medium |
| CN113315524A (en) * | 2021-05-26 | 2021-08-27 | 武汉理工大学 | Landmark data compression transmission method and device based on deep learning |
| WO2021189904A1 (en) * | 2020-10-09 | 2021-09-30 | 平安科技（深圳）有限公司 | Data anomaly detection method and apparatus, and electronic device and storage medium |
-
2021
- 2021-10-22 CN CN202111231028.2A patent/CN113971513B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109815223A (en) * | 2019-01-21 | 2019-05-28 | 北京科技大学 | A kind of complementing method and complementing device for industry monitoring shortage of data |
| CN111708739A (en) * | 2020-05-21 | 2020-09-25 | 北京奇艺世纪科技有限公司 | Method and device for detecting abnormality of time series data, electronic device and storage medium |
| WO2021189904A1 (en) * | 2020-10-09 | 2021-09-30 | 平安科技（深圳）有限公司 | Data anomaly detection method and apparatus, and electronic device and storage medium |
| CN112381790A (en) * | 2020-11-13 | 2021-02-19 | 天津大学 | Abnormal image detection method based on depth self-coding |
| CN112529767A (en) * | 2020-12-01 | 2021-03-19 | 平安科技（深圳）有限公司 | Image data processing method, image data processing device, computer equipment and storage medium |
| CN113315524A (en) * | 2021-05-26 | 2021-08-27 | 武汉理工大学 | Landmark data compression transmission method and device based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113971513B (en) | 2024-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8375218B2 (en) | Pre-processing biometric parameters before encoding and decoding | |
| US7779268B2 (en) | Biometric based user authentication and data encryption | |
| US8281138B2 (en) | Steganalysis of suspect media | |
| Li et al. | A novel support vector machine approach to high entropy data fragment classification | |
| JP5288935B2 (en) | Preprocessing method for biometric parameters before encoding and decoding | |
| CN115296862A (en) | Network data secure transmission method based on data coding | |
| US20240078330A1 (en) | A method and system for lossy compression of log files of data | |
| CN116389170B (en) | Network information security management method | |
| Tatwawadi et al. | On universal compression with constant random access | |
| Radhakrishnan et al. | An alternate partitioning technique to quantify the regularity of complex time series | |
| CN115277189B (en) | Unsupervised intrusion flow detection and identification method based on generation type countermeasure network | |
| CN117273764B (en) | Anti-counterfeiting management method and system for electronic atomizer | |
| CN113971513A (en) | Data storage and optimization method of enterprise security risk management platform | |
| Ferdowsi et al. | Sparse ternary codes for similarity search have higher coding gain than dense binary codes | |
| Voloshynovskiy et al. | Conception and limits of robust perceptual hashing: towards side information assisted hash functions | |
| EP4100954A1 (en) | Improved quality value compression framework in aligned sequencing data based on novel contexts | |
| US7984008B2 (en) | Evaluating the predictability of a pass phrase | |
| WO2022079466A1 (en) | Data authentication for data compression | |
| Pei et al. | Multiple images steganography of JPEG images based on optimal payload distribution | |
| CN113723071A (en) | Electronic file checking method, system, storage medium and equipment | |
| McCarthy et al. | A framework for soft hashing and its application to robust image hashing | |
| Zhou et al. | Polar codes for identification systems | |
| CN117220973B (en) | Data security exchange method and system based on public key encryption | |
| EP4300312A1 (en) | File regeneration method and computer programs for same | |
| CN115348010B (en) | Method and system suitable for eliminating residual error codes of continuous variable quantum key distribution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |