CN111753328B - Private data leakage risk detection method and system - Google Patents
Private data leakage risk detection method and system Download PDFInfo
- Publication number
- CN111753328B CN111753328B CN202010496260.8A CN202010496260A CN111753328B CN 111753328 B CN111753328 B CN 111753328B CN 202010496260 A CN202010496260 A CN 202010496260A CN 111753328 B CN111753328 B CN 111753328B
- Authority
- CN
- China
- Prior art keywords
- characteristic information
- time period
- historical
- target user
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
- G06F21/577—Assessing vulnerabilities and evaluating computer system security
Abstract
The embodiment of the specification provides a method and a system for detecting leakage risk of private data, wherein the method comprises the following steps: processing the behavior data of the target user in the current time period through the streaming processing node to obtain current characteristic information of the target user in each target dimension; acquiring current characteristic information and historical characteristic information of a target user in each target dimension through an analysis node, and detecting whether the target user has a risk of privacy data leakage according to the current characteristic information and the historical characteristic information; the historical characteristic information is obtained by processing historical behavior data of a target user in a historical time period by a batch processing node; the time length of the historical time period is greater than the time length of the current time period.
Description
Technical Field
The application relates to the technical field of data processing, in particular to a method and a system for detecting leakage risk of private data.
Background
With the continuous development of society, people have stronger safety awareness, such as data safety, information safety and the like. Especially for enterprises or merchants, in order to protect users or the interests of the enterprises or merchants, the security of information and the protection of private data are particularly emphasized.
Therefore, to prevent information leakage, user behavior may be analyzed through various detection analysis approaches at present. Therefore, how to accurately and timely analyze the user behavior to detect whether there is a risk of disclosure of private data becomes a technical problem that needs to be solved urgently at present.
Disclosure of Invention
The embodiment of the specification provides a private data leakage risk detection method. The detection method comprises the following steps: and processing the behavior data of the target user in the current time period through the streaming processing node to obtain the current characteristic information of the target user in each target dimension. And acquiring the current characteristic information and the historical characteristic information of the target user in each target dimension through an analysis node. The historical characteristic information is obtained by processing the historical behavior data of the target user in a historical time period by a batch processing node. The time length of the historical time period is greater than the time length of the current time period. And detecting whether the target user has the risk of privacy data leakage or not according to the current characteristic information and the historical characteristic information through the analysis node.
The embodiment of the specification further provides a system for detecting the leakage risk of the private data. Wherein, this system includes: a streaming processing node and an analysis node. And the streaming processing node is used for processing the behavior data of the target user in the current time period to obtain the current characteristic information of the target user in each target dimension. And the analysis node is used for acquiring the current characteristic information and the historical characteristic information of the target user in each target dimension. And detecting whether the target user has the risk of privacy data leakage or not according to the current characteristic information and the historical characteristic information through the analysis node. The historical characteristic information is obtained by processing historical behavior data of the target user in a historical time period by a batch processing node; the time length of the historical time period is greater than the time length of the current time period.
The embodiment of the specification also provides a storage medium. The storage medium is used for storing computer executable instructions, and the executable instructions realize the following processes when executed: and processing the behavior data of the target user in the current time period through the streaming processing node to obtain the current characteristic information of the target user in each target dimension. And acquiring the current characteristic information and the historical characteristic information of the target user in each target dimension through an analysis node. The historical characteristic information is obtained by processing historical behavior data of the target user in a historical time period by a batch processing node; the time length of the historical time period is greater than the time length of the current time period. And detecting whether the target user has the risk of privacy data leakage or not according to the current characteristic information and the historical characteristic information through the analysis node.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a method for detecting a risk of disclosure of private data according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of a method for detecting a risk of disclosure of private data according to an embodiment of the present disclosure;
fig. 3 is a second flowchart of a method for detecting a risk of disclosure of private data according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a system for detecting leakage risk of private data according to an embodiment of the present disclosure;
fig. 5 is a second schematic structural diagram of a system for detecting a risk of disclosure of private data according to an embodiment of the present disclosure.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The idea of the embodiment of the present specification is that behavior data of a user is analyzed by a way of combining streaming processing and batch processing, and the generated behavior data can be processed in time by the streaming processing, so that timeliness of data processing can be ensured, and data in the whole data period can be taken into account by the batch processing, so that accuracy of data processing can be satisfied; therefore, the timeliness and the accuracy of data processing can be considered simultaneously through a mode of combining streaming processing and batch processing, and whether the user has the risk of privacy data leakage can be timely and accurately found. Based on this, embodiments of the present specification provide a method and a system for detecting a leakage risk of private data, which will be described in detail below.
Fig. 1 is a flowchart of a method for detecting a risk of disclosure of private data according to an embodiment of the present disclosure, where the method shown in fig. 1 includes at least the following steps:
and 102, processing the behavior data of the target user in the current time period through the streaming processing node to obtain the current characteristic information of the target user in each target dimension.
The streaming processing node may be a server or a computing device capable of performing streaming processing, or the streaming processing node is one of processing modules on the computing device or the server. The specific value of the current time period may be a shorter time node with a length of 3 minutes, 5 minutes, or the like. For example, the current time period may be a time period ranging from 07 hours at 3/18/15/2020 to 09 minutes at 3/18/15/2020.
Optionally, in a specific application scenario, the behavior data of the target user may be browsing behavior information of the target user browsing a certain website or operation behavior information of the target user operating on a certain page, and the like.
Specifically, after the streaming processing node acquires the behavior data of the target user, the behavior data in the current time period is processed from different dimensions, so that a statistical result of each dimension is obtained and is used as the current feature information corresponding to the dimension.
For example, in a specific embodiment, the behavior data may be operation behavior data of the user a performing an operation on a certain application, and the target dimension may be fields of "total operation duration", "operation page information", "operation behavior", and the like.
104, acquiring current characteristic information and historical characteristic information of target users in each target dimension through analysis nodes; the historical characteristic information is obtained by processing historical behavior data of a target user in a historical time period by a batch processing node; the time length of the historical time period is greater than the time length of the current time period.
The batch processing node may be a server or a computing device capable of performing batch processing, or the batch processing node is a processing module on the server or the computing device.
Optionally, in a specific embodiment, the time length of the historical time period may be one or more output periods of the behavior data. For example, the target user may be presented with a time period from when the target user starts logging on to the application to when the current login is completed. Specifically, the specific value of the historical time period may be 24 hours, 36 hours, 48 hours, and the like. Specific values thereof are set according to actual application scenarios, which are not limited in the embodiments of this specification.
In addition, it should be noted that the historical time period generally refers to a time period before the current time period. For example, in a specific embodiment, the current time period may be a time period from 06 hours at 15 days 3/18/2020 to 09 minutes at 15 days 3/18/2020, and the historical time period may be a time period from 06 minutes at 15 days 15 at 3/17/2020 to 06 minutes at 15 days 15 at 3/18/2020. Of course, this is only an exemplary illustration, as long as the ending time point of the historical time period is earlier than the starting time point of the current time period, and the length of the value taking time of the historical time period may be set according to the actual application scenario.
And step 106, detecting whether the target user has the risk of privacy data leakage or not according to the current characteristic information and the historical characteristic information by the analysis node.
The analysis node may be a server or a computing device capable of performing behavioral anomaly analysis, or the analysis node is an analysis module on the server or the computing device.
According to the method for detecting the leakage risk of the private data, the behavior data are analyzed in a mode of combining stream processing and batch processing, and the generated behavior data can be processed in time through the stream processing, so that timeliness of data processing can be guaranteed, data in the whole data period can be taken into account through the batch processing, and accuracy of the data processing can be met; therefore, the timeliness and the accuracy of data processing can be considered simultaneously through a mode of combining streaming processing and batch processing, and whether the user has the risk of privacy data leakage can be timely and accurately found.
Optionally, in a specific embodiment, in the step 106, detecting whether the target user has a risk of private data leakage by using the analysis node according to the current feature information and the historical feature information, specifically includes the following steps:
analyzing the historical characteristic information by using a machine learning algorithm through an analysis node, and determining normal characteristic information and/or abnormal characteristic information in the historical characteristic information; and comparing the current characteristic information with the normal characteristic information and/or the abnormal characteristic information, and determining whether the target user has abnormal behaviors in the current time period according to a comparison result so as to detect whether the target user has the risk of privacy data leakage.
Specifically, in the embodiment of the present specification, a required machine learning algorithm, such as a k-nearest neighbor classification algorithm, may be preset in the analysis node. In this way, after the analysis node acquires the current feature information and the historical feature information, the preset k-nearest neighbor classification algorithm is used for performing cluster analysis on the historical feature information corresponding to each target dimension respectively to obtain normal feature information and abnormal feature information in each historical feature information.
For example, in a specific embodiment, the behavior data of the user in the historical time period is statistically analyzed from three dimensions by the batch processing node, so that the historical feature information 1 corresponding to the target dimension 1, the historical feature information 2 corresponding to the target dimension 2, and the historical feature information 3 corresponding to the target dimension 3 can be obtained. Because a plurality of pieces of behavior data exist in the historical time period, a plurality of pieces of historical feature information corresponding to each target dimension are provided. The historical characteristic information 1, the historical characteristic information 2 and the historical characteristic information 3 are respectively clustered through the analysis nodes, so that normal characteristic information and abnormal characteristic information in the historical characteristic information 1, normal characteristic information and abnormal characteristic information in the historical characteristic information 2 and normal characteristic information and abnormal characteristic information in the historical characteristic information 3 are obtained.
Therefore, by comparing the current characteristic information of the target user with the normal characteristic information and/or the abnormal characteristic information, whether the current characteristic information is abnormal characteristic information can be determined, and if the characteristic information of the target user in the current time period has abnormal characteristic information, the abnormal behavior of the target user in the current time period is indicated; and then, detecting whether the abnormal behavior has the risk of causing the leakage of the private data or not based on the abnormal behavior of the user.
The abnormal feature information refers to a behavior feature different from a normal behavior feature. For example, in a specific embodiment, after obtaining the information a each time, the user directly writes the information a into the database; however, after the information a is acquired, the information a is copied to the usb disk, and the behavior characteristic is an abnormal characteristic.
Optionally, in a specific embodiment, after the batch processing node processes the behavior data in the historical time period, the historical feature information of each target dimension obtained after the processing is stored in the database, so that after the analysis node obtains the current feature information of each target dimension sent by the streaming processing node, the historical feature information of each target dimension of the target user in the historical time period is read from the database.
The database for storing the historical feature information may be a Key-Value database.
Generally, if the analysis result obtained in step 106 indicates that the target user has an abnormal behavior in the current time period;
the method provided in this embodiment of the present specification further includes:
analyzing the behavior data and the current characteristic information of the abnormal behavior through an Online Analytical Processing (OLAP) node, and determining the generation reason of the abnormal behavior;
and/or the presence of a gas in the gas,
and executing early warning operation on the abnormal behavior through the early warning node.
Specifically, the OLAP node and the early warning node may be separate servers or devices, or may be a certain module on a server or a device.
In a specific implementation manner, the early warning node may send an early warning prompt to the relevant staff, for example, notify the relevant staff in the form of sending an email, a short message, an instant messaging message, and the like, so that the relevant staff can respond to and process the relevant staff in time. Of course, in some embodiments, the abnormal behavior of the target user may be classified into different levels, and the early warning operations with different degrees of urgency are executed for different levels.
Optionally, in a specific embodiment, if the analysis result in step 106 indicates that the target user has an abnormal behavior in the current time period;
correspondingly, the method provided by the embodiment of the present specification further includes:
and transmitting the behavior data corresponding to the abnormal behavior and the current characteristic information to the first message queue through the analysis node, so that the OLAP node and/or the early warning node reads the behavior data corresponding to the abnormal behavior and the current characteristic information from the first message queue according to the message arrangement sequence in the first message queue.
In a specific implementation manner, before the processing, by the streaming processing node, behavior data of the target user in the current time period, the method provided by the embodiment of the present specification further includes:
and when one piece of behavior data is generated, the behavior data is acquired through the data acquisition node and written into the second message queue.
Correspondingly, before the processing of the behavior data of the target user in the current time period by the streaming processing node, the method provided by the embodiment of the present specification further includes:
and reading the behavior data in the current time period from the second message queue through the streaming processing node.
Correspondingly, the method provided by the embodiment of the present specification further includes:
behavior data of the target user in a historical time period are obtained from the distributed storage database through the batch processing nodes, and the behavior data are processed to obtain historical characteristic information of the target user in each target dimension; and the distributed storage database reads the behavior data from the second message queue according to a set period.
Fig. 2 shows a schematic flowchart of a method for detecting a risk of disclosure of private data according to an embodiment of the present specification. As shown in fig. 2, behavior data of a user is collected by a data collection node, and the collected behavior data is written into a second message queue. And the streaming processing node reads the behavior data in the current time period from the message queue and processes the read behavior data to obtain current characteristic information corresponding to each target dimension. The distributed storage system reads behavior data of a user from the message queue according to a set time interval, the batch processing node reads the behavior data in a historical time period from the distributed storage system, processes the behavior data to obtain historical feature information corresponding to each target dimension, and stores the historical feature information into a Key-Value database. The stream processing node reads current characteristic information and historical characteristic information from a Key-Value database and sends the current characteristic information and the historical characteristic information to the analysis node, and the analysis node analyzes whether the user has abnormal behaviors in the current time period according to the current characteristic information and the historical characteristic information so as to detect whether the target user has risk of privacy data leakage. And if the analysis result indicates that the user has abnormal behaviors in the current time period, writing behavior data corresponding to the abnormal behaviors and the current characteristic information into the first message queue. And the OLAP node reads the behavior data and the current characteristic information corresponding to the abnormal behavior from the first message queue and analyzes the generation reason of the abnormal behavior according to the behavior data and the current characteristic information. In addition, the early warning node reads the behavior data and the current characteristic information corresponding to the abnormal behavior from the first message queue, so that early warning operation is executed.
In addition, in the embodiments of the present description, kafka may be used for data collection and message queuing, hdfs may be used for distributed storage, hadoop may be used for batch processing, hbase may be used for kv storage, and flink may be used for stream processing.
Fig. 3 is a second flowchart of a method for detecting a risk of disclosure of private data according to an embodiment of the present disclosure, where the method shown in fig. 2 at least includes the following steps:
And step 304, the streaming processing node reads the behavior data in the current time period from the second message queue and processes the behavior data to obtain the current feature information of the target user in each target dimension.
And step 308, reading historical characteristic information of the target user in the target dimension in the historical time period from the Key-Value database by the streaming processing node.
And 310, performing clustering analysis on the historical characteristic information through the analysis node, and determining normal characteristic information and abnormal characteristic information in the historical characteristic information.
And step 312, comparing the current characteristic information with the normal characteristic information and the abnormal characteristic information through the analysis node, and determining whether the target user has abnormal behavior in the current time period according to the comparison result so as to detect whether the target user has the risk of privacy data leakage.
In step 314, if the analysis result indicates that the target user has an abnormal behavior in the current time period, the behavior data corresponding to the abnormal behavior and the current feature information are transmitted to the first message queue.
According to the method for detecting the leakage risk of the private data, the behavior data are analyzed in a mode of combining the streaming processing and the batch processing, and the generated behavior data can be processed in time through the streaming processing, so that the timeliness of the data processing can be ensured, and the data in the whole data period can be taken into account through the batch processing, so that the accuracy of the data processing can be met; therefore, the timeliness and the accuracy of data processing can be considered simultaneously through a mode of combining streaming processing and batch processing, and whether the user has the risk of privacy data leakage can be timely and accurately found.
Based on the same idea, an embodiment of the present specification further provides a system for detecting a risk of disclosure of private data, which is used to execute the method for detecting a risk of disclosure of private data provided in the embodiment of the present specification, where fig. 4 is one of schematic structural diagrams of the system for detecting a risk of disclosure of private data provided in the embodiment of the present specification, and the system shown in fig. 4 includes: a streaming processing node 402 and an analysis node 404;
the streaming processing node 402 is configured to process behavior data of a target user in a current time period to obtain current feature information of the target user in each target dimension;
the analysis node 404 is configured to obtain the current feature information and historical feature information of the target user in each target dimension; detecting whether the target user has the risk of privacy data leakage or not according to the current characteristic information and the historical characteristic information through the analysis node;
the historical characteristic information is obtained by processing historical behavior data of the target user in the historical time period by a batch processing node; the time length of the historical time period is greater than the time length of the current time period.
Fig. 5 is a second schematic structural diagram of a system for detecting leakage risk of private data according to an embodiment of the present disclosure, and as shown in fig. 5, the system further includes a batch processing node 406;
the batch processing node 406 is configured to acquire behavior data of the target user in the historical time period from a distributed storage database, and process the behavior data to obtain historical feature information of the target user in each target dimension; and the distributed storage database reads the behavior data from the second message queue according to a set period.
Optionally, the analysis node 404 is specifically configured to:
analyzing the historical characteristic information by using a machine learning algorithm, and determining normal characteristic information and/or abnormal characteristic information in the historical characteristic information; and comparing the current characteristic information with the normal characteristic information and/or the abnormal characteristic information, and determining whether the target user has abnormal behaviors in the current time period according to a comparison result so as to detect whether the target user has the risk of privacy data leakage.
Optionally, the machine learning algorithm is a k-nearest neighbor classification algorithm.
Optionally, as shown in fig. 5, if the analysis result indicates that the target user has an abnormal behavior in the current time period;
the system further comprises:
the OLAP node 408 is configured to analyze behavior data of an abnormal behavior and the current feature information, and determine a cause of the abnormal behavior;
alternatively, the first and second electrodes may be,
and the early warning node 410 is configured to perform an early warning operation on the abnormal behavior.
Optionally, the analysis node 404 is further configured to:
and transmitting the behavior data and the current characteristic information corresponding to the abnormal behavior to a first message queue, so that the OLAP node and/or the early warning node reads the behavior data and the current characteristic information corresponding to the abnormal behavior from the first message queue according to the message arrangement sequence in the first message queue.
Optionally, the streaming processing node 402 is further configured to:
and reading the behavior data in the current time period from a second message queue.
Optionally, the system further includes a data acquisition node;
and the data acquisition node is used for acquiring the behavior data and writing the behavior data into the second message queue when generating one piece of behavior data.
According to the privacy data leakage risk detection system provided by the embodiment of the specification, behavior data are analyzed in a mode of combining stream processing and batch processing, and the generated behavior data can be processed in time through the stream processing, so that timeliness of data processing can be guaranteed, and data in the whole data period can be taken into account through the batch processing, so that accuracy of the data processing can be met; therefore, the timeliness and the accuracy of data processing can be considered simultaneously through a mode of combining streaming processing and batch processing, and whether the user has the risk of privacy data leakage can be timely and accurately found.
Further, based on the methods shown in fig. 1 to fig. 3, in a specific embodiment, the storage medium may be a usb disk, an optical disk, a hard disk, or the like, and when executed by a processor, the storage medium stores computer-executable instruction information that implements the following processes:
processing behavior data of a target user in a current time period through a streaming processing node to obtain current characteristic information of the target user in each target dimension;
acquiring the current characteristic information and historical characteristic information of the target user in each target dimension through an analysis node; the historical characteristic information is obtained by processing historical behavior data of the target user in the historical time period by a batch processing node; the time length of the historical time period is greater than that of the current time period;
and detecting whether the target user has the risk of privacy data leakage or not according to the current characteristic information and the historical characteristic information through the analysis node.
Optionally, when the computer-executable instruction information stored in the storage medium is executed by the processor, the detecting, by the analysis node, whether the target user is at risk of private data disclosure according to the current feature information and the historical feature information includes:
analyzing the historical characteristic information by using a machine learning algorithm through the analysis node, and determining normal characteristic information and/or abnormal characteristic information in the historical characteristic information;
and comparing the current characteristic information with the normal characteristic information and/or the abnormal characteristic information, and determining whether the target user has abnormal behaviors in the current time period according to a comparison result so as to detect whether the target user has the risk of privacy data leakage.
Optionally, the storage medium stores computer-executable instruction information that, when executed by the processor, the machine learning algorithm is a k-nearest neighbor classification algorithm.
Optionally, when the computer-executable instruction information stored in the storage medium is executed by the processor, if the analysis result indicates that the target user has an abnormal behavior in the current time period;
correspondingly, after the analysis node analyzes the abnormal behavior of the target user according to the current characteristic information and the historical characteristic information, the following steps can be further executed:
analyzing the behavior data of the abnormal behavior and the current characteristic information by an online analysis processing OLAP node, and determining the generation reason of the abnormal behavior;
and/or the presence of a gas in the atmosphere,
and executing early warning operation on the abnormal behavior through an early warning node.
Optionally, when the computer-executable instruction information stored in the storage medium is executed by the processor, if the analysis result indicates that the target user has an abnormal behavior in the current time period;
correspondingly, after the analysis node analyzes the abnormal behavior of the target user according to the current characteristic information and the historical characteristic information, the following steps can be further executed:
and transmitting the behavior data and the current characteristic information corresponding to the abnormal behavior to a first message queue through the analysis node, so that the OLAP node and/or the early warning node reads the behavior data and the current characteristic information corresponding to the abnormal behavior from the first message queue according to the message arrangement sequence in the first message queue.
Optionally, the storage medium stores computer-executable instruction information, which when executed by the processor, further performs the following steps:
acquiring behavior data of the target user in the historical time period from a distributed storage database through a batch processing node, and processing the behavior data to obtain historical characteristic information of the target user in each target dimension; and the distributed storage database reads the behavior data from the second message queue according to a set period.
Optionally, the storage medium stores computer-executable instruction information, which when executed by the processor, may further perform the following steps before the streaming processing node processes the behavior data of the target user in the current time period:
reading, by the streaming processing node, the behavior data in the current time period from a second message queue.
Optionally, the storage medium stores computer-executable instruction information, which when executed by the processor, may further perform the following steps before the streaming processing node processes the behavior data of the target user in the current time period:
and when one piece of behavior data is generated, acquiring the behavior data through a data acquisition node, and writing the behavior data into the second message queue.
When the computer-executable instruction information stored in the storage medium provided in the embodiment of the present specification is executed by the processor, behavior data is analyzed in a manner of combining stream processing and batch processing, and because the generated behavior data can be processed in time through stream processing, timeliness of data processing can be ensured, and data in the whole data cycle can be taken into account through batch processing, so that accuracy of data processing can be satisfied; therefore, the timeliness and the accuracy of data processing can be considered simultaneously through a mode of combining streaming processing and batch processing, and whether the user has the risk of privacy data leakage can be timely and accurately found.
In the 90 s of the 20 th century, improvements in a technology could clearly distinguish between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing an Integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as ABEL (Advanced Boolean Expression Language), AHDL (alternate Hardware Description Language), traffic, CUPL (core universal Programming Language), HDCal, jhddl (Java Hardware Description Language), lava, lola, HDL, PALASM, rhyd (Hardware Description Language), and vhigh-Language (Hardware Description Language), which is currently used in most popular applications. It will also be apparent to those skilled in the art that hardware circuitry for implementing the logical method flows can be readily obtained by a mere need to program the method flows with some of the hardware description languages described above and into an integrated circuit.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic for the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may thus be considered a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instruction information. These computer program instruction information may be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing apparatus to produce a machine, such that the instruction information executed by the processor of the computer or other programmable data processing apparatus produce a means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instruction information may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instruction information stored in the computer-readable memory produce an article of manufacture including instruction information means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instruction information may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instruction information executed on the computer or other programmable apparatus provides steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instruction information, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The application may be described in the general context of computer-executable instruction information, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
All the embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement or the like made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.
Claims (15)
1. A method of risk of disclosure of private data, the method comprising:
processing behavior data of a target user in a current time period through a streaming processing node to obtain current feature information of the target user in each target dimension;
acquiring the current characteristic information and historical characteristic information of the target user in each target dimension through an analysis node; the historical characteristic information is obtained by processing historical behavior data of the target user in a historical time period by a batch processing node; the time length of the historical time period is greater than that of the current time period; the time length of the historical time period is one or more output periods of the behavior data;
analyzing the historical characteristic information by using a machine learning algorithm through the analysis node, and determining normal characteristic information and/or abnormal characteristic information in the historical characteristic information;
and comparing the current characteristic information with the normal characteristic information and/or the abnormal characteristic information, and determining whether the target user has abnormal behaviors in the current time period according to a comparison result so as to detect whether the target user has the risk of leaking privacy data.
2. The method of claim 1, the machine learning algorithm being a k-nearest neighbor classification algorithm.
3. The method of claim 1, wherein if the analysis result indicates that the target user has abnormal behavior within the current time period;
correspondingly, the method further comprises the following steps:
analyzing the behavior data of the abnormal behavior and the current characteristic information by an online analysis processing OLAP node, and determining the generation reason of the abnormal behavior;
and/or the presence of a gas in the atmosphere,
and executing early warning operation on the abnormal behavior through an early warning node.
4. The method of claim 3, wherein if the analysis result indicates that the target user has abnormal behavior within the current time period;
correspondingly, the method further comprises the following steps:
and transmitting the behavior data and the current characteristic information corresponding to the abnormal behavior to a first message queue through the analysis node, so that the OLAP node and/or the early warning node reads the behavior data and the current characteristic information corresponding to the abnormal behavior from the first message queue according to the message arrangement sequence in the first message queue.
5. The method of claim 1, further comprising:
acquiring behavior data of the target user in the historical time period from a distributed storage database through a batch processing node, and processing the behavior data to obtain historical characteristic information of the target user in each target dimension; and the distributed storage database reads the behavior data from the second message queue according to a set period.
6. The method of claim 1, before processing the behavior data of the target user over the current time period by the streaming processing node, the method further comprising:
reading, by the streaming processing node, the behavior data in the current time period from a second message queue.
7. The method of claim 5 or 6, before processing the behavior data of the target user in the current time period by the streaming processing node, the method further comprising:
and when one piece of behavior data is generated, acquiring the behavior data through a data acquisition node, and writing the behavior data into the second message queue.
8. A system for detecting leakage risk of private data, the system comprising a streaming processing node and an analysis node;
the streaming processing node is used for processing the behavior data of the target user in the current time period to obtain the current characteristic information of the target user in each target dimension;
the analysis node is used for acquiring the current characteristic information and the historical characteristic information of the target user in each target dimension; analyzing the historical characteristic information by using a machine learning algorithm through the analysis node, and determining normal characteristic information and/or abnormal characteristic information in the historical characteristic information; comparing the current characteristic information with the normal characteristic information and/or the abnormal characteristic information, and determining whether the target user has abnormal behaviors in the current time period according to a comparison result so as to detect whether the target user has a risk of privacy data leakage;
the historical characteristic information is obtained by processing historical behavior data of the target user in a historical time period by a batch processing node; the time length of the historical time period is greater than that of the current time period; the time length of the historical time period is one or more output periods of the behavior data.
9. The system of claim 8, further comprising a batch processing node;
the batch processing node is used for acquiring behavior data of the target user in the historical time period from a distributed storage database, and processing the behavior data to obtain historical characteristic information of the target user in each target dimension; and the distributed storage database reads the behavior data from the second message queue according to a set period.
10. The system of claim 8, the machine learning algorithm is a k-nearest neighbor classification algorithm.
11. The system of claim 8, wherein if the analysis result indicates that the target user has abnormal behavior within the current time period;
the system further comprises:
the OLAP node is used for analyzing the behavior data of the abnormal behavior and the current characteristic information and determining the generation reason of the abnormal behavior;
alternatively, the first and second electrodes may be,
and the early warning node is used for executing early warning operation on the abnormal behavior.
12. The system of claim 11, if the analysis result indicates that the target user has abnormal behavior within the current time period;
the analysis node is further configured to:
and transmitting the behavior data and the current characteristic information corresponding to the abnormal behavior to a first message queue, so that the OLAP node and/or the early warning node reads the behavior data and the current characteristic information corresponding to the abnormal behavior from the first message queue according to the message arrangement sequence in the first message queue.
13. The system of claim 8, the streaming processing node further to:
and reading the behavior data in the current time period from a second message queue.
14. The system of claim 9 or 13, further comprising a data collection node;
and the data acquisition node is used for acquiring the behavior data and writing the behavior data into the second message queue when generating one piece of behavior data.
15. A storage medium storing computer-executable instructions that, when executed, implement the following:
processing behavior data of a target user in a current time period through a streaming processing node to obtain current feature information of the target user in each target dimension;
acquiring the current characteristic information and historical characteristic information of the target user in each target dimension through an analysis node; the historical characteristic information is obtained by processing historical behavior data of the target user in a historical time period by a batch processing node; the time length of the historical time period is greater than that of the current time period; the time length of the historical time period is one or more output periods of the behavior data;
analyzing the historical characteristic information by using a machine learning algorithm through the analysis node, and determining normal characteristic information and/or abnormal characteristic information in the historical characteristic information;
and comparing the current characteristic information with the normal characteristic information and/or the abnormal characteristic information, and determining whether the target user has abnormal behaviors in the current time period according to a comparison result so as to detect whether the target user has the risk of privacy data leakage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010496260.8A CN111753328B (en) | 2020-06-03 | 2020-06-03 | Private data leakage risk detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010496260.8A CN111753328B (en) | 2020-06-03 | 2020-06-03 | Private data leakage risk detection method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111753328A CN111753328A (en) | 2020-10-09 |
| CN111753328B true CN111753328B (en) | 2023-03-17 |
Family
ID=72673937
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010496260.8A Active CN111753328B (en) | 2020-06-03 | 2020-06-03 | Private data leakage risk detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111753328B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112364346B (en) * | 2020-10-27 | 2024-03-22 | 杭州安恒信息技术股份有限公司 | Leakage data detection method, device, equipment and medium |
| CN112541193A (en) * | 2020-12-10 | 2021-03-23 | 支付宝(杭州)信息技术有限公司 | Method and device for protecting private data |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006228037A (en) * | 2005-02-18 | 2006-08-31 | Hitachi Software Eng Co Ltd | Information leakage preventing method for database and web database system |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599709B (en) * | 2015-10-15 | 2021-08-17 | 中兴通讯股份有限公司 | Method, device and terminal for preventing privacy information leakage |
| CN109003075A (en) * | 2017-06-07 | 2018-12-14 | 阿里巴巴集团控股有限公司 | A kind of Risk Identification Method and device |
| CN109684863B (en) * | 2018-09-07 | 2024-01-19 | 平安科技（深圳）有限公司 | Data leakage prevention method, device, equipment and storage medium |
| CN109525558B (en) * | 2018-10-22 | 2022-02-22 | 深信服科技股份有限公司 | Data leakage detection method, system, device and storage medium |
| CN110322349B (en) * | 2019-06-25 | 2023-08-22 | 创新先进技术有限公司 | Data processing method, device and equipment |
| CN110458571B (en) * | 2019-07-05 | 2023-06-02 | 创新先进技术有限公司 | Risk identification method, device and equipment for information leakage |
| CN110798472B (en) * | 2019-11-01 | 2022-01-07 | 杭州数梦工场科技有限公司 | Data leakage detection method and device |
| CN111143175A (en) * | 2019-11-29 | 2020-05-12 | 北京浪潮数据技术有限公司 | Risk behavior detection method, device, equipment and computer storage medium |
-
2020
- 2020-06-03 CN CN202010496260.8A patent/CN111753328B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006228037A (en) * | 2005-02-18 | 2006-08-31 | Hitachi Software Eng Co Ltd | Information leakage preventing method for database and web database system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111753328A (en) | 2020-10-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107526667B (en) | Index abnormality detection method and device and electronic equipment | |
| CN107066519B (en) | Task detection method and device | |
| CN111753328B (en) | Private data leakage risk detection method and system | |
| CN110569428A (en) | recommendation model construction method, device and equipment | |
| CN110635962B (en) | Abnormity analysis method and device for distributed system | |
| CN110895503B (en) | Application performance monitoring method and client | |
| CN112966113A (en) | Data risk prevention and control method, device and equipment | |
| CN110888756A (en) | Diagnostic log generation method and device | |
| CN111930810A (en) | Data rule mining method and device | |
| CN111639011A (en) | Data monitoring method, device and equipment | |
| CN110675028A (en) | Block chain-based food safety supervision method, device, equipment and system | |
| CN113760658A (en) | Monitoring method, device and equipment | |
| CN111078435A (en) | Service processing method and device and electronic equipment | |
| CN110968483A (en) | Service data acquisition method and device and electronic equipment | |
| CN113435950B (en) | Bill processing method and device | |
| CN115567371A (en) | Abnormity detection method, device, equipment and readable storage medium | |
| CN114722972A (en) | Anomaly detection method and device | |
| CN115204395A (en) | Data processing method, device and equipment | |
| CN111242195B (en) | Model, insurance wind control model training method and device and electronic equipment | |
| CN110245136B (en) | Data retrieval method, device, equipment and storage equipment | |
| CN112015570A (en) | Message reminding processing method and device | |
| CN113254823A (en) | Webpage data processing method and device | |
| CN115510927B (en) | Fault detection method, device and equipment | |
| CN112215471B (en) | Index transaction detection method and device | |
| CN111461352B (en) | Model training method, service node identification device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40039463 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |