CN111639359A - Method and system for detecting and early warning privacy risks of social network pictures - Google Patents
Method and system for detecting and early warning privacy risks of social network pictures Download PDFInfo
- Publication number
- CN111639359A CN111639359A CN202010323133.8A CN202010323133A CN111639359A CN 111639359 A CN111639359 A CN 111639359A CN 202010323133 A CN202010323133 A CN 202010323133A CN 111639359 A CN111639359 A CN 111639359A
- Authority
- CN
- China
- Prior art keywords
- picture
- privacy
- nodes
- data set
- key elements
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9536—Search customisation based on social or collaborative filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Bioethics (AREA)
- Neurology (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Medical Informatics (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Image Analysis (AREA)
Abstract
A method for social network picture privacy risk detection and early warning, comprising: the method comprises the following steps: extracting key elements in the picture by using a target detection frame and obtaining information of the key elements; step two: collecting a data set of whether the picture is private or not, carrying out operation of the step one on each picture in the data set, then carrying out statistics on the whole data set to obtain the association degrees of various key elements with the private and public pictures, and constructing a knowledge graph according to the association degrees; step three: extracting the characteristics of the whole picture and the key elements of the picture by using a neural network, and constructing the neural network by using the knowledge graph in the step two, so as to fuse the characteristics of the whole picture and the key elements of the picture and obtain the final expression of the picture; and step four: and predicting the privacy risk of the picture by utilizing a neural network based on the final expression of the picture in the third step.
Description
Technical Field
The invention relates to privacy protection of a social network, in particular to a privacy risk detection and early warning method and system used in social network picture sharing.
Background
With the popularization of mobile internet, social networks have become a part of people's daily life. Devices such as smart phones and cameras provide a way to conveniently acquire pictures, so that a large number of pictures are shared in a social network and are used for sharing the daily life of people. After 2014, pictures have surpassed plain text, being the most abundant form of sharing in social networks, with more than 1 and 3 billion pictures uploaded on Instagram and Facebook, respectively, each day.
The shared pictures contain a large amount of information, and the privacy of the user is likely to be revealed. Unlike text content that needs to be thought and entered, the user only needs to press the shutter to obtain a picture. Thus, while social network providers allow users to set the visible range of content to protect user privacy, many users are unaware of privacy risks in pictures. In one study, researchers described the content of pictures to users, investigated the users 'expectations for privacy settings of pictures, and showed that there was a difference between the user's expectations and the actual privacy setting status of the pictures. Therefore, an effective method is needed to infer the privacy risk in the picture shared by the user, identify the picture which may reveal the privacy and give an early warning.
Some existing methods using machine learning are mainly classified into two categories. The first category uses the whole picture for classification and gives a suggestion of privacy settings. Because of the difficult-to-interpret problem of machine learning, the suggestions given by this method are difficult to understand by the user, and the user has difficulty finding out which regions in the picture are more likely to reveal privacy; another class of object detection-based methods is easy to understand, but is limited to predefined object classes and cannot deal with privacy disclosure caused by objects outside the list.
Disclosure of Invention
Aiming at the problems, the invention provides a method and a system for detecting and early warning privacy risks of pictures in a social network, aiming at predicting privacy risks of pictures uploaded by users in the social network, and early warning pictures with high privacy risks to remind the users to upload carefully.
The invention provides a method for detecting and early warning privacy risks of social network pictures, which comprises the following steps: the method comprises the following steps: extracting key elements in the picture by using a target detection frame and obtaining information of the key elements; step two: collecting a data set of whether the picture is private or not, carrying out operation of the step one on each picture in the data set, then carrying out statistics on the whole data set to obtain the association degrees of various key elements with the private and public pictures, and constructing a knowledge graph according to the association degrees; step three: extracting the characteristics of the whole picture and the key elements of the picture by using a neural network, and constructing the neural network by using the knowledge graph in the step two, so as to fuse the characteristics of the whole picture and the key elements of the picture and obtain the final expression of the picture; and step four: and predicting the privacy risk of the picture by utilizing a neural network based on the final expression of the picture in the third step.
The invention also provides a system for detecting and early warning the privacy risk of the social network pictures, which comprises the following steps: the picture element extraction module: extracting key elements in the picture by using a target detection frame and obtaining information of the key elements; a knowledge graph construction module: collecting a data set of whether the picture is private or not, carrying out operation in a picture element extraction module on each picture in the data set, then carrying out statistics on the whole data set to obtain the association degrees of various key elements with the private and public pictures, and constructing a knowledge graph according to the association degrees; the picture information fusion module: extracting the characteristics of the whole picture and the key elements of the picture by using a neural network, and constructing the neural network by using the knowledge graph in the knowledge graph construction module, so as to fuse the characteristics of the whole picture and the key elements of the picture and obtain the final expression of the picture; and a privacy detection early warning module: and based on the final expression of the picture in the picture information fusion module, predicting the privacy risk of the picture by using a neural network and carrying out early warning.
According to the method, key elements in the picture and the whole picture are comprehensively considered, the constructed knowledge graph is utilized to fuse information, so that the privacy risk of the picture is obtained, the picture with high privacy risk is early warned, and a user is reminded to upload cautiously.
The invention is described in detail below with reference to the drawings and specific examples, but the invention is not limited thereto.
Drawings
Fig. 1 is a schematic flow chart of a method for social network picture privacy risk detection and early warning according to the present invention.
Detailed Description
The invention will be described in detail with reference to the following drawings, which are provided for illustration purposes and the like:
the method comprises the following steps of firstly, extracting key elements based on target detection.
In order to obtain the position and category information of the key elements in the picture, the extraction is performed by using a mature target detection framework, such as fast-RCNN, MASK-RCNN, and the like, and of course, other target detection frameworks exist, which is not limited by the invention. The key elements thus obtained are all the categories that can be detected by the target detection framework.
And secondly, constructing a knowledge graph based on key elements.
The method comprises the steps that a data set of the social network pictures is collected on a social network platform needing to protect the privacy of the pictures, a label for judging whether the pictures can reveal the privacy or not is obtained, multiple people mark the pictures, opinions of the multiple people are taken as final marks of the pictures, and the pictures of the data set are divided into two categories, namely privacy pictures and public pictures according to the marks. If the picture data set on the social network platform is difficult to obtain, other marked picture privacy data sets disclosed in the previous research can be used as an alternative.
And after the data set is obtained, performing the operation in the first step on each picture, extracting key elements on each picture and acquiring the information of the key elements. After the key elements and the information thereof are extracted, statistics is carried out on the whole picture data set, so as to obtain the frequency of each key element in the privacy picture and the public picture respectively, and then the number of the pictures in each category is normalized respectively and is used as the association strength between each key element and the privacy and public two categories.
Constructing a knowledge graph, wherein the graph comprises two types of nodes: the first type of two nodes represent privacy and public categories respectively, so the first type of two nodes are category nodes; the second type of nodes is the key element nodes, since the number of the nodes is equal to the type of the key element. And establishing a connecting edge between the category node and the key element, wherein the weight of the edge is the association strength between the key element and the privacy and disclosure categories.
And thirdly, fusing the global information and the key element information based on the graph neural network.
For a picture, the overall characteristics of the picture are extracted by utilizing a neural network and are used as the initialization of the characteristics of the first two nodes, namely privacy and disclosure. If the picture contains certain key elements, cutting the characteristics of the corresponding area of the key elements as the initialization of the corresponding second type nodes, namely the key element nodes. The remaining uninitialized node signatures are set to zero.
After all the nodes are initialized, the global information contained in the category nodes and the local information contained in the key element nodes are fused by using a graph neural network. And finally, giving different weights to the fused features for the features of different key element nodes, and splicing the fused features and the fused global features into a final expression of the picture.
And fourthly, deducing the privacy risks of the pictures based on the fusion information.
And predicting the privacy risk of the picture by utilizing a neural network based on the final expression of the picture obtained in the third step and carrying out early warning.
The model is trained based on the collected data set, resulting in a model that can be applied to reality. After the user uploads the picture, the model is used for prediction, when the privacy risk predicted by the model is high, the detected key elements are marked, the user is reminded of cautiously sharing, the key elements which possibly reveal the privacy are noticed, and therefore the privacy risk of social network picture sharing is reduced.
The present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof, and it should be understood that various changes and modifications can be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (14)
1. A method for social network picture privacy risk detection and early warning is characterized by comprising the following steps:
the method comprises the following steps: extracting key elements in the picture by using a target detection frame and obtaining information of the key elements;
step two: collecting a data set of whether the picture is private or not, carrying out operation of the step one on each picture in the data set, then carrying out statistics on the whole data set to obtain the association degrees of various key elements with the private and public pictures, and constructing a knowledge graph according to the association degrees;
step three: extracting the characteristics of the whole picture and the key elements of the picture by using a neural network, and constructing the neural network by using the knowledge graph in the step two, so as to fuse the characteristics of the whole picture and the key elements of the picture and obtain the final expression of the picture; and
step four: and predicting the privacy risk of the picture by utilizing a neural network based on the final expression of the picture in the third step.
2. The method for social networking picture privacy risk detection and pre-warning as claimed in claim 1, wherein the target detection framework in the first step can adopt fast-RCNN or MASK-RCNN.
3. The method for social network picture privacy risk detection and warning as claimed in claim 1, wherein the data set in the second step can be a picture data set collected on a social platform protecting picture privacy or an existing picture data set marked with privacy or not.
4. The method for social network picture privacy risk detection and early warning as claimed in claim 3, wherein the picture data set collected on the social platform for protecting picture privacy is divided into two types, namely a private picture and a public picture, according to a judgment basis that most of labels of whether a platform user thinks that the picture reveals privacy are marked.
5. The method for social network picture privacy risk detection and early warning as claimed in claim 1, wherein the association degree between each type of key element and the privacy and public pictures in the second step is characterized by the frequency of occurrence of each type of key element in the privacy and public pictures respectively.
6. The method for social network picture privacy risk detection and warning as claimed in claim 1, wherein the knowledge graph in the second step includes two types of nodes:
the category nodes represent two categories of privacy and disclosure respectively; and
key element nodes, the number of which is equal to the type of the key elements;
and establishing a connecting edge between the two types of nodes, wherein the weight of the edge is the association degree of the key element and the privacy/disclosure category.
7. The method for social network picture privacy risk detection and early warning as recited in claim 6, wherein the third step specifically comprises:
for a picture, extracting the overall characteristics of the picture by utilizing a neural network, taking the overall characteristics as the initialization of the characteristics of two nodes of a first class, and cutting the characteristics of a corresponding area for certain key elements contained in the picture as the initialization of the corresponding nodes of a second class;
after all nodes are initialized, adopting a graph neural network to fuse global information contained in the class nodes and local information contained in the key element nodes;
and giving different weights to the fused features for the features of different key element nodes, and splicing the fused features and the fused global features into a final expression of the picture.
8. A system for social network picture privacy risk detection and early warning, comprising:
the picture element extraction module: extracting key elements in the picture by using a target detection frame and obtaining information of the key elements;
a knowledge graph construction module: collecting a data set of whether the picture is private or not, carrying out operation in a picture element extraction module on each picture in the data set, then carrying out statistics on the whole data set to obtain the association degrees of various key elements with the private and public pictures, and constructing a knowledge graph according to the association degrees;
the picture information fusion module: extracting the characteristics of the whole picture and the key elements of the picture by using a neural network, and constructing the neural network by using the knowledge graph in the knowledge graph construction module, so as to fuse the characteristics of the whole picture and the key elements of the picture and obtain the final expression of the picture; and
privacy detects early warning module: and based on the final expression of the picture in the picture information fusion module, predicting the privacy risk of the picture by using a neural network and carrying out early warning.
9. The system for social networking picture privacy risk detection and warning as claimed in claim 8, wherein the object detection framework in the picture element extraction module can employ fast-RCNN or MASK-RCNN.
10. The system for social network picture privacy risk detection and warning as claimed in claim 8, wherein the data set in the picture information fusion module can adopt a picture data set collected on a social platform protecting picture privacy or an existing picture data set labeled with or without privacy.
11. The system for social network picture privacy risk detection and warning as claimed in claim 10, wherein the picture data set collected on the social platform for protecting picture privacy is divided into two categories, namely a private picture and a public picture, according to a majority of labels that a platform user considers that the picture reveals privacy.
12. The system for social network picture privacy risk detection and warning of claim 8, wherein the association degree of each type of key element in the knowledge graph building module with the privacy and public pictures is characterized by the frequency of occurrence of each type of key element in the privacy and public pictures.
13. The system for social network picture privacy risk detection and warning of claim 8, wherein the knowledge-graph in the knowledge-graph construction module includes two types of nodes:
the category nodes represent two categories of privacy and disclosure respectively; and
key element nodes, the number of which is equal to the type of the key elements;
and establishing a connecting edge between the two types of nodes, wherein the weight of the edge is the association degree of the key element and the privacy/disclosure category.
14. The method for social network picture privacy risk detection and warning as recited in claim 13, wherein the specific operations in the picture information fusion module include:
for a picture, extracting the overall characteristics of the picture by utilizing a neural network, taking the overall characteristics as the initialization of the characteristics of two nodes of a first class, and cutting the characteristics of a corresponding area for certain key elements contained in the picture as the initialization of the corresponding nodes of a second class;
after all nodes are initialized, adopting a graph neural network to fuse global information contained in the class nodes and local information contained in the key element nodes;
and giving different weights to the fused features for the features of different key element nodes, and splicing the fused features and the fused global features into a final expression of the picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010323133.8A CN111639359B (en) | 2020-04-22 | 2020-04-22 | Method and system for detecting and early warning privacy risk of social network picture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010323133.8A CN111639359B (en) | 2020-04-22 | 2020-04-22 | Method and system for detecting and early warning privacy risk of social network picture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111639359A true CN111639359A (en) | 2020-09-08 |
| CN111639359B CN111639359B (en) | 2023-09-12 |
Family
ID=72328846
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010323133.8A Active CN111639359B (en) | 2020-04-22 | 2020-04-22 | Method and system for detecting and early warning privacy risk of social network picture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111639359B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111967565A (en) * | 2020-10-23 | 2020-11-20 | 支付宝(杭州)信息技术有限公司 | Neural network system, method and device for risk assessment |
| CN114513337A (en) * | 2022-01-20 | 2022-05-17 | 电子科技大学 | Privacy protection link prediction method and system based on mail data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105260676A (en) * | 2015-10-16 | 2016-01-20 | 合肥工业大学 | Image privacy decision recommendation system and method under social network on the basis of privacy rule and perceptual Hash |
| US20160180162A1 (en) * | 2014-12-22 | 2016-06-23 | Yahoo! Inc. | Generating preference indices for image content |
| CN106682694A (en) * | 2016-12-27 | 2017-05-17 | 复旦大学 | Sensitive image identification method based on depth learning |
| CN107704877A (en) * | 2017-10-09 | 2018-02-16 | 哈尔滨工业大学深圳研究生院 | A kind of image privacy cognitive method based on deep learning |
| CN110675951A (en) * | 2019-08-26 | 2020-01-10 | 北京百度网讯科技有限公司 | Intelligent disease diagnosis method and device, computer equipment and readable medium |
-
2020
- 2020-04-22 CN CN202010323133.8A patent/CN111639359B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160180162A1 (en) * | 2014-12-22 | 2016-06-23 | Yahoo! Inc. | Generating preference indices for image content |
| US20170193336A1 (en) * | 2014-12-22 | 2017-07-06 | Yahoo! Inc. | Generating preference indices for image content |
| CN105260676A (en) * | 2015-10-16 | 2016-01-20 | 合肥工业大学 | Image privacy decision recommendation system and method under social network on the basis of privacy rule and perceptual Hash |
| CN106682694A (en) * | 2016-12-27 | 2017-05-17 | 复旦大学 | Sensitive image identification method based on depth learning |
| CN107704877A (en) * | 2017-10-09 | 2018-02-16 | 哈尔滨工业大学深圳研究生院 | A kind of image privacy cognitive method based on deep learning |
| CN110675951A (en) * | 2019-08-26 | 2020-01-10 | 北京百度网讯科技有限公司 | Intelligent disease diagnosis method and device, computer equipment and readable medium |
Non-Patent Citations (1)
| Title |
|---|
| 朱光;丰米宁;陈叶;杨嘉韵;: "大数据环境下社交网络隐私风险的模糊评估研究", 情报科学, vol. 34, no. 09, pages 94 - 98 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111967565A (en) * | 2020-10-23 | 2020-11-20 | 支付宝(杭州)信息技术有限公司 | Neural network system, method and device for risk assessment |
| CN111967565B (en) * | 2020-10-23 | 2021-04-27 | 支付宝(杭州)信息技术有限公司 | Neural network system, method and device for risk assessment |
| CN114513337A (en) * | 2022-01-20 | 2022-05-17 | 电子科技大学 | Privacy protection link prediction method and system based on mail data |
| CN114513337B (en) * | 2022-01-20 | 2023-04-07 | 电子科技大学 | Privacy protection link prediction method and system based on mail data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111639359B (en) | 2023-09-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10198667B2 (en) | System and method of detecting offensive content sent or received on a portable electronic device | |
| US10572771B2 (en) | Systems and methods for image object recognition based on location information and object categories | |
| US10698945B2 (en) | Systems and methods to predict hashtags for content items | |
| EP2741473B1 (en) | Human-machine interaction data processing method and apparatus | |
| US20160350675A1 (en) | Systems and methods to identify objectionable content | |
| CN105404860A (en) | Method and device for managing information of lost person | |
| US10360466B2 (en) | Systems and methods for image description generation | |
| CN102831176A (en) | Method and server for recommending friends | |
| CN111639359A (en) | Method and system for detecting and early warning privacy risks of social network pictures | |
| CN104040963A (en) | System and methods for spam detection using frequency spectra of character strings | |
| US10638298B1 (en) | Public event detection platform | |
| US20200210684A1 (en) | System and method of biometric identification and storing and retrieving suspect information | |
| WO2021082505A1 (en) | Picture processing method, apparatus and device, storage medium, and computer program | |
| US20180060354A1 (en) | Systems and methods for processing media content that depict objects | |
| US10360255B2 (en) | Systems and methods to determine location of media items | |
| CN114595382A (en) | Privacy decision recommendation system based on image data and deep learning | |
| Boddhu et al. | Context-aware event detection smartphone application for first responders | |
| KR102320005B1 (en) | Image analysis based abnormal object detection system and method | |
| US10275471B2 (en) | Systems and methods for processing media content that depict objects | |
| Brou et al. | Effects of the internet of things (IoT): A systematic review of the benefits and risks | |
| CN114419354A (en) | Picture privacy risk prediction method and system based on key area dynamic analysis, and model training method and system | |
| JP2019117553A (en) | Information presentation device, method and program | |
| CN107682624A (en) | system maintenance method and related product | |
| US10750346B1 (en) | Emergency incident detection platform | |
| CN114338346A (en) | Alarm message processing method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |