WO2019071754A1 - 一种基于深度学习的图像隐私感知方法 - Google Patents

一种基于深度学习的图像隐私感知方法 Download PDF

Info

Publication number
WO2019071754A1
WO2019071754A1 PCT/CN2017/113068 CN2017113068W WO2019071754A1 WO 2019071754 A1 WO2019071754 A1 WO 2019071754A1 CN 2017113068 W CN2017113068 W CN 2017113068W WO 2019071754 A1 WO2019071754 A1 WO 2019071754A1
Authority
WO
WIPO (PCT)
Prior art keywords
privacy
image
feature
bilinear
deep
Prior art date
Application number
PCT/CN2017/113068
Other languages
English (en)
French (fr)
Inventor
王鸿鹏
张阳
尤磊
何华门
黄兴森
Original Assignee
哈尔滨工业大学深圳研究生院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 哈尔滨工业大学深圳研究生院 filed Critical 哈尔滨工业大学深圳研究生院
Priority to US16/099,836 priority Critical patent/US11256952B2/en
Publication of WO2019071754A1 publication Critical patent/WO2019071754A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2148—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the process organisation or structure, e.g. boosting cascade
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Definitions

  • the invention relates to artificial intelligence, in particular to an image privacy sensing method based on deep learning.
  • the existing image privacy sensing method only completes the image-level privacy perception, that is, distinguishes whether the entire image is a privacy image, and does not perceive the image privacy area.
  • the present invention provides an image privacy sensing method based on deep learning.
  • the invention provides an image privacy sensing method based on deep learning, which comprises the following steps:
  • step S1 comprises: pre-training the deep convolutional neural network model on the large-scale image data set, and then constructing the privacy classification data set, and pre-training the deep convolutional neural network model in the privacy classification Fine-tune on the data set.
  • step S2 includes: adding a bilinear operation layer after the last layer of the convolutional layer of the deep convolutional neural network, enhancing the feature expression capability of the deep convolutional neural network model, and simultaneously changing the fully connected layer For the pooling layer.
  • step S3 includes: obtaining a weighted high-level feature map as a attention distribution map according to the correspondence between each node weight of the pooling layer and the feature map subjected to bilinear operation, and positioning by scale transformation The privacy zone in the original image.
  • the bilinear operation layer mainly calculates the result of two or two points multiplication between the feature maps after convolution
  • ⁇ represents the dot multiplication of the matrix
  • the representation is rounded up
  • n represents the number of original feature maps
  • i represents the subscript of the bilinear feature map.
  • the dimensional reduction operation is performed on the bilinear feature map.
  • the Tensor Sketch algorithm is used to perform dimensionality reduction on the bilinear feature map.
  • the bilinear feature map is c w*h matrices
  • the input of the Tensor Sketch algorithm is a vector
  • each position in the bilinear feature graph is sequentially calculated using the Tensor Sketch algorithm, that is, respectively
  • the w*h c-dimensional vectors are operated and remapped into the space of the w*h*d dimension; firstly, the parameter set h k ⁇ 1,...,d ⁇ c ,s k for hashing is randomly generated.
  • the cumulatively calculated Count Sketch vector is obtained by the convolution theorem.
  • the convolution of the time domain or the spatial domain is equal to the product in the corresponding frequency domain; therefore, the two Fast Sketch vectors are converted to the frequency domain using the fast Fourier transform.
  • the product in the frequency domain is then converted back to the spatial domain by inverse Fourier transform, and the convolution of the Count Sketch vector is calculated.
  • the fully connected layer is changed to an average pooling layer, and the average pooling layer performs a pooling operation on the entire feature map, and averages the elements of each feature map to obtain a d-dimensional vector.
  • the average pooling layer node and the feature map have a corresponding relationship, and the attention distribution map is obtained by weighted summation of the feature map;
  • the resulting attention distribution map is A, and its calculation formula is as follows:
  • n is the category label of the input image after being classified. Representing the connection weight of the corresponding category n of the kth node of the pooling layer;
  • the specific method is to change the attention distribution map obtained by the above steps into a scale change, and set a threshold value to complete image binarization, and to solve the minimum image after binarization.
  • the add-in matrix is the result of local perception of the privacy image.
  • the invention has the beneficial effects that the end-to-end training and testing are completed based on the deep neural network, and the privacy image and the non-private image can be accurately distinguished and the privacy region in the image is located, thereby facilitating selective protection of the privacy information in the image. , provides a good prerequisite for the privacy protection process. From the aspect of method advancement, the invention effectively overcomes the problems of low accuracy, poor generalization ability, and relying on additional information of users in the traditional privacy sensing method, and the privacy perception is correct without increasing the training neural network model. The perception of the image as a whole extends to the perception of the image privacy zone.
  • FIG. 1 is a flow chart of a depth learning based image privacy sensing method of the present invention.
  • FIG. 2 is a structural diagram of a deep convolutional neural network based on a deep learning-based image privacy sensing method of the present invention.
  • an image privacy sensing method based on deep learning the main steps include:
  • Neural network pre-training training deep convolutional neural networks on large-scale image data sets (eg, ImageNet);
  • Neural network improvement and training Improve the pre-trained neural network and fine-tune the privacy image dataset
  • Image overall privacy perception automatically determine whether the input image is a privacy image
  • Image privacy zone awareness Automatic detection of privacy zones in images.
  • the pre-trained convolutional neural network is improved, and after the last layer of the convolutional layer, a bilinear operation layer is added to enhance the feature expression capability of the model, and the fully connected layer is modified.
  • a bilinear operation layer is added to enhance the feature expression capability of the model, and the fully connected layer is modified.
  • the pooling layer lay the foundation for privacy zone perception.
  • Image privacy zone awareness does not require retraining the network. According to the correspondence between the weights of each node of the classification network pooling layer and the feature maps of the bilinear operation, the weighted high-level feature map is obtained, and the attention distribution map is obtained through the scale change, and the attention concentration area is used as the privacy area. .
  • Keywords mainly refer to categories such as ID photos, family/group photos, and file snapshots.
  • a correlation model that can calculate the similarity between words (for example, the word2vec and GloVe models after a large amount of corpus training) is used to help generate similar words of input keywords, thereby increasing privacy keywords and facilitating searching. More images. Then, a small number of privacy-independent images obtained by the search were manually filtered, and 4384 private images were collected. For non-private images, 200 common objects in the ImageNet dataset are selected, 4800 images are randomly selected, and the training set and test set are divided according to the ratio of 1:1 to facilitate the training and testing of the subsequent neural network.
  • Neural Network Pre-Training This step trains deep convolutional neural networks on the ImageNet large-scale image dataset.
  • the ImageNet dataset contains approximately 1.2 million images and covers 1000 common objects.
  • the reason for pre-training is that the privacy data set is small, while the deep convolutional neural network has many parameters, and direct training is difficult to converge. If the pre-training is carried out on a large-scale data set to obtain a good initial weight and obtain a certain feature expression ability, it can quickly converge on a small data set and obtain a better classification effect.
  • the pre-trained neural network uses the currently effective VGG16 convolutional neural network.
  • the VGG16 consists of 16 layers of convolutional layers and 2 layers of fully connected layers, which can achieve good results in general classification tasks.
  • Neural network improvement and training First, the pre-trained model is improved and trained on the privacy data set.
  • the main improvements are as follows:
  • a bilinear operation layer is added after the last layer of convolutional layer to enhance the feature expression ability of the model.
  • ⁇ represents the dot multiplication of the matrix
  • the representation is rounded up
  • n represents the number of original feature maps
  • i represents the subscript of the bilinear feature map.
  • the present invention uses the Tensor Sketch algorithm (TS algorithm for short) to achieve data dimensionality reduction, which is a vector outgrowth estimation method based on Count Sketch.
  • Count Sketch is a method of data hashing. It was first used in the mining of frequent itemsets of data streams. Later, it was proved by Pham et al. that the outer product of two vectors can be estimated by calculating the convolution of Count Sketch (ie, between vectors). Each element is multiplied by two).
  • the present invention sequentially calculates each position in the feature map when using the TS algorithm, that is, respectively for w*h c
  • the dimension vector is computed and remapped into the space of the w*h*d dimension.
  • randomly generate a parameter set h k ⁇ 1,...,d ⁇ c , s k ⁇ 1,-1 ⁇ c (k 1, 2) for performing the hash operation, where h k is used to store the input vector
  • the remapped index, s k implements a random negation of the values of the elements of the input vector.
  • the remapping Count Sketch vector can be obtained by the cumulative calculation. It can be seen from the convolution theorem that the convolution of the time domain or the spatial domain is equal to the product in the corresponding frequency domain. So you can use the Fast Fourier Transform (FFT) to convert the two Count Sketch vectors into the frequency domain, find the product in the frequency domain, and then convert back to the spatial domain by inverse Fourier transform to calculate the convolution of the Count Sketch vector.
  • FFT Fast Fourier Transform
  • the present invention also changes the fully connected layer behind the last layer of the convolution layer in the original network structure to the Average Pooling layer, which pools the entire feature map. Operation, averaging the elements of each feature map, and finally obtaining a vector of d dimensions.
  • the use of the pooling layer instead of the fully connected layer is because the pooling layer has no parameters to learn, which greatly reduces the model parameters, speeds up the convergence, and avoids over-fitting to some extent.
  • the correspondence between the feature map and the pooled feature vector is guaranteed, which creates conditions for the subsequent extraction of the attention distribution map.
  • Image overall privacy perception This step is used to automatically identify whether the input image is a privacy image, input the test image into the trained privacy-aware network, and determine whether it is a privacy image according to the subordinate probability of each category output by the network.
  • Image privacy zone awareness This step is used to automatically detect the privacy zone in the image.
  • the attention distribution map is extracted mainly through the deep convolution feature of the network, and the attention concentration area is located to complete the perception of the privacy area.
  • the attention distribution map can be obtained by weighted summation of the feature maps.
  • n is the category label of the input image after being classified. Indicates the connection weight corresponding to the category n of the kth node of the pooling layer.
  • the present invention performs local positioning of the privacy image according to the above result.
  • the specific method is to change the attention distribution map obtained by the above steps into a scale of the original image.
  • the threshold is used to complete the image binarization, and the minimum circumscribed matrix of the binarized image is solved as the result of local perception of the privacy image.
  • the invention is widely used, for example:
  • Solution 1 In social networks, photo sharing has become an increasingly popular form of communication. However, users have certain security risks in photo sharing. For example, many people, especially young people, directly expose photos that may reveal personal privacy without adequate consideration of their own security. Sharing social networks, some lawless elements may use this information to engage in illegal activities, which undoubtedly poses a certain security threat to themselves or their relatives and friends. In this regard, if the privacy awareness mechanism in the present invention is used, the privacy of the uploader's photos may be promptly reminded to play a role in preventing micro-duration. In addition, in some cases, the user wishes to mask or obfuscate the area of the public photo that involves privacy. The processing of the privacy area requires a lot of manpower and time.
  • the method for sensing the image privacy sensitive area provided by the present invention can better solve the above problem. The method can automatically locate the privacy area in the image, facilitating subsequent processing, and avoiding Manual operation.
  • Solution 2 Currently, cloud storage applications are more and more widely used. The cloud platform brings together a large number of users' personal information, and a large part of them are image data. However, most cloud platforms are untrusted systems, and it is not uncommon for cloud platforms to leak personal data. In order to protect personal privacy from being leaked, some companies use encryption or data perturbation to protect privacy, but processing a large amount of image data requires a lot of computing resources. At this time, if the image data is analyzed by using the method involved in the invention, the privacy image is first distinguished or the privacy sensitive area is located, and the targeted protection is performed, so that the calculation overhead can be greatly reduced while ensuring information security.
  • One aspect of the present invention improves some of the shortcomings of the existing image privacy sensing method.
  • the privacy awareness problem is extended to the perception of the image privacy area to meet different needs.
  • the present invention only trains image content features and categories, and is not restricted by user-set image tags and access policies, and can play a role in various application scenarios.
  • the invention uses a deep convolution network, which has stronger feature expression than the traditional feature extraction method, and increases the classification accuracy and generalization ability of the model.
  • the invention proposes an automatic privacy awareness mechanism, which can sense the privacy of images and images locally, and satisfies the diversified needs of image privacy protection. It can selectively protect the privacy image under the premise of ensuring the privacy of the user, and greatly saves the computational overhead of privacy protection.
  • the privacy-aware data set constructed in the present invention includes images obtained by searching according to a large number of private corpus, so that the model can perceive various common privacy categories including document photos, file snapshots, and the like, and has strong versatility.
  • the invention adopts an end-to-end mode in the training and testing phases (the input end is the original picture, the output end is the perceptual result, the process does not need human intervention), and the same model is used to simultaneously complete the privacy image and the image privacy area.
  • Perceptual the model is easy to use and easy to promote Among the practical application scenarios.
  • the optimization strategy of bilinear operation is introduced.
  • the feature expression ability is further improved, which is beneficial to improve the image perception accuracy rate and has a great benefit to the location of the privacy zone.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Bioethics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

本发明提供了一种基于深度学习的图像隐私感知方法，包括以下步骤：S1、构建带类别标注的隐私分类数据集，使用迁移学习的方法训练隐私感知网络；S2、使用面向隐私感知的深度卷积神经网络完成隐私图像的识别；S3、根据神经网络深层卷积特征提取注意力分布图，并定位注意力集中区域完成对图像隐私区域的感知。本发明的有益效果是：基于深度神经网络完成了端到端的训练和测试，可以准确地区分隐私图像并定位图像中的隐私区域，方便对图像中的隐私信息进行选择性保护。

Description

一种基于深度学习的图像隐私感知方法 技术领域
本发明涉及人工智能，尤其涉及一种基于深度学习的图像隐私感知方法。
背景技术
隐私感知是隐私保护过程中重要的先决条件，而图像是目前社交网络中最主要的信息类型之一，对海量的图像数据进行隐私感知变得尤为关键。鉴于隐私的概念具有很强的主观性，目前存在的图像隐私感知方法通常先界定通用意义上的图像隐私(如个人证件、家庭合照、机密文件快照等)或者利用用户个体在社交网络上的标注。现有的方法主要有以下缺点：
一、在隐私图像特征提取方面，现有的大部分方法是使用传统的图像特征提取方法，如SIFT特征、RGB特征、颜色直方图等等，在特征表达能力上有一定的局限性，无法表示深层的语义级别的特征，分类模型泛化能力差。
二、目前存在的一些感知图像隐私的方法需要使用额外的信息才能达到可以接受的感知效果，比如用户对图像主观标记的图像描述标签或者用户自己设定的访问控制策略，然而这些信息在大部分情况下很难获取，对应用场景的要求也较为苛刻，模型的通用性较差。
三、现有的图像隐私感知方法只完成了图像级别的隐私感知，即区分整张图像是否为隐私图像，并没有对图像隐私区域进行感知。然而在实际应用中，有时需要对图像的隐私区域进行遮挡或模糊化处理，以达到隐私保护的目的。
发明内容
为了解决现有技术中的问题，本发明提供了一种基于深度学习的图像隐私感知方法。
本发明提供了一种基于深度学习的图像隐私感知方法，包括以下步骤：
S1、构建带类别标注的隐私分类数据集，使用迁移学习的方法训练隐私感知网络；
S2、使用面向隐私感知的深度卷积神经网络完成隐私图像的识别；
S3、根据神经网络深层卷积特征提取注意力分布图，并定位注意力集中区 域完成对图像隐私区域的感知。
作为本发明的进一步改进，步骤S1包括：先在大规模图像数据集上对深度卷积神经网络模型进行预训练，然后构建隐私分类数据集，将预训练的深度卷积神经网络模型在隐私分类数据集上进行微调。
作为本发明的进一步改进，步骤S2包括：在深度卷积神经网络的最后一层卷积层后加入双线性运算层，增强深度卷积神经网络模型的特征表达能力，同时将全连接层改为池化层。
作为本发明的进一步改进，步骤S3包括：根据池化层的各节点权重和经过双线性运算的特征图的对应关系，得到加权的高层特征图作为注意力分布图，并通过尺度变换，定位原图中的隐私区域。
作为本发明的进一步改进，双线性运算层主要计算卷积后的特征图之间两两点乘的结果，
设原特征图集合M＝{m1,m2,…,mn}，
输出的双线性特征图集合为M’＝{m’1,m’2,…,m’n×n},
则转换的公式如下：
Figure PCTCN2017113068-appb-000001
其中“ο”代表矩阵的点乘，
Figure PCTCN2017113068-appb-000002
代表向上取整，n表示原特征图的个数，i表示双线性特征图的下标。
作为本发明的进一步改进，对双线性特征图进行降维操作。
作为本发明的进一步改进，采用Tensor Sketch算法对双线性特征图进行降维操作。
作为本发明的进一步改进，双线性特征图为c个w*h的矩阵，而Tensor Sketch算法的输入为向量，使用Tensor Sketch算法时对双线性特征图中每个位置依次计算，即分别对w*h个c维向量进行运算，重映射到w*h*d维的空间中；首先随机生成用于进行哈希操作的参数集合hk∈{1,…,d}c，sk∈{1,-1}c(k＝1,2)，其中hk用于存储输入向量重映射后的索引，sk实现了输入向量各元素数值的随机取反；根据上述参数集合，通过累加计算得到重映射后的Count Sketch向量；由卷积定理可知，时域或空间域的卷积等于对应频域内的乘积；所以使用快速傅立叶变换将两个Count Sketch向量转 换到频域，求其在频域的乘积，然后通过傅里叶反变换转换回空间域，计算得到Count Sketch向量的卷积。
作为本发明的进一步改进，将全连接层改为平均池化层，该平均池化层对整张特征图进行池化操作，对每张特征图的元素求平均值，最终得到d维的向量。
作为本发明的进一步改进，平均池化层节点和特征图有对应关系，通过特征图的加权求和得到注意力分布图；
设通过降维后的双线性特征图序列P＝{p1,p2,…,pd}，
最后生成的注意力分布图为A，则其计算公式如下：
Figure PCTCN2017113068-appb-000003
其中n为输入图像经过分类后从属的类别标签,
Figure PCTCN2017113068-appb-000004
表示对池化层第k个节点对应类别n的连接权重；
根据上述结果进行隐私图像局部的定位，具体方法是将由上述步骤得到的注意力分布图进行尺度变化，转换为原图大小，并设定阈值完成图像二值化，求解二值化后图像的最小外接矩阵作为隐私图像局部感知的结果。
本发明的有益效果是：基于深度神经网络完成了端到端的训练和测试，可以准确地区分隐私图像和非隐私图像并定位图像中的隐私区域，，方便对图像中的隐私信息进行选择性保护，为隐私保护过程提供了良好的先决条件。从方法先进性层面，该发明有效地克服了传统隐私感知方法的准确率低、泛化能力差、依赖用户额外信息等问题，并在不增加训练神经网络模型的情况下，将隐私感知从对图像整体的感知延伸到了对图像隐私区域的感知。
附图说明
图1是本发明一种基于深度学习的图像隐私感知方法的流程图。
图2是本发明一种基于深度学习的图像隐私感知方法的深度卷积神经网络结构图。
具体实施方式
下面结合附图说明及具体实施方式对本发明作进一步说明。
如图1至图2所示，一种基于深度学习的图像隐私感知方法，主要步骤包括：
构建隐私数据集：收集相关图像并按隐私与非隐私进行标注；
神经网络预训练：将深度卷积神经网络在大规模图像数据集(例如，ImageNet)上进行训练；
神经网络改进与训练：将预训练好的神经网络进行改进，并在隐私图像数据集上微调；
图像整体隐私感知：自动判断输入图像是否为隐私图像；
图像隐私区域感知：自动检测图像中的隐私区域。
在所述神经网络改进与训练步骤中对预训练好的卷积神经网络进行改进，在最后一层卷积层后加入双线性运算层，增强模型的特征表达能力，同时将全连接层改为池化层，为隐私区域感知打下基础。
图像隐私区域感知无需重新训练网络。本发明根据分类网络池化层各节点权重和经过双线性运算的特征图的对应关系，得到加权的高层特征图，并通过尺度变化，得到注意力分布图，定位注意力集中区域作为隐私区域。
各个步骤具体实施方式如下：
构建隐私数据集：为提高数据集构建效率，使用关键词搜索的方式，取百度和谷歌图像搜索的前n张备选图像。关键词主要涉及证件照片、家庭/团体合照、文件快照等类别。在关键词获取过程中，使用了可以计算词间相似度的相关模型(例如，经大量语料训练后的word2vec和GloVe模型)，帮助产生输入关键词的相似词汇，从而增加隐私关键词，便于搜索更多的图像。接着人工筛选搜索得到的少数与隐私无关的图像，收集了4384张隐私图像。对于非隐私图像，选取ImageNet数据集中的200类常见物体，随机抽取得到4800张图像，最后按照1：1的比率划分训练集和测试集，方便后续神经网络的训练和测试。
神经网络预训练：本步骤将深度卷积神经网络在ImageNet大规模图像数据集上进行训练。ImageNet数据集包含了约120万张图像，涉及了1000类常见的物体。预训练的原因是隐私数据集较小，而深度卷积神经网络参数较多，直接训练很难收敛。而如果先在大规模数据集上进行预训练，使神经网络得到较好的初始权重，同时获得一定的特征表达能力,就可以在小数据集上快速收敛，并得到较好的分类效果。预训练的神经网络使用目前效果较好的VGG16卷积神经网络，VGG16包含16层卷积层和2层全连接层，在通用分类任务上能达到很好的效果。
神经网络改进与训练：首先将预训练好的模型进行改进，并在隐私数据集上进行训练。主要的改进内容如下：
对预训练好的VGG16网络，在最后一层卷积层后加入双线性运算层，增强模型的特征表达能力。双线性运算层主要计算卷积后的特征图之间两两点乘的结果，设原特征图集合M＝{m1,m2,…,mn},输出的双线性特征图集合为M’＝{m’1,m’2,…,m’n×n},则转换的公式如下：
Figure PCTCN2017113068-appb-000005
其中“ο”代表矩阵的点乘，
Figure PCTCN2017113068-appb-000006
代表向上取整，n表示原特征图的个数，i表示双线性特征图的下标。
然而在系统实现过程中，直接计算最后一层卷积的双线性特征图会出现维度灾难的问题。如在本框架中最后一层有512个特征图，根据公式(1)求得的双线性特征图有512×512个，从而带来了后续大量的计算开销，因此需要对双线性特征图实现降维操作。对此，本发明使用了Tensor Sketch算法(简称TS算法)来实现数据的降维，该算法是一种基于Count Sketch的向量外积估计方法。Count Sketch是一种数据哈希的方法，最早主要用在数据流频繁项集的挖掘中，随后被Pham等人证明可以通过计算Count Sketch的卷积来估计两个向量的外积(即向量间各元素两两相乘)。
由于上述提到的特征图为c个w*h的矩阵，而TS算法的输入为向量，所以本发明在使用TS算法时对特征图中每个位置依次计算，即分别对w*h个c维向量进行运算，重映射到w*h*d维的空间中。首先随机生成用于进行哈希操作的参数集合hk∈{1,…,d}c，sk∈{1,-1}c(k＝1,2)，其中hk用于存储输入向量重映射后的索引，sk实现了输入向量各元素数值的随机取反。根据上述参数集合，可以通过累加计算得到重映射后的Count Sketch向量。由卷积定理可知，时域或空间域的卷积等于对应频域内的乘积。所以可以使用快速傅立叶变换(FFT)将两个Count Sketch向量转换到频域，求其在频域的乘积，然后通过傅里叶反变换转换回空间域，计算得到Count Sketch向量的卷积。具体算法如下所示。
Figure PCTCN2017113068-appb-000007
Figure PCTCN2017113068-appb-000008
除了上述双线性运算层的改进，本发明还将原网络结构中最后一层卷积层后面的全连接层改为平均池化层(Average Pooling)，该层对整张特征图进行池化操作，对每张特征图的元素求平均值，最终得到d维的向量。使用池化层代替全连接层是因为池化层并没有可学习的参数，这样做大大减少了模型参数，加快了收敛速度，并在一定程度上避免了过拟合的发生。同时保证了特征图与池化后的特征向量之间的对应关系，为后续提取注意力分布图创造了条件。
图像整体隐私感知：该步骤用于自动鉴别输入图像是否为隐私图像，将测试图像输入上述训练好的隐私感知网络，根据网络输出的各类别从属概率确定其是否为隐私图像。
图像隐私区域感知：本步骤用于自动检测图像中的隐私区域。主要通过网络的深层卷积特征提取注意力分布图，定位注意力集中区域从而完成对隐私区域的感知。
由于池化层节点和特征图有对应关系，可以通过特征图的加权求和得到注意力分布图。设通过降维后的双线性特征图序列P＝{p1,p2,…,pd}，最后生成的注意力分布图为A，则其计算公式如下：
Figure PCTCN2017113068-appb-000009
其中n为输入图像经过分类后从属的类别标签,
Figure PCTCN2017113068-appb-000010
表示对池化层第k个节点对应类别n的连接权重。
本发明根据上述结果进行隐私图像局部的定位，具体方法是将由上述步骤得到的注意力分布图进行尺度变化，转换为原图大小。并设定阈值完成图像二值化，求解二值化后图像的最小外接矩阵作为隐私图像局部感知的结果。
本发明应用广泛，例如：
方案一、在社交网络中，照片分享已经成为越来越受欢迎的交流方式。然而用户在进行照片分享中存在一定的安全隐患，例如，很多人尤其是年轻人在没有足够考虑到自己安全的情况下直接将可能暴露个人隐私的照片 分享到社交网络，一些不法分子可能会利用这些信息从事非法活动，这无疑给自己或自己的亲友带来了一定的安全威胁。对此，如果使用本发明中的隐私感知机制，就可以及时提醒上传者照片可能涉及的隐私，起到防微杜渐的作用。另外，在一些情况下，用户希望对公开照片中涉及隐私的区域进行遮蔽或模糊化处理。而对隐私区域进行处理需要耗费大量人力和时间，本发明所提供的图像隐私敏感区域感知的方法能够较好地解决上述问题，该方法可以自动定位图像中的隐私区域，方便后续处理，避免了人工的操作。
方案二、当前云存储的应用越来越广泛，云平台汇集了大量用户的个人信息，其中有很大一部分是图像数据。然而大部分的云平台属于不可信系统，云平台泄漏个人数据的事件屡见不鲜。为了保障个人隐私不被泄漏，一些公司使用加密或数据扰动的方式进行隐私保护，但对于海量的图像数据全部进行处理需要大量的计算资源。这时如果使用发明中所涉及的方法对图像数据进行分析，先区分隐私图像或者定位隐私敏感区域，进行有针对性的保护，就能够在保证信息安全的情况下大大减少计算的开销。
本发明一方面改进了现有图像隐私感知方法的一些缺陷，另一方面将隐私感知问题引申到了对图像隐私区域的感知，以满足不同的需求。对比于传统隐私感知方法，本发明只对于图像内容特征和类别进行训练，不受用户设定的图像标签和访问策略的约束，能够在多种应用场景上发挥作用。同时本发明使用了深度卷积网络，相比传统的特征提取方法具有更强的特征表达性，增加了模型的分类准确率和泛化能力。
本发明提供的一种基于深度学习的图像隐私感知方法具有如下优点：
一、为图像隐私保护提供了很好的先决条件。本发明提出了自动的隐私感知机制，能对图像和图像局部的隐私进行感知，满足了图像隐私保护的多样化需求。能够在保障用户隐私安全的前提下，选择性地保护隐私图像，大幅度节省隐私保护的计算开销。
二、本发明中构建的隐私感知数据集包含了根据大量隐私语料搜索所得的图像，使模型能够感知包括证件照、文件快照等多种常见的隐私类别，具有较强的通用性。
三、本发明在训练和测试阶段都采用端到端的方式(输入端为原始图片，输出端为感知结果，过程不需要人为干涉)，并使用同一模型同时完成了对隐私图像和图像隐私区域的感知，模型使用方便，并很容易推广到多 种实际应用场景当中。
四、引入了双线性运算的优化策略，在原模型的基础上进一步改善了特征表达能力，有利于提高图像感知准确率，并对隐私区域的定位有较大裨益。
以上内容是结合具体的优选实施方式对本发明所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干简单推演或替换，都应当视为属于本发明的保护范围。

Claims (10)

  1. 一种基于深度学习的图像隐私感知方法，其特征在于，包括以下步骤：
    S1、构建带类别标注的隐私分类数据集，使用迁移学习的方法训练隐私感知网络；
    S2、使用面向隐私感知的深度卷积神经网络完成隐私图像的识别；
    S3、根据神经网络深层卷积特征提取注意力分布图，并定位注意力集中区域完成对图像隐私区域的感知。
  2. 根据权利要求1所述的基于深度学习的图像隐私感知方法，其特征在于，步骤S1包括：先在大规模图像数据集上对深度卷积神经网络模型进行预训练，然后构建隐私分类数据集，将预训练的深度卷积神经网络模型在隐私分类数据集上进行微调。
  3. 根据权利要求2所述的基于深度学习的图像隐私感知方法，其特征在于，步骤S2包括：在深度卷积神经网络的最后一层卷积层后加入双线性运算层，增强深度卷积神经网络模型的特征表达能力，同时将全连接层改为池化层。
  4. 根据权利要求3所述的基于深度学习的图像隐私感知方法，其特征在于，步骤S3包括：根据池化层的各节点权重和经过双线性运算的特征图的对应关系，得到加权的高层特征图作为注意力分布图，并通过尺度变换，定位原图中的隐私区域。
  5. 根据权利要求3所述的基于深度学习的图像隐私感知方法，其特征在于：双线性运算层主要计算卷积后的特征图之间两两点乘的结果，
    设原特征图集合M＝{m1,m2,…,mn}，
    输出的双线性特征图集合为M’＝{m’1,m’2,…,m’n×n},
    则转换的公式如下：
    Figure PCTCN2017113068-appb-100001
    其中
    Figure PCTCN2017113068-appb-100002
    代表矩阵的点乘，
    Figure PCTCN2017113068-appb-100003
    代表向上取整，n表示原特征图的个数，i表示双线性特征图的下标。
  6. 根据权利要求5所述的基于深度学习的图像隐私感知方法，其特征在于：
    对双线性特征图进行降维操作。
  7. 根据权利要求6所述的基于深度学习的图像隐私感知方法，其特征在于：采用Tensor Sketch算法对双线性特征图进行降维操作。
  8. 根据权利要求7所述的基于深度学习的图像隐私感知方法，其特征在于：双线性特征图为c个w*h的矩阵，而Tensor Sketch算法的输入为向量，使用Tensor Sketch算法时对双线性特征图中每个位置依次计算，即分别对w*h个c维向量进行运算，重映射到w*h*d维的空间中；首先随机生成用于进行哈希操作的参数集合hk∈{1,…,d}c，sk∈{1,-1}c(k＝1,2)，其中hk用于存储输入向量重映射后的索引，sk实现了输入向量各元素数值的随机取反；根据上述参数集合，通过累加计算得到重映射后的Count Sketch向量；由卷积定理可知，时域或空间域的卷积等于对应频域内的乘积；所以使用快速傅立叶变换将两个Count Sketch向量转换到频域，求其在频域的乘积，然后通过傅里叶反变换转换回空间域，计算得到CountSketch向量的卷积。
  9. 根据权利要求7所述的基于深度学习的图像隐私感知方法，其特征在于：将全连接层改为平均池化层，该平均池化层对整张特征图进行池化操作，对每张特征图的元素求平均值，最终得到d维的向量。
  10. 据权利要求9所述的基于深度学习的图像隐私感知方法，其特征在于：平均池化层节点和特征图有对应关系，通过特征图的加权求和得到注意力分布图；
    设通过降维后的双线性特征图序列P＝{p1,p2,…,pd}，
    最后生成的注意力分布图为A，则其计算公式如下：
    Figure PCTCN2017113068-appb-100004
    其中n为输入图像经过分类后从属的类别标签,
    Figure PCTCN2017113068-appb-100005
    表示对池化层第k个节点对应类别n的连接权重；
    根据上述结果进行隐私图像局部的定位，具体方法是将由上述步骤得到的注意力分布图进行尺度变化，转换为原图大小，并设定阈值完成图像二值化，求解二值化后图像的最小外接矩阵作为隐私图像局部感知的结果。
PCT/CN2017/113068 2017-10-09 2017-11-27 一种基于深度学习的图像隐私感知方法 WO2019071754A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/099,836 US11256952B2 (en) 2017-10-09 2017-11-27 Image privacy perception method based on deep learning

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710928967.XA CN107704877B (zh) 2017-10-09 2017-10-09 一种基于深度学习的图像隐私感知方法
CN201710928967.X 2017-10-09

Publications (1)

Publication Number Publication Date
WO2019071754A1 true WO2019071754A1 (zh) 2019-04-18

Family

ID=61184658

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/113068 WO2019071754A1 (zh) 2017-10-09 2017-11-27 一种基于深度学习的图像隐私感知方法

Country Status (3)

Country Link
US (1) US11256952B2 (zh)
CN (1) CN107704877B (zh)
WO (1) WO2019071754A1 (zh)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110163177A (zh) * 2019-05-28 2019-08-23 李峥嵘 一种风电机组叶片无人机自动感知识别方法
CN110717856A (zh) * 2019-09-03 2020-01-21 天津大学 一种用于医学成像的超分辨率重建算法
CN111724424A (zh) * 2020-06-24 2020-09-29 上海应用技术大学 图像配准方法
CN111784757A (zh) * 2020-06-30 2020-10-16 北京百度网讯科技有限公司 深度估计模型的训练方法、深度估计方法、装置及设备
CN111814165A (zh) * 2020-07-07 2020-10-23 重庆大学 一种基于深度神经网络中间层的图像隐私保护方法
CN111967318A (zh) * 2020-07-13 2020-11-20 北京邮电大学 一种基于隐私保护原则的摄像头辅助车联网无线通信方法
CN112347512A (zh) * 2020-11-13 2021-02-09 支付宝(杭州)信息技术有限公司 图像处理方法、装置、设备及存储介质
CN113095989A (zh) * 2021-03-31 2021-07-09 西安理工大学 一种基于图像风格迁移化的零水印版权保护算法
CN113642717A (zh) * 2021-08-31 2021-11-12 西安理工大学 一种基于差分隐私的卷积神经网络训练方法
CN115114967A (zh) * 2020-09-21 2022-09-27 武汉科技大学 基于自组织增量-图卷积神经网络的钢材微观组织自动分类方法
US11704433B2 (en) 2020-09-21 2023-07-18 International Business Machines Corporation Dynamic photograph classification
CN116721302A (zh) * 2023-08-10 2023-09-08 成都信息工程大学 一种基于轻量级网络的冰雪晶粒子图像分类方法

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11734567B2 (en) * 2018-02-13 2023-08-22 Samsung Electronics Co., Ltd. Method and system for reducing deep neural network architectures
CN109101523A (zh) * 2018-06-14 2018-12-28 北京搜狗科技发展有限公司 一种图像处理方法、装置和电子设备
CN109145816B (zh) * 2018-08-21 2021-01-26 北京京东尚科信息技术有限公司 商品识别方法和系统
CN109376757B (zh) * 2018-09-06 2020-09-08 苏州飞搜科技有限公司 一种多标签分类方法及系统
CN109743579A (zh) * 2018-12-24 2019-05-10 秒针信息技术有限公司 一种视频处理方法及装置、存储介质和处理器
CN109743580A (zh) * 2018-12-24 2019-05-10 秒针信息技术有限公司 一种视频处理方法及装置、存储介质和处理器
CN109756842B (zh) * 2019-02-19 2020-05-08 山东大学 基于注意力机制的无线室内定位方法及系统
CN109993207B (zh) * 2019-03-01 2022-10-25 华南理工大学 一种基于目标检测的图像隐私保护方法和系统
CN109993212B (zh) * 2019-03-06 2023-06-20 西安电子科技大学 社交网络图片分享中的位置隐私保护方法、社交网络平台
CN110334571B (zh) * 2019-04-03 2022-12-20 复旦大学 一种基于卷积神经网络的毫米波图像人体隐私保护方法
US10762607B2 (en) 2019-04-10 2020-09-01 Alibaba Group Holding Limited Method and device for sensitive data masking based on image recognition
CN110163218A (zh) * 2019-04-10 2019-08-23 阿里巴巴集团控股有限公司 基于图像识别的脱敏处理方法以及装置
CN109903289B (zh) * 2019-04-17 2023-05-05 广东工业大学 一种太赫兹图像无损检测的方法、装置以及设备
CN110069947B (zh) * 2019-04-22 2020-09-15 鹏城实验室 图片隐私的预测方法及装置、存储介质及电子设备
CN111860068A (zh) * 2019-04-30 2020-10-30 四川大学 一种基于跨层精简双线性网络的细粒度鸟类识别方法
CN110175469B (zh) * 2019-05-16 2020-11-17 山东大学 一种社交媒体用户隐私泄漏检测方法、系统、设备及介质
KR102234097B1 (ko) * 2019-07-17 2021-04-01 부산대학교 산학협력단 딥러닝을 위한 이미지 처리 방법 및 이미지 처리 시스템
CN111177757A (zh) * 2019-12-27 2020-05-19 支付宝(杭州)信息技术有限公司 一种图片中隐私信息保护的处理方法及装置
CN111639359B (zh) * 2020-04-22 2023-09-12 中国科学院计算技术研究所 一种用于社交网络图片隐私风险检测与预警的方法及系统
CN111753885B (zh) * 2020-06-09 2023-09-01 华侨大学 一种基于深度学习的隐私增强数据处理方法和系统
WO2021143267A1 (zh) * 2020-09-07 2021-07-22 平安科技（深圳）有限公司 基于图像检测的细粒度分类模型处理方法、及其相关设备
US20220286438A1 (en) * 2021-03-08 2022-09-08 Adobe Inc. Machine learning techniques for mitigating aggregate exposure of identifying information
CN113837269A (zh) * 2021-09-23 2021-12-24 中国特种设备检测研究院 基于双线性卷积神经网络的金相组织识别方法
CN114091651B (zh) * 2021-11-03 2024-05-24 支付宝(杭州)信息技术有限公司 多方联合训练图神经网络的方法、装置及系统
CN114419719B (zh) * 2022-03-29 2022-08-12 北京爱笔科技有限公司 一种生物特征的处理方法及装置
CN115906186B (zh) * 2023-02-16 2023-05-16 广州优刻谷科技有限公司 一种人脸图像隐私保护方法、装置及存储介质
CN116740650B (zh) * 2023-08-10 2023-10-20 青岛农业大学 一种基于深度学习的作物育种监测方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104102929A (zh) * 2014-07-25 2014-10-15 哈尔滨工业大学 基于深度学习的高光谱遥感数据分类方法
CN106295584A (zh) * 2016-08-16 2017-01-04 深圳云天励飞技术有限公司 深度迁移学习在人群属性的识别方法
CN106682694A (zh) * 2016-12-27 2017-05-17 复旦大学 一种基于深度学习的敏感图像识别方法
CN106778740A (zh) * 2016-12-06 2017-05-31 北京航空航天大学 一种基于深度学习的tfds非故障图像检测方法
US20170193336A1 (en) * 2014-12-22 2017-07-06 Yahoo! Inc. Generating preference indices for image content

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10878320B2 (en) * 2015-07-22 2020-12-29 Qualcomm Incorporated Transfer learning in neural networks
WO2017129804A1 (en) * 2016-01-29 2017-08-03 Kiwisecurity Software Gmbh Methods and apparatus for using video analytics to detect regions for privacy protection within images from moving cameras

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104102929A (zh) * 2014-07-25 2014-10-15 哈尔滨工业大学 基于深度学习的高光谱遥感数据分类方法
US20170193336A1 (en) * 2014-12-22 2017-07-06 Yahoo! Inc. Generating preference indices for image content
CN106295584A (zh) * 2016-08-16 2017-01-04 深圳云天励飞技术有限公司 深度迁移学习在人群属性的识别方法
CN106778740A (zh) * 2016-12-06 2017-05-31 北京航空航天大学 一种基于深度学习的tfds非故障图像检测方法
CN106682694A (zh) * 2016-12-27 2017-05-17 复旦大学 一种基于深度学习的敏感图像识别方法

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110163177B (zh) * 2019-05-28 2022-12-09 李峥嵘 一种风电机组叶片无人机自动感知识别方法
CN110163177A (zh) * 2019-05-28 2019-08-23 李峥嵘 一种风电机组叶片无人机自动感知识别方法
CN110717856A (zh) * 2019-09-03 2020-01-21 天津大学 一种用于医学成像的超分辨率重建算法
CN111724424A (zh) * 2020-06-24 2020-09-29 上海应用技术大学 图像配准方法
CN111724424B (zh) * 2020-06-24 2024-05-14 上海应用技术大学 图像配准方法
CN111784757A (zh) * 2020-06-30 2020-10-16 北京百度网讯科技有限公司 深度估计模型的训练方法、深度估计方法、装置及设备
CN111784757B (zh) * 2020-06-30 2024-01-23 北京百度网讯科技有限公司 深度估计模型的训练方法、深度估计方法、装置及设备
CN111814165A (zh) * 2020-07-07 2020-10-23 重庆大学 一种基于深度神经网络中间层的图像隐私保护方法
CN111814165B (zh) * 2020-07-07 2024-01-26 重庆大学 一种基于深度神经网络中间层的图像隐私保护方法
CN111967318A (zh) * 2020-07-13 2020-11-20 北京邮电大学 一种基于隐私保护原则的摄像头辅助车联网无线通信方法
US11704433B2 (en) 2020-09-21 2023-07-18 International Business Machines Corporation Dynamic photograph classification
CN115114967A (zh) * 2020-09-21 2022-09-27 武汉科技大学 基于自组织增量-图卷积神经网络的钢材微观组织自动分类方法
CN112347512A (zh) * 2020-11-13 2021-02-09 支付宝(杭州)信息技术有限公司 图像处理方法、装置、设备及存储介质
CN113095989B (zh) * 2021-03-31 2023-07-07 西安理工大学 一种基于图像风格迁移化的零水印版权保护算法
CN113095989A (zh) * 2021-03-31 2021-07-09 西安理工大学 一种基于图像风格迁移化的零水印版权保护算法
CN113642717A (zh) * 2021-08-31 2021-11-12 西安理工大学 一种基于差分隐私的卷积神经网络训练方法
CN113642717B (zh) * 2021-08-31 2024-04-02 西安理工大学 一种基于差分隐私的卷积神经网络训练方法
CN116721302A (zh) * 2023-08-10 2023-09-08 成都信息工程大学 一种基于轻量级网络的冰雪晶粒子图像分类方法
CN116721302B (zh) * 2023-08-10 2024-01-12 成都信息工程大学 一种基于轻量级网络的冰雪晶粒子图像分类方法

Also Published As

Publication number Publication date
CN107704877A (zh) 2018-02-16
CN107704877B (zh) 2020-05-29
US11256952B2 (en) 2022-02-22
US20210224586A1 (en) 2021-07-22

Similar Documents

Publication Publication Date Title
WO2019071754A1 (zh) 一种基于深度学习的图像隐私感知方法
Wu et al. A network intrusion detection method based on semantic Re-encoding and deep learning
Al-Qershi et al. Evaluation of copy-move forgery detection: datasets and evaluation metrics
CN107992764B (zh) 一种敏感网页识别与检测方法及装置
CN109271546A (zh) 图像检索特征提取模型建立、数据库建立及检索方法
Uma et al. Copy-move forgery detection of digital images using football game optimization
Cheng et al. An efficient fire detection algorithm based on multi‐scale convolutional neural network
Deng et al. Self-feedback image retrieval algorithm based on annular color moments
Zhou et al. Residual visualization-guided explainable copy-relationship learning for image copy detection in social networks
Qin et al. Multi-scaling detection of singular points based on fully convolutional networks in fingerprint images
WO2021081741A1 (zh) 一种基于多关系社交网络的图像分类方法及系统
CN111144453A (zh) 构建多模型融合计算模型的方法及设备、网站数据识别方法及设备
Nair et al. Identification of multiple copy-move attacks in digital images using FFT and CNN
Chen et al. Remote sensing image monitoring and recognition technology for the conservation of rare wild animals
Qiao et al. Toward intelligent detection modelling for adversarial samples in convolutional neural networks
Monson et al. Behaviour knowledge space-based fusion for image forgery detection
Ligade et al. Content Based Image Retrieval Using Interactive Genetic Algorithm with Relevance Feedback Technique—Survey
Greenwell et al. GeoFaceExplorer: Exploring the geo-dependence of facial attributes
Wang et al. Smilies: A Soft-Multi-Label-Guided Weakly Supervised Semantic Segmentation Framework for Remote Sensing Images
Wang et al. Research on Digital Media Image Data Tampering Forensics Technology Based on Improved CNN Algorithm
Lakshminarasimha et al. Deep Learning Base Face Anti Spoofing-Convolutional Restricted Basis Neural Network Technique
Rajath et al. A Comprehensive Analysis on Deep Learning based Image Retrieval
CN117131503B (zh) 一种用户行为的威胁链识别方法
Pho et al. Attention-driven retinanet for parasitic egg detection
Qian et al. [Retracted] Cloud Data Access Prevention Method in Face Recognition Technology Based on Computer Vision

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17928570

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17928570

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17928570

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17928570

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25/01/2021)