WO2021258284A1 - Edge processing data de-identification - Google Patents

Edge processing data de-identification Download PDF

Info

Publication number
WO2021258284A1
WO2021258284A1 PCT/CN2020/097708 CN2020097708W WO2021258284A1 WO 2021258284 A1 WO2021258284 A1 WO 2021258284A1 CN 2020097708 W CN2020097708 W CN 2020097708W WO 2021258284 A1 WO2021258284 A1 WO 2021258284A1
Authority
WO
WIPO (PCT)
Prior art keywords
features
feature
data
identifying
captured data
Prior art date
Application number
PCT/CN2020/097708
Other languages
English (en)
French (fr)
Inventor
Ying Chen
Yucheng Zhao
Yangyan LI
Original Assignee
Alibaba Group Holding Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Limited filed Critical Alibaba Group Holding Limited
Priority to PCT/CN2020/097708 priority Critical patent/WO2021258284A1/en
Priority to CN202080100159.4A priority patent/CN115443466A/zh
Publication of WO2021258284A1 publication Critical patent/WO2021258284A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G06F21/6254—Protecting personal data, e.g. for financial or medical purposes by anonymising data, e.g. decorrelating personal data from the owner's identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification

Definitions

  • Modern cloud computing services commonly rely upon deep learning computation, in the form of backend computation at a cloud computing system connected to many end devices for collecting and processing data.
  • Such technology may provide backend computation for Internet of Things ( “IoT” ) services such as smart homes, smart appliances and home security.
  • IoT Internet of Things
  • Deep learning cloud services are often provided by computing complex data in multiple formats received from end devices.
  • cloud computing systems may be high in processing power, including examples such as distributed computing frameworks deployed on servers of a data center, which may include powerful microprocessors such as Neural Processing Units ( “NPUs” )
  • servers and other network nodes at edges of the computing system are generally low-power in specification. These edge nodes may have limited processing power and local storage and memory.
  • end devices themselves may be similarly low-power in specification.
  • deep learning computation may be deployed at computing systems which are inherently low-powered.
  • IoT services capture data of individual end users at individual end devices, and providers of IoT services desire to store and aggregate this captured data as a component of machine learning to improve backend technology of IoT services.
  • captured data may be inspected, analyzed, or otherwise utilized to identify individual end users.
  • a privacy boundary may be demarcated between computing systems where data is captured and computing systems where captured data is stored. While captured data has not crossed this conceptual boundary, to the extent that the captured data may be utilized to identify individual end users, access to the captured data remains limited. However, after the captured data has crossed this conceptual boundary, access to the captured data is outside of the control of individual end users. For such reasons, individual end users may distrust the collection of captured data at computing systems on an opposite site of a privacy boundary, such as cloud computing systems.
  • FIG. 1 illustrates an example of facial feature extraction as known to persons skilled in the art.
  • FIG. 2 illustrates an example of facial recognition as known to persons skilled in the art.
  • FIG. 3 illustrates an example feature embedding
  • FIGS. 4A, 4B, 4C, 4D, and 4E illustrate an image capture sensor as known to persons skilled in the art.
  • FIG. 5 illustrates an architecture of an identifying system according to example embodiments of the present disclosure.
  • FIG. 6 illustrates an architectural diagram of a computing system according to example embodiments of the present disclosure.
  • FIG. 7 illustrates a feature synthesizing model according to example embodiments of the present disclosure.
  • FIG. 8 illustrates a flowchart of a de-identifying method according to example embodiments of the present disclosure.
  • FIG. 9 illustrates an example system for implementing the processes and methods described above for implementing de-identification of captured data.
  • Systems and methods discussed herein are directed to implementing data de-identification, and more specifically replacement of features extracted from captured data at edge devices before the captured data is collected at a cloud computing system.
  • a learning model may be a defined computation algorithm executable by one or more processors of a computing system to perform tasks that include processing captured data having various parameters and outputting results.
  • a learning model may be, for example, a layered model such as a deep neural network, which may have a fully-connected structure, may have a feedforward structure such as a convolutional neural network ( “CNN” ) , may have a backpropagation structure such as a recurrent neural network ( “RNN” ) , or may have other architectures suited to the computation of particular tasks.
  • Tasks may include, for example, classification, clustering, matching, regression, and the like.
  • Tasks may provide output for the performance of functions such as recognizing entities in images and/or video; tracking movement of entities in video in real-time; matching recognized entities in images and/or video to other images and/or video; recognizing audio such as spoken words or noises; providing annotations or transcriptions of images, video, and/or audio in real-time; and the like.
  • a learning model may configure a computing system to perform computations for a task on captured data of, for example, any or several types as described above, wherein the captured data are generally compressed and features extracted therefrom before the computing system performs computations upon the extracted features.
  • biometrics generally refer to characteristics which may identify a particular person; biometrics may further be used in technological systems as authenticating identifiers to implement access control, limiting access to some particular secured information, resource, system, and the like to particular individuals.
  • biometric identifiers include immutable physical identifiers, such as a face and features thereof; retinas and/or irises and features thereof; DNA and features thereof; hands and features thereof such as hand geometric or veins; and the like.
  • Such examples also include behavioral identifiers, such as voice and features thereof; typing cadence and features thereof; gait and features thereof; and the like.
  • biometric identifiers may be extracted from captured data.
  • a learning model may be trained to configure a computing system to compute a task taking captured facial images as input, such tasks providing output for functions including facial recognition or facial matching.
  • features extracted therefrom may include, for example, eye features, nose features, and mouth features.
  • Facial features have various characteristics which may facilitate tasks such as facial recognition or facial matching. Facial features are generally independent from other facial features; for example, shapes of eyes, noses, and mouths are generally independent of each other. Furthermore, facial features may allow determination of characteristics which may indirectly lead to or narrow down identification of an individual person, such as age and gender. Additionally, facial features may allow determination of behavioral characteristics such as emotion, not just physical characteristics, enabling a different pathway of identifying an individual person.
  • FIG. 1 illustrates an example of facial feature extraction as known to persons skilled in the art.
  • the multi-task cascaded convolutional neural network 100 ( “MTCNN” ) as FIG. 1 illustrates establishes a three-stage cascaded network, including a first-stage proposal network ( “P-Net 102” ) ; a second-stage refine network ( “R-Net 104” ) ; and a third-stage output network 106 ( “O-Net 106” ) .
  • Sample image data, prior to input into a P-Net 102 may be transformed by blurring and subsampling to generate multiple copies of the sample image data forming an image pyramid 108.
  • convolutional layers 114 of the P-Net 102 perform bounding box regression upon the image pyramid 108.
  • the P-Net 102 outputs bounding boxes described by regression vectors, which are possible bounding boxes of facial features in sample image data of the image pyramid 108.
  • Candidate regions 110 for facial features sought to be identified by classification in the sample image data are identified based on the bounding boxes, by non-maximum suppression ( “NMS” ) , causing the bounding boxes to be merged where they overlap to generate new regression vectors; each candidate region 110 may receive a classification 112 as being likely a particular type of facial feature.
  • NMS non-maximum suppression
  • the candidate regions 110 are then input into the R-Net 104, wherein convolutional layers 116 of the R-Net 104 perform refinement of the regression vectors of the candidate regions 110 by further NMS.
  • the candidate regions 110 are then input into the O-Net 106, wherein convolutional layers 118 of the O-Net 106 perform further supervised refinement of the regression vectors of the candidate regions 110.
  • the O-Net 106 may output candidate regions 110, their classifications 112, and positions of some number of facial landmarks 118, which may assist in determining a correct alignment and orientation of a face identified by facial regions.
  • a learning model may be trained to configure a computing system to compute a task taking captured images or video as input, such as tasks providing output for functions such as image classification, computer vision, video tracking, video annotation.
  • image feature (s) features extracted therefrom
  • video feature (s) features extracted therefrom
  • video feature (s) may include, for example, motion features as known to persons skilled in the art.
  • Cloud computing systems may provide collections of servers hosting computing resources to provide distributed computing, parallel computing, improved availability of physical or virtual computing resources, and such benefits. Cloud computing systems may host learning models to provide these benefits for the application of computing using learning models. Nevertheless, to alleviate computational overhead in executing learning models on a cloud computing system, learning models may be pre-trained to provide ready-made parameters and weights which may be stored on storage of the cloud computing system and, upon execution, loaded into memory of the cloud computing system as a “backbone” learning model. For example, with regard to tasks relating to the function of image recognition, commonly available pre-trained image classifier learning models include ResNet, GoogLeNet, VGGNet, Inception, Xception, and the like.
  • a backbone learning model may be trained to compute inputs for a task taking images as input, such tasks providing output for functions including image classification or computer vision.
  • features may include, for example, edge features, corner features, and blob features as known to persons skilled in the art.
  • a backbone learning model may be trained to compute inputs for a task taking facial images as input, such tasks providing output for functions including facial recognition or facial matching.
  • features may include, for example, eye features, nose features, and mouth features as known to persons skilled in the art.
  • a backbone learning model may be trained to compute inputs for a task taking video as input, such as tasks providing output for functions such as video tracking or video annotation.
  • features may include, for example, motion features as known to persons skilled in the art.
  • a backbone learning model may be trained to compute inputs for a task taking audio as input, such as tasks providing output for functions such as audio recognition, source separation or audio annotation.
  • features may include, for example, zero crossing rate features, energy features, spectral shape features such as spectral centroid features, spectral spread features, and the like, and such features as known to persons skilled in the art.
  • a central network of a learning model may be trained to compute functions for a task taking text as input, such as tasks providing output for functions such as an image search function.
  • features which are heterogeneous from features of non-text input may include, for example, word count features, word vectors, and such features as known to persons skilled in the art.
  • FIG. 2 illustrates an example of facial recognition as known to persons skilled in the art.
  • the FaceNet convolutional neural network 200 as FIG. 2 illustrates may first input extracted facial features into a backbone 202, whereupon the backbone 202 performs computations of a nature as described above.
  • the facial features may then be input into a normalization layer 204, whereupon the normalization layer 204 may compute a feature embedding 206 of the facial features.
  • Feature embedding generally refers to translating a data set into a dimensional space of reduced dimensionality so as to increase, or maximize, distances between data points (such as individual images) which need to be distinguished in computing a task for a particular function, and decrease, or minimize, distances between data points which need to be matched, clustered, or otherwise found similar in computing a task for a particular function.
  • functions for expressing distance between two data points may be any function which expresses Euclidean distance, such as L 2 -norm; Manhattan distance; any function which expresses cosine distance, such as the negative of cosine similarity; or any other suitable distance function as known to persons skilled in the art.
  • FaceNet is implemented to reduce dimensionality of facial features to a comparatively compact set of 128 dimensions, by L 2 -norm.
  • Training of the learning model may, in part, be performed to train the learning model on a loss function to learn a feature embedding of the extracted features.
  • the loss function may be any function having a first distance and a second distance as parameters which may be simultaneously optimized for a minimal value of the first distance and a maximal value of the second distance.
  • the second loss function may be a triplet loss function, which, generally, is a function which takes, as parameters, an anchor data point a, a positive data point p which matches the anchor data point with regard to a feature thereof, and a negative data point n which does not match the anchor data point with regard to the feature.
  • the triplet loss function may calculate a first distance between the anchor data point a and the positive data point p, calculate a second distance between the anchor data point a and the negative data point n, and calculate a difference between the first distance and the second distance; the difference between the first distance and the second distance may penalize the learning model. Therefore, training the learning model on the triplet loss function may generate a learned feature embedding which optimizes for minimizing the difference between the first distance and the second distance.
  • the triplets input into the loss function may be selected to include semi-hard negatives.
  • Hard negative data points refer to those negative data points nearest the anchor data point, where “hardest negative” refers to negative data points nearest to the anchor data point and nearer than positive data points, and therefore the most difficult to distinguish from positive data points in general, and “semi-hard negative” refers to negative data points nearest to the anchor data point without being nearer than positive data points, and therefore difficult to distinguish from positive data points but less difficult than hardest negatives. Therefore, training the learning model on the triplet loss function may generate a learned feature embedding which simultaneously optimizes for hard positive data points being nearer to the target data point and hard negative data points being further from the target point.
  • a reference dataset may be obtained for training the neural network at 208.
  • Reference data may generally be any labeled dataset indicating whether data points therein are positive or negative for a particular result.
  • the dataset may be labeled to indicate that a particular data point is positive or negative for a particular result.
  • the dataset may be labeled to indicate classes, clusters, fittings, or other characteristics of data points, such that the labels may indicate that a particular data point does or does not belong to a particular class, cluster, fitting, and the like.
  • the neural network may be trained over some number of epochs, where an epoch refers to a period during which an entire dataset (the abovementioned reference dataset) is computed by the learning model once and a weight set is updated based thereupon.
  • An epoch is divided into multiple batches; during each batch, a subset of the reference data is computed by the learning model.
  • the reference dataset may be segmented into multiple subsets, each for input during one batch.
  • loss functions such as triplet loss functions where data points are obtained, to avoid the excess computation overhead of obtaining data points from the entire dataset, data points obtained during a batch may be obtained from the subset for input during that batch.
  • a weight set is then updated based on a feature embedding learned by the learning model.
  • the weight set may be updated according to gradient descent ( “GD” ) (that is, updated after computation completes for an epoch) , stochastic gradient descent ( “SGD” ) , mini-batch stochastic gradient descent ( “MB-SGD” ) (that is, updated after computation of each batch) , backpropagation ( “BP” ) , or any suitable other manner of updating weight sets as known to persons skilled in the art.
  • GD gradient descent
  • SGD stochastic gradient descent
  • M-SGD mini-batch stochastic gradient descent
  • BP backpropagation
  • the neural network may then generate feature embeddings on test data at 210.
  • FIG. 3 illustrates an example feature embedding 300, wherein relative positions of an anchor data point 302, positive data points 304, semi-hard data points 306, and, conversely, easy negatives 308 are shown relative to each other.
  • a cloud computing system may connect to various end devices which capture data to be input into learning models to train the learning models and/or in association with various tasks for the computation and output of results required for the performance of those tasks.
  • End devices may connect to the cloud computing system through edge nodes of the cloud computing system.
  • An edge node may be any server providing an outbound connection from connections to other nodes of the cloud computing system, and thus may demarcate a logical edge, and not necessarily a physical edge, of a network of the cloud computing system.
  • an edge node may be edge-based logical nodes that deploy non-centralized computing resources the cloud computing system, such as cloudlets, fog nodes, and the like.
  • End devices may be responsible for multiple types of task involving multiple types of captured data.
  • an end device may be a camera which captures still images, video, audio, facial image data, and other types of data which may originate from users of an IoT service such as a cloud security service, monitoring service, smart home service, and the like; connects to an edge node of a cloud computing system which hosts the IoT service by performing real-time monitoring of locations surrounding the end devices; and sends each of these types of data in real-time to the cloud computing system to perform various backend tasks supporting IoT services.
  • an IoT service such as a cloud security service, monitoring service, smart home service, and the like
  • FIGS. 4A, 4B, 4C, 4D, and 4E illustrate an image capture sensor as known to persons skilled in the art.
  • the image capture sensor may include a cavity array 400 as FIG. 4A illustrates. Cavities of the cavity array 400 may be photosites arrayed in numbers of millions or billions. As FIG. 4B illustrates, individual cavities 402 may be uncovered by operation of a shutter mechanism which exposes the cavity array 400 to photons of light. Each cavity 402 may measure a number of photons which struck the respective cavity 402 and quantify the number as a digital signal. Bit depth of the image capture sensor may determine a range of possible values for the digital signal; the range may be enhanced to output captured images at, for example, High Dynamic Range ( “HDR” ) .
  • HDR High Dynamic Range
  • the cavity array 400 may be a color filter array so that the image capture sensor is operable to capture images in color, wherein each individual cavity 402 is covered by a colored filter in red, green, or blue.
  • the cavity array 400 as illustrated may be a Bayer array, wherein alternating rows of cavities 402 include either red and green filters or blue and green filters. Image capture devices may utilized such arrays to implement image capture sensors.
  • FIG. 4D illustrates, individual cavities 402 may each be hit by only photons of a same color as their respective filters. At each photosite, therefore, signals representing the filtered color may be quantified and signals representing other colors may be approximated.
  • image capture devices further incorporate inter-cavity microlenses 404 which redirect photons into cavities 402.
  • the incorporation of microlenses 404 may enable an image capture sensor to be implemented without a cavity array 400 covering the entire surface of the sensor; instead, photons which would have struck sites without cavities are directed to adjacent cavities 402.
  • An image capture device further incorporates an image signal processor ( “ISP” ) .
  • Digital signals output by image capture sensors may be processed by an ISP to generate a captured image.
  • An ISP according to example embodiments of the present disclosure may include a frontend processing module and a post-processing module.
  • a frontend processing module may perform black level correction upon the digital signals, so as to compensate for baseline signal levels running through the image capture sensor while it is not exposed to any photons.
  • Black level correction may be performed by subtracting a baseline value from digital signal values, or by subtracting a drift curve modeled by a linear function based on temperature and gain from digital signal values.
  • a frontend processing module may perform lens shading correction upon the digital signals, so as to compensate for light fall-off occurring at the edge of a lens of an image capture device focusing photons onto the image capture sensor.
  • a frontend processing module may perform demosaicing upon the digital signals, so as to reconstruct full color information from the differently filtered photons captured by different cavities of the cavity array as described above.
  • a frontend processing module may perform auto-exposure, auto-focus, and auto-white balance operations based on the digital signal to control operation of the image capture device.
  • Auto-exposure operations may detect luminance levels of the digital signals, and utilize exposure metering, such as center-weighted metering, spot exposure metering, and area exposure metering, to control diaphragm, gain level, and shutter speed of the image capture device and maintain approximately constant luminance levels.
  • exposure metering such as center-weighted metering, spot exposure metering, and area exposure metering
  • Auto-focus operations may detect contrast levels of the digital signals, and define a focus distance and focus position of the image capture device based thereon to minimize noise.
  • Auto-white balance operations may perform color correction on the digital signals to cause average color over the captured image to be approximately gray. Furthermore, auto-white balance operations may estimate true colors over the captured image to further correct the digital signals to be closer to the estimated true colors.
  • a frontend processing module may perform global tone mapping upon the digital signals, wherein by a mapping function such as an exponential function, a sigmoid function, and the like, digital signal inputs having the same red, green, and blue intensities are mapped to a same output pixel value.
  • the mapping function may be implemented as a look-up table to reduce computation time.
  • a post-processing module may perform operations upon a captured image output by the frontend processing module.
  • a post-processing module may perform de-noising upon the captured image, such as by applying a low-pass filter and a bilateral filter thereto.
  • a post-processing module may perform edge sharpening upon the captured image, such as by obtaining a high-frequency signal of the captured image and adding the high-frequency signal, or a weighted high-frequency signal, to the captured image.
  • a frontend processing module or a post-processing module may perform further operations upon the digital signals or the captured image such as local tone mapping, histogram equalization, and the like as known to persons skilled in the art.
  • edge devices may have some degree of computational power, local storage and memory with which to perform such computation.
  • An ISP may output a captured image to a video codec, which may encode a sequence of captured images, compressing the size of the captured images in the process.
  • a captured image from a source may be encoded to generate a reconstructed frame, and the reconstructed frame may be output at a destination such as a buffer.
  • the frame may be input into a coding loop and output as a reconstructed frame.
  • Coding may be performed according to established standards such as the H. 264/AVC (Advanced Video Coding) and H. 265/HEVC (High Efficiency Video Coding) standards. Coding may be performed according to standards currently in development, such as the proposed Versatile Video Coding ( “VVC” ) specification.
  • coding may incur latency time between coding of each captured image and output of the captured image at a remote computing system, which shall be described subsequently.
  • acceptable latency may be greater or lesser than with regard to other IoT services.
  • acceptable latency may be in the range of seconds; with regard to a video conferencing service, acceptable latency may be limited to the range of milliseconds.
  • Workload of computing may be allocated between different edge devices in accordance with such acceptable latency in light of the inherent computational overhead of coding; for example, as edge nodes may have more computational power than end devices, captured images may be transferred to edge nodes for coding in order to leverage the increased computational power.
  • Edge devices of the IoT network may be geographically isolated from the computational resources of the cloud computing system, and may also be logically isolated from the cloud computing system.
  • captured data at an edge device may be separated from the cloud computing system by one or more data planes, such as data planes defining one or more networks which convey data between the edge device and the cloud computing system.
  • learning models may configure edge devices to compute backend tasks as described above.
  • learning models running on edge devices may not be trained at those same edge devices, and may be trained instead at higher-powered computing systems such as cloud computing systems.
  • Some computation by the learning model may be offloaded to localized computation at edge devices, by architectural considerations such as the following.
  • Raw captured data such as still images, video, facial image data, and audio signals, being large in file size, may be compressed or otherwise preprocessed at edge devices prior to being transferred over one or more networks, and/or edge devices may be configured to perform edge processing of backend tasks as described above using the captured data. Edge processing may be limited in computation capacity and scope.
  • captured data may be delivered from edge devices to one or more remote computing hosts over one or more networks through interfaces hosted at a cloud computing system.
  • the captured data may, conceptually, cross over a privacy boundary, defined as follows with reference to the architecture of a learning model according to example embodiments of the present disclosure.
  • FIG. 5 illustrates an architecture of an identifying system 500 according to example embodiments of the present disclosure.
  • the system 500 includes a registering module 502 and a querying module 504.
  • the registering module 502 and the querying module 504 each may take captured data, such as captured still images or images from video, as input.
  • the registering module 502 may take as input captured data which are labeled with individual user identities; features, such as facial features, may be extracted from the captured images by a feature extracting module 506.
  • the feature extracting module 506 may operate in a manner as described above with reference to FIG. 1.
  • the registering module 502 may register extracted features output by the feature extracting module 506 in a database 508 in association with their respective labeled individual user identities.
  • the querying module 504 may take as input captured data which are not labeled with individual user identities, such as images captured in real time during operation of IoT services as described above; features may be extracted from the captured images by the feature extracting module 506.
  • the feature extracting module 506 may operate in a manner as described above with reference to FIG. 1.
  • a matching module 510 may match the extracted features against labeled features registered in the database 508. Based on matches having smallest distances to labeled features, the matching module 510 may output user identities which match the unlabeled features most closely.
  • the above-described features of the identifying system 500 may be hosted on a remote cloud computing system as described above.
  • a conceptual privacy boundary may be defined having a first side including computing systems where data is captured but not computed by an identifying system as described above, and a second side including other computing systems where captured data is computed by an identifying system as described above.
  • an identifying system needs only a comparatively compact set of extracted features from captured data in order to match extracted features against labeled features recorded in its database.
  • the compact set of extracted features may be a small subset of all features which may be extracted from captured data.
  • captured data may be inspected by humans to identify individual users whose physical or behavioral characteristics are present in captured data, or may be computed by an identifying system which extracts features to be matched against labeled features.
  • Humans inspecting captured data may identify individual users by primarily overtly visible physical characteristics, such as by looking at faces of individuals in captured images. However, an identifying system may identify individual users by more nuanced and less overt physical characteristics as well as behavioral characteristics which are less evident to humans inspecting captured data.
  • captured data may have a substantial number of features therein replaced with non-identifying features, so as to obstruct the ability of humans to identify individual users by inspecting the captured data.
  • compact subsets of features may be retained in the data so as to not obstruct the ability of identifying systems according to example embodiments of the present disclosure to identify individual users by extracting the features from the captured data.
  • a replacement process shall be referred to as “de-identification. ”
  • FIG. 6 illustrates an architectural diagram of a computing system 600 according to example embodiments of the present disclosure.
  • the computing system 600 may be implemented over a cloud network 602 of physical or virtual server nodes 604 (1) , 604 (2) , ..., 604 (N) (where any unspecified server node may be referred to as a server node 604) connected by physical or virtual network connections.
  • the network 602 terminates at physical or virtual edge nodes 606 (1) , 606 (2) , ..., 606 (N) (where any unspecified edge node may be referred to as an edge node 606) located at physical and/or logical edges of the cloud network 602.
  • the edge nodes 606 (1) to 606 (N) may connect to any number of end devices 608 (1) , 608 (2) , ..., 608 (N) (where any unspecified end device may be referred to as an end device 608) .
  • An end device 608, such as, by way of example, end device 608 (1) may capture data from any number of sensors 610 (1) , 610 (2) , ..., 610 (N) (where any unspecified sensor 610 may be referred to as a sensor 610) , where the different sensors may operative to collect one or more types of data (such as images, videos, text, spoken audio, and the like) .
  • a pre-trained learning model may be implemented on special-purpose processor (s) 612, which may be hosted at a data center 614.
  • the data center 614 may be part of the cloud network 602 or in communication with the cloud network 602 by network connections.
  • Special-purpose processor (s) 612 may be computing devices having hardware or software elements facilitating computation of neural network computing tasks such as training and inference computations.
  • special-purpose processor (s) 612 may be accelerator (s) , such as Neural Network Processing Units ( “NPUs” ) , Graphics Processing Units ( “GPUs” ) , Tensor Processing Units ( “TPU” ) , implementations using field programmable gate arrays ( “FPGAs” ) and application specific integrated circuits ( “ASICs” ) , and/or the like.
  • accelerator s
  • NPUs Neural Network Processing Units
  • GPUs Graphics Processing Units
  • TPU Tensor Processing Units
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • a learning model may include an identifying model 616 and any number of de-identifying models 618 (1) , 618 (2) , ..., 618 (N) (where any unspecified de-identifying model may be referred to as a de-identifying model 618) .
  • the identifying model 616 may be stored on physical or virtual storage of the data center 614 ( “data center storage 620” ) , and may be loaded into physical or virtual memory of the data center 614 ( “data center memory 622” ) (which may be dedicated memory of the special-purpose processor (s) 612) alongside a trained weight set 624 in order for the special-purpose processor (s) 612 to execute the identifying model 616 to extract features from captured data and identify individual users based on those extracted features.
  • the de-identifying model 618 may be stored on physical or virtual edge storage of any number of the edge nodes 606 and/or end devices 608 ( “edge storage 626” ) , and may be loaded into physical or virtual edge memory of any number of the edge nodes 606 or end devices 608 ( “edge memory 628” ) alongside a trained weight set 630 in order for one or more physical or virtual edge processor (s) of the edge node 606 or end device 608 ( “edge processor (s) 632” ) to execute a sub-network 618 of the learning model to compute input related to one or more tasks.
  • edge devices for brevity.
  • Execution of a de-identifying model 618 at an edge device may cause the edge device to load the de-identifying model 618 into edge memory 628 and compute input data captured from sensors 610 of an end device 608.
  • the data may be captured by end devices 608 for the computation of backend tasks supporting IoT services, as described above.
  • the de-identifying model 618 may perform de-identifying operations upon the captured data, as shall be described subsequently.
  • the input may be obtained directly from the edge device if the edge device is an end device 608, or may be delivered to the edge device over a network connection from an end device 608 if the edge device is an edge node 606.
  • the de-identifying model 618 may replace features of the captured data and deliver this de-identified data to the central network 616 over the cloud network 602.
  • Execution of the identifying network 616 may then cause the special-purpose processor (s) 612 to load the identifying network 616 into data center memory 622 and extract features remaining in the captured data de-identified by de-identifying models 618.
  • the identifying network 616 may output results required for the performance of IoT services, including user identities.
  • Example embodiments of the present disclosure provide methods for de-identifying captured data by generating synthetic features to replace some number of biometric features which may be extracted from captured data, and replacing those features with the synthetic features, without passing captured data over a privacy boundary.
  • a conceptual privacy boundary 634 is depicted herein as a horizontal broken line, thought it should be understood that the privacy boundary does not necessarily correspond to any physical or virtual elements depicted in FIG. 6.
  • FIG. 7 illustrates a feature synthesizing model 700 according to example embodiments of the present disclosure.
  • the feature synthesizing model 700 may perform operations of a generative adversarial network ( “GAN” ) .
  • GAN generative adversarial network
  • the feature synthesizing model 702 may include a generator 704, which may be a first convolutional neural network, and a discriminator 706, which may be a second convolutional neural network.
  • the generator 704 may be trained based on captured data and extracted features to generate synthetic data which incorporates synthetic biometric features. For example, based on captured facial images and facial features extracted therefrom, the generator 704 may generate synthetic data which utilizes context of facial features (such as features related to physical characteristics, features related to behavioral characteristics such as expressions, contextual information such as lighting, skin color, wrinkles and the like, and labeled identities) to generate synthetic facial images.
  • facial features such as features related to physical characteristics, features related to behavioral characteristics such as expressions, contextual information such as lighting, skin color, wrinkles and the like, and labeled identities
  • the generator 704 may generate synthetic data which utilizes context of retinal data (such as retinal and iris patterns) to generate synthetic retinal and/or iris images; utilizes context of fingerprint data (such as arches, loops, and whorls) to generate synthetic fingerprints; utilizes context of handwriting (such as shapes, size, spacing, spacing, repetition, and sloping) to generate synthetic signatures; utilizes context of motion (such as step length, stride length, cadence, speed, and the like) to generate synthetic gait data.
  • Non-identifying features in general, may be features which, in combination, cannot be matched by an identifying system according to example embodiments of the present disclosure against records of any individual user.
  • the discriminator 706 may be trained based on real data 708 and synthetic data 710 to learn to discriminate synthetic data from real data. Based thereon, the discriminator 706 may feed output back to the generator 704 to train the generator 704 to improve synthesis of images.
  • edge devices may generate non-identifying features to replace extracted features from data captured by end devices.
  • features do not necessarily need to be generated.
  • edge devices may store generic features and replace extracted features from captured data with common generic features.
  • Generic features may be features of one particular individual person who is not any individual user of the IoT services with regard to example embodiments of the present disclosure; for example, generic features may be features of a “placeholder” person who has granted consent for their identifying features to be used in such a manner.
  • a generator 704 may synthesize features for sequences of captured data, such as captured data in the form of a video containing many individual frames. In these cases, a generator 704 should not necessarily synthesize features for each individual frame, as such synthesis would be significant in computational cost. Instead, the generator 704 may synthesize features for alternating frames of a video or for frames of a video separated by some number of predetermined steps. Alternatively, the generator 704 may, according to specifications of coding standards as described above, synthesize features for a keyframe of a sequence of frames; by incorporating such synthesizing mechanisms into the coding specification, those synthetic features may be carried to other frames of the same sequence by mechanisms such as motion prediction. Thus, according to example embodiments of the present disclosure, at least parts of a feature synthesizing model may be incorporated into a proposed coding standard specification, such as the VVC specification.
  • the feature synthesizing model 700 may be stored and executed on an edge device (either an edge node or an end device) as described above with reference to FIG. 6. However, it is also desirable to periodically train the feature synthesizing model 700 using random data, synthetic data and/or generic data which is continually updated to improve de-identification of features synthesized by the generator 704, and to improve GAN functionality by improving discriminating operations of the discriminator 706. Thus, a copy of the feature synthesizing model 700 may be periodically trained at the cloud network 602 as described with reference to FIG.
  • a feature synthesizing model 700 stored at an edge device may be periodically updated with trained weights, feature embeddings, and the like from the cloud network 602. Such updates should be one-way across the privacy boundary so that no data travels from the edge device to the cloud network.
  • FIG. 8 illustrates a flowchart of a de-identifying method 800 according to example embodiments of the present disclosure.
  • an end device captures data containing biometric features of one or more individual users.
  • Biometric features may be features which describe physical and/or behavioral characteristics as described above, some of which may be physical and/or behavioral identifiers. Not all biometric features need be identifying of individual users, though whether a particular biometric feature is identifying or not is merely a consequence of the stage of advancement of identifying technology. Biometric features which cannot be utilized by persons skilled in the art presently to identify individual users may be utilized to identify individual users in the near future as machine learning technology progresses. Therefore, the present disclosure shall not be limiting as to whether any particular biometric feature is an identifier or not.
  • Users of IoT services may, at some juncture, permit data containing their own biometric features to be labeled with their own identities and stored at a cloud computing system as described above with reference to a database of an identifying system. This may be part of a user registration process so as to establish and enrich a profile of a user of particular IoT services.
  • captured data may be labeled with user identities in an open and transparent fashion without replacement of features; however, beyond such instances of informed identity sharing, according to example embodiments of the present disclosure, users should be assured that the IoT service backend is not collecting captured data on a passive or ongoing basis which may be inspected to identify those users.
  • an edge device extracts features from the captured data.
  • an edge device may be an end device as mentioned in step 802, or may be an edge node of a cloud computing system to which captured data is transported.
  • the end device of step 802 may or may not be the edge device.
  • the edge device as described herein with reference to FIG. 8 may also be a component of the end device, such as a component of a sensor which captured the data (in which case the sensor itself may be part of a capture device which constitutes a computing system as described subsequently with reference to FIG. 9) . In either case, the captured data does not cross a conceptual privacy boundary as long as it remains on an edge device.
  • the edge device determines an identifying feature of the extracted features.
  • the edge device may determine an identifying feature of the extracted feature by executing an identifying system as described above with reference to FIG. 5 so as to determine which extracted features were matched against a database of the identifying system.
  • edge devices are not necessarily sufficiently powerful or possess sufficient storage to store or execute an identifying system.
  • the edge device may also determine an identifying feature of the extracted feature by receiving information from the cloud network of FIG. 6 with regard to a compact set of extracted features which are operative to enable an identifying system of FIG. 5 to identify individuals.
  • the edge device may learn that a particular feature is an identifying feature without actually identifying any individuals using the particular feature.
  • identifying systems may rely on different sets of features to identify individuals.
  • the identifying features determined by the edge device in step 806 may change.
  • the edge device generates a synthetic feature by a feature synthesizing model.
  • the synthetic feature may be a non-identifying feature or a generic feature, for example.
  • the edge device replaces the identifying feature of the captured data with the synthetic feature.
  • replacement may be performed differently.
  • replacement may include replacing some number of pixels of captured image data (such as those pixels delineated by regression vectors as described above with reference to FIG. 1. ) with pixels of the synthetic feature.
  • the captured data is a string
  • replacement may include replacing some number of characters of the captured data with a string of the synthetic feature.
  • the captured data is audio
  • replacement may include replacing some portion of the captured data over a period of time with audio data of the synthetic feature.
  • replacement may include replacing a quantified number with a quantification of the synthetic feature.
  • filters may further be applied around the replaced region to blend the replacement data with the original data.
  • the adulterated captured data may be freely transported across the privacy boundary. Prior to this, individual users may be given the opportunity to review the adulterated captured data on one or more end devices, so as to receive assurance that the adulterated captured data cannot be used to identify the individual users by human inspection. However, upon being transported to the cloud network, the adulterated captured data may still be utilized by an identifying model to identify individual users and contribute information to a user profile stored at the data center. The adulterated captured data may further be stored at the data center, lowering the chance that user identities can be compromised as a result of security breaches and theft of data.
  • FIG. 9 illustrates an example system 900 for implementing the processes and methods described above for implementing de-identification of captured data.
  • the techniques and mechanisms described herein may be implemented by multiple instances of the system 900, as well as by any other computing device, system, and/or environment.
  • the system 900 may be a single computing system or an edge host providing physical or virtual computing resources as known by persons skilled in the art. Examples thereof include edge devices such as end devices and edge nodes as described above with reference to FIG. 6; however, the system 900 should not encompass computing resources of the cloud network as described in FIG. 6.
  • the system 900 shown in FIG. 9 is only one example of a system and is not intended to suggest any limitation as to the scope of use or functionality of any computing device utilized to perform the processes and/or procedures described above.
  • the system 900 may include one or more processors 902 and system memory 904 communicatively coupled to the processor (s) 902.
  • the processor (s) 902 and system memory 904 may be physical or may be virtualized and/or distributed.
  • the processor (s) 902 may execute one or more modules and/or processes to cause the processor (s) 902 to perform a variety of functions.
  • the processor (s) 902 may include a central processing unit ( “CPU” ) , a graphics processing unit ( “GPU” ) , both CPU and GPU, or other processing units or components known in the art. Additionally, each of the processor (s) 902 may possess its own local memory, which also may store program modules, program data, and/or one or more operating systems.
  • system memory 904 may be volatile, such as RAM, non-volatile, such as ROM, flash memory, miniature hard drive, memory card, and the like, or some combination thereof.
  • the system memory 904 may include one or more computer-executable modules 906 that are executable by the processor (s) 902.
  • the modules 906 may include, but are not limited to, a data capturing module 908, a feature extracting module 910, an identifying feature determining module 912, a feature synthesizing module 914, and a feature replacing module 916.
  • the feature synthesizing module 914 may further include a generator submodule 918 and a discriminator submodule 920.
  • the data capturing module 908 may be configured to capture data containing biometric features of one or more individual users as described above with reference to FIG. 8.
  • the feature extracting module 910 may be configured to extract features from the captured data as described above with reference to FIG. 8.
  • the identifying feature determining module 912 may be configured to determine an identifying feature of the extracted features as described above with reference to FIG. 8.
  • the feature synthesizing module 914 may be configured to generate a synthetic feature by a feature synthesizing model as described above with reference to FIG. 8. Therein, the generator submodule 918 and the discriminator submodule 920 may be configured to perform the functions of the generator 704 and the discriminator 706 as described above with reference to FIG. 7.
  • the feature replacing module 916 may be configured to replace the identifying feature of the captured data with the synthetic feature as described above with reference to FIG. 8.
  • the system 900 may additionally include an input/output (I/O) interface 940 and a communication module 950 allowing the system 900 to communicate with other systems and devices over a network, such as the cloud network as described above with reference to FIG. 6.
  • the network may include the Internet, wired media such as a wired network or direct-wired connections, and wireless media such as acoustic, radio frequency ( “RF” ) , infrared, and other wireless media.
  • RF radio frequency
  • Computer-readable instructions include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like.
  • Computer-readable instructions can be implemented on various system configurations, including single-processor or multiprocessor systems, minicomputers, mainframe computers, personal computers, hand-held computing devices, microprocessor-based, programmable consumer electronics, combinations thereof, and the like.
  • the computer-readable storage media may include volatile memory (such as random-access memory ( “RAM” ) ) and/or non-volatile memory (such as read-only memory ( “ROM” ) , flash memory, etc. ) .
  • volatile memory such as random-access memory ( “RAM” )
  • non-volatile memory such as read-only memory ( “ROM” ) , flash memory, etc.
  • the computer-readable storage media may also include additional removable storage and/or non-removable storage including, but not limited to, flash memory, magnetic storage, optical storage, and/or tape storage that may provide non-volatile storage of computer-readable instructions, data structures, program modules, and the like.
  • a non-transient computer-readable storage medium is an example of computer-readable media.
  • Computer-readable media includes at least two types of computer-readable media, namely computer-readable storage media and communications media.
  • Computer-readable storage media includes volatile and non-volatile, removable and non-removable media implemented in any process or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data.
  • Computer-readable storage media includes, but is not limited to, phase change memory ( “PRAM” ) , static random-access memory ( “SRAM” ) , dynamic random-access memory ( “DRAM” ) , other types of random-access memory ( “RAM” ) , read-only memory ( “ROM” ) , electrically erasable programmable read-only memory ( “EEPROM” ) , flash memory or other memory technology, compact disk read-only memory ( “CD-ROM” ) , digital versatile disks ( “DVD” ) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device.
  • communication media may embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanism. As defined herein, computer-readable storage media do not include communication media.
  • the computer-readable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, may perform operations described above with reference to FIGS. 1-8.
  • computer-readable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types.
  • the order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes.
  • the present disclosure provides replacement of features extracted from captured data at edge devices before the captured data is collected at a cloud computing system.
  • some features may include biometric characteristics of individual users which are furthermore identifying characteristics.
  • a feature synthesizing model may be trained based on random, synthetic, and/or generic data to generate synthetic features which are not identifying of users.
  • the identifying features from the captured data may be replaced with these synthetic features before the captured data is transported across a privacy boundary to a cloud computing system for backend computations supporting IoT services by one or more learning models. In this manner, data may be collected and stored in a manner which avoids individual users being exposed by inspection, while still maintaining utility in allowing identifying models to function and user profiles to be updated.
  • a method comprising: capturing, by an end device, data containing biometric features of one or more individual users; extracting, by an edge device, a plurality of features from the captured data; determining, by the edge device, one or more identifying features of the extracted features; generating, by the edge device, one or more synthetic features by a feature synthesizing model; and replacing, by the edge device, the one or more identifying features of the captured data with the one or more synthetic features.
  • the feature synthesizing model comprises a generative adversarial network further comprising a generator and a discriminator.
  • a system comprising: one or more processors; and memory communicatively coupled to the one or more processors, the memory storing computer-executable modules executable by the one or more processors that, when executed by the one or more processors, perform associated operations, the computer-executable modules comprising: a data capturing device configured to capture biometric features of one or more individual users; a feature extracting module configured to extract a plurality of features from the captured data; an identifying feature determining module configured to determine one or more identifying features of the extracted features; a feature synthesizing module configured to generate one or more synthetic features by a feature synthesizing model; and a feature replacing module configured to replace the one or more identifying features of the captured data with the one or more synthetic features.
  • the feature synthesizing model comprises a generative adversarial network further comprising a generator and a discriminator.
  • a computer-readable storage medium storing computer-readable instructions executable by one or more processors, that when executed by the one or more processors, cause the one or more processors to perform operations comprising: capturing, by an end device, data containing biometric features of one or more individual users; extracting, by an edge device, features from the captured data; determining, by the edge device, an identifying feature of the extracted features; generating, by the edge device, a synthetic feature by a feature synthesizing model; and replacing, by the edge device, the identifying feature of the captured data with the synthetic feature.
  • the computer-readable storage medium as paragraph O recites, wherein the feature synthesizing model comprises a generative adversarial network further comprising a generator and a discriminator.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Human Computer Interaction (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioethics (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)
PCT/CN2020/097708 2020-06-23 2020-06-23 Edge processing data de-identification WO2021258284A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/097708 WO2021258284A1 (en) 2020-06-23 2020-06-23 Edge processing data de-identification
CN202080100159.4A CN115443466A (zh) 2020-06-23 2020-06-23 边缘处理数据去识别

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/097708 WO2021258284A1 (en) 2020-06-23 2020-06-23 Edge processing data de-identification

Publications (1)

Publication Number Publication Date
WO2021258284A1 true WO2021258284A1 (en) 2021-12-30

Family

ID=79282663

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/097708 WO2021258284A1 (en) 2020-06-23 2020-06-23 Edge processing data de-identification

Country Status (2)

Country Link
CN (1) CN115443466A (zh)
WO (1) WO2021258284A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116405635A (zh) * 2023-06-02 2023-07-07 山东正中信息技术股份有限公司 一种基于边缘计算的多模态会议记录方法及系统
CN117787349A (zh) * 2024-02-23 2024-03-29 天津普智芯网络测控技术有限公司 一种适用边缘设备的图片识别架构和设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130170700A1 (en) * 2011-12-30 2013-07-04 Altek Corporation Image Capturing Device Capable of Simplifying Characteristic Value Sets of Captured Images and Control Method Thereof
CN111274602A (zh) * 2020-01-15 2020-06-12 腾讯科技（深圳）有限公司 一种图像特征信息替换方法、装置、设备及介质
CN111310734A (zh) * 2020-03-19 2020-06-19 支付宝(杭州)信息技术有限公司 保护用户隐私的人脸识别方法和装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130170700A1 (en) * 2011-12-30 2013-07-04 Altek Corporation Image Capturing Device Capable of Simplifying Characteristic Value Sets of Captured Images and Control Method Thereof
CN111274602A (zh) * 2020-01-15 2020-06-12 腾讯科技（深圳）有限公司 一种图像特征信息替换方法、装置、设备及介质
CN111310734A (zh) * 2020-03-19 2020-06-19 支付宝(杭州)信息技术有限公司 保护用户隐私的人脸识别方法和装置

Also Published As

Publication number Publication date
CN115443466A (zh) 2022-12-06

Similar Documents

Publication Publication Date Title
Hernandez-Ortega et al. Deepfakeson-phys: Deepfakes detection based on heart rate estimation
Al-Allaf Review of face detection systems based artificial neural networks algorithms
KR101254177B1 (ko) 방사형 기저 함수 신경회로망 알고리즘을 이용한 실시간 얼굴 인식 시스템
US11281921B2 (en) Anti-spoofing
KR100608595B1 (ko) 얼굴 인식 방법 및 장치
López Deep learning in biometrics: a survey
US10733279B2 (en) Multiple-tiered facial recognition
Malgheet et al. Iris recognition development techniques: a comprehensive review
Parashar et al. Deep learning pipelines for recognition of gait biometrics with covariates: a comprehensive review
WO2021258284A1 (en) Edge processing data de-identification
Parashar et al. Intra-class variations with deep learning-based gait analysis: A comprehensive survey of covariates and methods
Wang et al. ScleraSegNet: An attention assisted U-Net model for accurate sclera segmentation
Hernandez-Ortega et al. Deepfakes detection based on heart rate estimation: Single-and multi-frame
Jia et al. Face spoofing detection under super-realistic 3D wax face attacks
Akhtar et al. Utility of deep learning features for facial attributes manipulation detection
Satapathy et al. A lite convolutional neural network built on permuted Xceptio-inception and Xceptio-reduction modules for texture based facial liveness recognition
Booysens et al. Ear biometrics using deep learning: A survey
Majeed et al. A novel method to enhance color spatial feature extraction using evolutionary time-frequency decomposition for presentation-attack detection
Jaswanth et al. Deep learning based intelligent system for robust face spoofing detection using texture feature measurement
Venkata Kranthi et al. Real-time facial recognition using deep learning and local binary patterns
Johnson et al. Multimodal biometric identification based on overlapped fingerprints, palm prints, and finger knuckles using BM-KMA and CS-RBFNN techniques in forensic applications
Chen Design and simulation of AI remote terminal user identity recognition system based on reinforcement learning
Mustafa et al. A comparative analysis for extracting facial features to detect deepfake videos by various machine learning methods
Shelke et al. Combining computer vision techniques and intraframe noise methods to detect a deepfake
Li et al. Residual-network-based supervised gaze prediction for first-person videos

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20942083

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20942083

Country of ref document: EP

Kind code of ref document: A1