CN106909905A - A kind of multi-modal face identification method based on deep learning - Google Patents
A kind of multi-modal face identification method based on deep learning Download PDFInfo
- Publication number
- CN106909905A CN106909905A CN201710122193.1A CN201710122193A CN106909905A CN 106909905 A CN106909905 A CN 106909905A CN 201710122193 A CN201710122193 A CN 201710122193A CN 106909905 A CN106909905 A CN 106909905A
- Authority
- CN
- China
- Prior art keywords
- mode
- modal
- similarity
- face
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of multi-modal face identification method based on deep learning, Face datection and alignment are carried out to RGB facial images including (1), human face data the collection S0, S1, S2 ... for making RGB mode and other mode are cut according to the mapping relations between mode；(2) a depth convolutional neural networks structure N1 for multi-modal fusion is designed, and trains N1 networks；(3) a multi-modal shared depth convolutional neural networks structure N2 is designed, and trains N2 networks；(4) feature stage, (5) Similarity Measure stage and (6) similarity fusing stage are extracted.The present invention uses multimodal systems, by the collection for carrying out various face modal datas, using the respective advantage of multiple modalities information, overcome by convergence strategy in some of single mode system in weakness, multiple modalities information is sufficiently utilized simultaneously, effectively improve the performance of face identification system so that recognition of face is more quick accurate.
Description
Technical field
The present invention relates to a kind of face identification method, more particularly to a kind of multi-modal recognition of face side based on deep learning
Method.
Background technology
Three-dimensional face identification has it to illumination robust, by factors such as attitude and expressions relative to two-dimension human face identification
The advantages of influence is smaller, therefore developed rapidly in three dimensional data collection technology and the quality and precision of three-dimensional data are greatly promoted
Afterwards, during their research is put into the field by many scholars.
The image of face different modalities is easily influenceed by different factors etc., and these factors have impact on to a certain extent
The stability and accuracy of single mode face identification system.CN104778441A proposes a kind of fusion half-tone information and depth letter
The multi-modal face identification device and method of breath, its core methed are the (institutes in invention after extracting multi-modal face characteristic
The feature for using is the feature of hand-designed), it is stitched together to form a feature pool, it is characterized each feature construction one in pond
Individual Weak Classifier, then using Adaboost algorithm, picks out for the maximally efficient feature, last base of classifying in feature pool
In the feature that multi-modal Feature-level fusion is obtained, matching fraction is calculated using nearest neighbor classifier, multi-modal people is realized with this
Face is recognized.But the feature that the invention is used is the feature of engineer, ability to express is not strong enough；And the feature of the invention
Fusion Features and feature selecting are carried out using Adaboost algorithm, it is less efficient；And the invention is for specific both modalities which is designed
, with limitation.
The content of the invention
For the weak point in solving the above problems, the invention provides a kind of multi-modal face based on deep learning
Recognition methods.
In order to solve the above technical problems, the technical solution adopted by the present invention is：It is a kind of based on the multi-modal of deep learning
Face identification method, comprises the following steps：
(1) Face datection is carried out to RGB facial images, positioning feature point, alignment cuts, and makes the RGB mode after cutting
Human face data collection S0；According to the coordinate mapping relations between RGB mode and other mode, the feature of other mode faces is found
Point, and cut human face data the collection S1, S2 ... for making other mode；
(2) a depth convolutional neural networks structure N1 for multi-modal fusion is designed, in this structure, first half is
Several independent neutral net branches, input one mode of correspondence of each branch, then with specific network structure multiple
Bimodal branch is fused into a branch, reconnects a series of neural network structure units；Then by S0, S1, S2 ... are sent to
In the different branches of N1, N1 networks, the model for training is trained to be represented with M1；
Above-mentioned neural network structure unit includes but is not limited to convolutional layer, normalization layer, non-linear layer, pond layer and connects entirely
Connect layer and distribution normalization layer；Mode includes but is not limited to RGB mode, depth mode and near-infrared mode；Specific network knot
In structure, there is respective Classification Loss in each branch as the respective supervisory signals of each mode, and structure fusion method is included but not
It is limited to simple merging features；
(3) a multi-modal shared depth convolutional neural networks structure N2 is designed, by S0, S1, S2 ... are not added with distinguishing
The N2 of feeding together in, train N2 networks, the model for training represents with M2；
(4) feature stage is extracted, for registered set and the image of query set, its modality range is in training set modality range
It is interior.The different modalities of certain image can be expressed as I0, I1, I2 ..., and then I0, I1, I2 ... are respectively in model M 1 and M2
Upper extraction feature, feature can use F0, F0C, F1, F1C, F2, F2C ... to represent, it is the feature extracted from M2 that C is represented；
(5) the similarity S11's, GF2 and PF2 of similarity S00, GF1 and PF1 respectively between calculating GF0 and PF0 is similar
Degree S22；The similarity S01 between GF0C and PF1C is calculated, similar calculates cross-module state similarity S02, S03, S12, S13,
S23……；
Above-mentioned GF0 represents that the F0 of registered set image, PF0 represent that the F0 of query set image, GF0C represent registered set image
F0C, PF1C represent the F1C of query set image；
(6) summation fusion is weighted to all of registered set and query set similarity, final fusion similarity is obtained
S, recognition of face and confirming face are carried out on the similarity matrix that fusion similarity S is constituted.
In step (2), when training N1 networks, loss layer using softmax with loss or can use it
His loss layer.
In step (3), by S0, S1, S2 ... be not added with distinguish feeding N2 in when, if their port number is not
Together, then can take and all be changed into single pass mode and be normalized to same channels, or to repeat single channel identical to port number,
Then it is re-fed into being trained in network structure.
In step (4), for registered set and the image of query set, its modality range is within training set modality range.
The present invention uses multimodal systems, by carrying out the collection of various face modal datas, using multiple modalities information
Respective advantage, is overcome in some of single mode system in weakness by convergence strategy, while sufficiently utilizing multiple modalities
Information, effectively improves the performance of face identification system so that recognition of face is more quick accurate.
Brief description of the drawings
The present invention is further detailed explanation with reference to the accompanying drawings and detailed description.
Fig. 1 is the flow chart of face recognition algorithms of the present invention.
Fig. 2 is the structured flowchart of the depth convolutional neural networks of multi-modal fusion of the present invention.
Fig. 3 is the concrete structure block diagram of Loss4 in Fig. 2.
Specific embodiment
As shown in figure 1, the present invention specifically includes following steps：
(1) Face datection is carried out to RGB facial images, positioning feature point, alignment cuts, and makes the RGB mode after cutting
Human face data collection S0；Then according to the coordinate mapping between RGB mode and other mode (such as depth information, Near Infrared Information etc.)
Relation, finds the characteristic point of other mode faces, and cuts human face data the collection S1, S2 ... for making other mode；
(2) a deep neural network structure N1 for multi-modal fusion is designed, in this structure, first half is several
Independent neutral net branch, one mode of input correspondence (such as the RGB mode, depth mode, near-infrared mode of each branch
Deng), multiple bimodal branch are then permeated a neutral net branch of synthesis (such as these with specific network structure
Feature is coupled together, or is stacked up by passage, or other attachment structures, such as structure etc. as accompanying drawing 3), reconnect
A series of neural network structure units (such as convolutional layer, normalization layer, non-linear layer, pond layer, full articulamentum)；Then will
S0, S1, S2 ... are sent in the different branches of N1, train N1 networks, the model for training to be represented with M1；
(3) a multi-modal shared deep neural network structure N2 is designed, by S0, S1, S2 ... are not added with for distinguishing
Rise in feeding N2, train N2 networks, the model for training to be represented with M2；
(4) feature stage is extracted, for registered set and the image of query set, its modality range is in training set modality range
Interior, the different modalities of certain image can be expressed as I0, I1, I2 ..., and then I0, I1, I2 ... are respectively in model M 1 and M2
Upper extraction feature, feature can use F0, F0C, F1, F1C, F2, F2C ... to represent, it is the feature extracted from M2 that C is represented；
(5) similarity between GF0 and PF0 (representing the F0 of the registered set image and F0 of query set image respectively) is calculated
The similarity S22 of the similarity S11, GF2 and PF2 of S00, GF1 and PF1；Calculate GF0C and PF1C (represents registered set image respectively
F0C and query set image F1C) between similarity S01, similar calculates cross-module state similarity S02, S03, S12,
S13, S23 ...；
(6) similarity to all of registered set and query set is weighted summation fusion, obtains final fusion similar
Degree S, recognition of face and confirming face are carried out on the similarity matrix that fusion similarity S is constituted.
The formula of above-mentioned weighted sum fusion can be expressed as：S=p1*S1+p2*S2+p3*S3 ..., wherein p1:p2:
P3 can be S1, and S2, S3 individually carry out the inverse ratio (1/r1 of the recognition correct rate (r1, r2, r3) that face recognition experiment is obtained:1/
r2:1/r3)。
In deep neural network structure N1 in step (2), network structure elements can include but is not limited to convolutional layer,
Pond layer, nonlinear function layer, full articulamentum, distribution normalization layer etc., and include but is not limited to any combination of these layers；
The simple combination of network structure elements brand-new design, or can be changed based on existing open network structure,
The simple combination of network structure elements is not in itself the scope of protection, as long as meet the present invention being described, first half is
Several independent neutral net branches, a branch for the individual synthesis that permeated by ad hoc structure, and each mode in ad hoc structure
There are respective independent supervisory signals in branch, connects a series of structure shape of neural network structure units after the branch after synthesis again
Formula, then belong to the scope of protection.
One is given as shown in Figure 2 based on Google Inception-v2 (a kind of depth network structure that Google is proposed)
The network structure come according to above-mentioned rules modification, in the network structure, if removing Loss4 loss structures, removes depth mould
State and near-infrared bimodal branch, a remaining RGB branch is directly connected to 3*Inception structures, and (* represents multiple similar structure strings
Connection, is 3 Inception structures in series such as herein, and Inception structures are a sub-network structure of Google's definition,
[2] have the expression to Inception structures in the Figure 3 in) on, then completely it is exactly Google Inception-v2 network knots
In itself, in the Figure 5 of [1], the 2* (convolution+pond) of accompanying drawing 2 corresponds to the convolution (convolutional layer) of Figure 5 to structure,
Max pool (maximum pond layer), this four layers of convolution (convolutional layer), max pool (maximum pond layer), and from it is lower to
On first 3*Inception then correspond to [1] in inception (3a), inception (3b), inception (3c),
4*Inception thereon then corresponds to inception (4a), inception (4b), inception (4c) in [1],
Inception (4d), then 3*Inception thereon then corresponds to inception (4e), the inception (5a) in [1],
Inception (5b), Loss3 ponds layer then corresponds to the avg pool (average pond layer) in [1].
In our network structure, we first allow the data (RGB, depth information, Near Infrared Information etc.) of different modalities
By a series of neural network structures, (each respective supervisory signals of mode can in fusion structure for study to the distinctive feature of mode
To ensure and promote the study of mode characteristic feature), then fusion connection is got up by a series of neural network structures, to help
Network carries out the complementary feature learning of mode, and can be used when fusion connection will in the way of passage (channel) is stacked
Characteristic pattern is stacked up.For Loss4 structures, it is the structure of particular design, except that can promote network convergence, also may be used
To promote e-learning mode complementary characteristic, a kind of structure of Loss4 is shown in accompanying drawing 3, three branches correspond to three respectively
Mode, picks out from three branches of accompanying drawing 2, and FC is to represent full articulamentum in structure, after FC or under numeral be to represent
The node quantity of the full articulamentum；It is 5*5 that average pond Hua Ceng pondizations are interval, and interval steps are 3；Convolutional layer convolution kernel size is 1*
1, interval steps are 1；The total number of persons of training set, is the categorical measure of label (label)；+ expression is input to three 512 therein
The full articulamentum of node is averaging (fc3 [i]=(fcc3 [i]+fcd3 [i]+fcn3 [i])/3, i=1,2,3 ... by node
512), it is clear that obtain an or full articulamentum for 512 nodes, namely fc3；FC2048 is three the 512 of mode and connects entirely
Connect layer and three modality fusions 512 full articulamentums be connected in series after the full articulamentums of 2048 dimensions that obtain；
When present invention training N1 networks, loss layer can use softmax with loss (classical Classification Loss
Layer), it is also possible to use unknown losses layer.The used algorithm of training is that Back Propagation reversely return algorithm, is passed through
Return the error of loss layer to update each layer of parameter so that network parameter is updated, and finally gives convergence.Its is specific
Training step, as a example by the data set that is used during with present invention experiment (non-public, to be gathered with partner cooperation), the data set
Training set scale be for about 500,000 samples, about 500 people.Use 32 batch (data block) scale, basic learning rate
0.045, every 6400 iterative learning rate is multiplied by 0.9, weight decay (weights decay), and for 0.0002, momentum, (momentum is joined
Number) it is 0.9. about 400,000 iteration of training.
The present invention in deep neural network structure N2, likewise, network structure elements can include but is not limited to convolution
Layer, pond layer, nonlinear function layer, full articulamentum, distribution normalization layer etc., and include but is not limited to any group of these layers
Close；In this step, network structure is not the scope of protection, trains multi-modal public characteristic empty by way of not differentiating between mode
Between method be protection scope, network structure can be with brand-new design, it is also possible to using network structure disclosed in academia such as
The networks such as AlexNet, GoogleNet, ResNet.By S0, S1, S2 etc. be not added with distinguish the N2 of feeding together in when, if
Their port number is different, then can take all be changed into single pass mode to be normalized to same channels (such as RGB triple channels change
It is gray-scale map single channel), or to repeat single channel identical (such as repeat near-infrared single channel figure three times to triple channel) to port number,
Then it is re-fed into being trained in network structure.
The present embodiment is with RGB, depth information, as a example by Near Infrared Information, because depth information and Near Infrared Information are list
The RGB figures of all samples, can be switched to single channel gray-scale map by passage, be re-fed into being trained in N2 networks.With Google
As a example by Inception-v2 [1], by all samples of 500,000 training sets regardless of mode regard as 1,500,000 samples (RGB,
Depth, three kinds of mode of near-infrared, wherein RGB switchs to gray-scale map herein), study is trained in input Inception-v2, make
With 32 batch block sizes, basic learning rate 0.045, every 19200 iterative learning rates are multiplied by 0.9, weight decay
(weights decay) is that 0.0002, momentum (momentum parameter) is 0.9, trains about 1,200,000 iteration.The model for obtaining
It is set to M2.Then for the feature of model, it is believed that be the feature of cross-module state, the RGB of sample (having switched to a gray-scale map) feature
Can be compared with its depth information feature, the RGB feature of query set can carry out similarity-rough set with the depth information feature of registered set,
This is the rudimentary algorithm of cross-module state identification division of the present invention.
Feature stage is being extracted, can be training set modality range for the modality range of registered set and query set image
Complete or collected works, or proper subclass, but can not be beyond training set modality range；The position for extracting feature can be in the He of model M 1
The top layer of M2, it is also possible at non-top layer (such as middle a certain layer).
In above-mentioned step (5), the distance metric method for calculating similarity can be COS distance, or European
Distance, or other distance metrics such as mahalanobis distance.The present invention by taking COS distance as an example, for two characteristic vectors x1, x2, its
COS distance d=x1 ' * x2/ (| x1 | * | x2 |).Where it is assumed that x1, x2 are column vector, x1 ' is the transposition of x1, and x1 ' * x2 are
The dot product of x1 ' and x2, | x1 |, | x2 | are x1, and the mould of x2 is long, wherein mould satisfaction | x1 | 2=x1 ' * x1 long.
Face recognition experiment of the present invention refers to giving a registered set, and registered set has a series of different face figures
Picture, when a query image is given, which individual that the query image is registered set is found, and method can think registration
Collection image and that people of query image similarity highest；Confirming face experiment refers to given two query images, determines this
Whether two images are same persons, and method can be to determine a threshold value, if two image similarities are higher than threshold value, then it is assumed that
It is same person, otherwise it is assumed that not being same person.
The present invention has several key technology points：1) multi-modal fusion is carried out using network structure, fusion position is in network
Middle part, neither bottom input position also non-top layer loss layer position, the mode quantity of fusion can be two, or many
It is individual, for each mode provides single supervisory signals in fusion structure, to allow each mode feature of itself to be filled
Divide and excavate and be unlikely to be fallen into oblivion by other modal characteristics.Therefore the most abundant Generalization Capability for recognition of face can be obtained
Feature representation and syncretizing effect, and be not limited to two mode, be adapted to multiple mode, more flexibly.2) for across
The mode of learning in mode public characteristic space, the face image data of multiple modalities is not added with differentiation is added to depth as input
It is trained in degree neutral net, mode quantity can be two, or multiple, and the label of image is the label of people, no
Mode is distinguished, the public characteristic space that training is obtained can choose the top layer of network, it is also possible to select the non-top layer of network；
Technique effect can be the recognition effect of cross-module state between the various face mode of acquisition.3) it is similar to all multi-modal mode inside
Similarity is weighted the mode of summation fusion between degree and cross-module morphotype state, obtains the multi-modal fusion phase of final two images
Like spend, for test image mode can completely can also be imperfect, its modality type set can be training set matched moulds
The complete or collected works of state type set can also be proper subclass；Technique effect be so that multi-modal recognition of face obtain very strong flexibility and
Applicability, goes for many scenes (as only registered set is multi-modal, query image single mode etc.) of multi-modal recognition of face.
Therefore, the present invention is excavated and study face by designing multi-modal and cross-module state deep learning network, sufficiently
Complementary characteristic between multi-modal, greatly improves the performance of recognition of face, and there is provided very strong multi-modal and cross-module state
The flexibility of face recognition application and applicability.
Above-mentioned implementation method is not limitation of the present invention, and the present invention is also not limited to the example above, this technology neck
Change, remodeling, addition or replacement that the technical staff in domain is made in the range of technical scheme, also belong to this hair
Bright protection domain.
Claims (5)
1. a kind of multi-modal face identification method based on deep learning, it is characterised in that comprise the following steps：
(1) Face datection is carried out to RGB facial images, positioning feature point, alignment cuts, and makes the RGB mode faces after cutting
Data set S0；According to the coordinate mapping relations between RGB mode and other mode, the characteristic point of other mode faces is found, and
Cut human face data the collection S1, S2 ... for making other mode；
(2) a depth convolutional neural networks structure N1 for multi-modal fusion is designed, in this structure, first half is several
Independent neutral net branch, input one mode of correspondence of each branch, then with specific network structure multiple mode
Branch is fused into a branch, reconnects a series of neural network structure units；Then by S0, S1, S2 ... are sent to N1's
In respective branches, N1 networks, the model for training is trained to be represented with M1；
Above-mentioned mode includes but is not limited to RGB mode, depth mode and near-infrared mode；In specific network structure, each point
Branch has respective Classification Loss as the respective supervisory signals of each mode, and structure fusion method includes but is not limited to simple spy
Levy splicing；
(3) a multi-modal shared depth convolutional neural networks structure N2 is designed, by S0, S1, S2 ... are not added with for distinguishing
Rise in feeding N2, train N2 networks, the model for training to be represented with M2；
(4) extract feature stage, for registered set and the image of query set, its modality range in training set modality range, certain
The different modalities of image can be expressed as I0, I1, I2 ..., and then I0, I1, I2 ... are extracted in model M 1 and M2 respectively
Feature, feature can use F0, F0C, F1, F1C, F2, F2C ... to represent, it is the feature extracted from M2 that C is represented；
(5) similarity of the similarity S11, GF2 and PF2 of similarity S00, GF1 and PF1 respectively between calculating GF0 and PF0
S22；The similarity S01 between GF0C and PF1C is calculated, similar calculates cross-module state similarity S02, S03, S12, S13,
S23……；
Above-mentioned GF0 represents that the F0 of registered set image, PF0 represent that the F0 of query set image, GF0C represent the F0C of registered set image,
PF1C represents the F1C of query set image；
(6) summation fusion is weighted to all of registered set and query set similarity, final fusion similarity S is obtained,
Recognition of face and confirming face are carried out on the similarity matrix that fusion similarity S is constituted.
2. the multi-modal face identification method based on deep learning according to claim 1, it is characterised in that：The nerve
Network structure elements include but is not limited to convolutional layer, normalization layer, non-linear layer, pond layer and full articulamentum and distribution normalization
Layer.
3. the multi-modal face identification method based on deep learning according to claim 1, it is characterised in that：The step
(2) in, when training N1 networks, loss layer can use softmax withloss or use unknown losses layer.
4. the multi-modal face identification method based on deep learning according to claim 1, it is characterised in that：The step
(3) in, S0 when S1, S2 ... are not added with the feeding N2 for distinguishing, if their port number is different, can take
All it is changed into single pass mode and is normalized to same channels, or to repeat single channel identical to port number, is then re-fed into net
It is trained in network structure.
5. the multi-modal face identification method based on deep learning according to claim 1, it is characterised in that：The step
(4) in, for registered set and the image of query set, its modality range is within training set modality range.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710122193.1A CN106909905B (en) | 2017-03-02 | 2017-03-02 | Multi-mode face recognition method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710122193.1A CN106909905B (en) | 2017-03-02 | 2017-03-02 | Multi-mode face recognition method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106909905A true CN106909905A (en) | 2017-06-30 |
| CN106909905B CN106909905B (en) | 2020-02-14 |
Family
ID=59186347
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710122193.1A Active CN106909905B (en) | 2017-03-02 | 2017-03-02 | Multi-mode face recognition method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106909905B (en) |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107273872A (en) * | 2017-07-13 | 2017-10-20 | 北京大学深圳研究生院 | The depth discrimination net model methodology recognized again for pedestrian in image or video |
| CN107423701A (en) * | 2017-07-17 | 2017-12-01 | 北京智慧眼科技股份有限公司 | The non-supervisory feature learning method and device of face based on production confrontation network |
| CN107480206A (en) * | 2017-07-25 | 2017-12-15 | 杭州电子科技大学 | A kind of picture material answering method based on multi-modal low-rank bilinearity pond |
| CN107506822A (en) * | 2017-07-26 | 2017-12-22 | 天津大学 | A kind of deep neural network method based on Space integration pond |
| CN107844744A (en) * | 2017-10-09 | 2018-03-27 | 平安科技（深圳）有限公司 | With reference to the face identification method, device and storage medium of depth information |
| CN108197587A (en) * | 2018-01-18 | 2018-06-22 | 中科视拓（北京）科技有限公司 | A kind of method that multi-modal recognition of face is carried out by face depth prediction |
| CN108229497A (en) * | 2017-07-28 | 2018-06-29 | 北京市商汤科技开发有限公司 | Image processing method, device, storage medium, computer program and electronic equipment |
| CN108229440A (en) * | 2018-02-06 | 2018-06-29 | 北京奥开信息科技有限公司 | One kind is based on Multi-sensor Fusion indoor human body gesture recognition method |
| CN108875777A (en) * | 2018-05-03 | 2018-11-23 | 浙江大学 | Kinds of fibers and blending rate recognition methods in textile fabric based on two-way neural network |
| CN108960337A (en) * | 2018-07-18 | 2018-12-07 | 浙江大学 | A kind of multi-modal complicated activity recognition method based on deep learning model |
| CN109284597A (en) * | 2018-11-22 | 2019-01-29 | 北京旷视科技有限公司 | A kind of face unlocking method, device, electronic equipment and computer-readable medium |
| CN109299639A (en) * | 2017-07-25 | 2019-02-01 | 虹软(杭州)多媒体信息技术有限公司 | A kind of method and apparatus for Expression Recognition |
| CN109583387A (en) * | 2018-11-30 | 2019-04-05 | 龙马智芯（珠海横琴）科技有限公司 | Identity identifying method and device |
| CN109815965A (en) * | 2019-02-13 | 2019-05-28 | 腾讯科技（深圳）有限公司 | A kind of image filtering method, device and storage medium |
| CN109934198A (en) * | 2019-03-22 | 2019-06-25 | 北京市商汤科技开发有限公司 | Face identification method and device |
| CN110009003A (en) * | 2019-03-14 | 2019-07-12 | 北京旷视科技有限公司 | Training method, the device and system of image procossing and image comparison model |
| CN110046551A (en) * | 2019-03-18 | 2019-07-23 | 中国科学院深圳先进技术研究院 | A kind of generation method and equipment of human face recognition model |
| CN110110120A (en) * | 2018-06-11 | 2019-08-09 | 北方工业大学 | A kind of image search method and device based on deep learning |
| WO2019169942A1 (en) * | 2018-03-09 | 2019-09-12 | 华南理工大学 | Anti-angle and occlusion interference fast face recognition method |
| CN110458828A (en) * | 2019-08-12 | 2019-11-15 | 广东工业大学 | A kind of laser welding defect identification method and device based on multi-modal fusion network |
| CN110674677A (en) * | 2019-08-06 | 2020-01-10 | 厦门大学 | Multi-mode multi-layer fusion deep neural network for anti-spoofing of human face |
| CN111316290A (en) * | 2017-11-03 | 2020-06-19 | 通用电气公司 | System and method for interactive representation learning migration through deep learning of feature ontologies |
| CN111369567A (en) * | 2018-12-26 | 2020-07-03 | 腾讯科技（深圳）有限公司 | Method and device for segmenting target object in three-dimensional image and electronic equipment |
| CN111401107A (en) * | 2019-01-02 | 2020-07-10 | 上海大学 | Multi-mode face recognition method based on feature fusion neural network |
| CN111611909A (en) * | 2020-05-18 | 2020-09-01 | 桂林电子科技大学 | Multi-subspace-domain self-adaptive face recognition method |
| CN111881706A (en) * | 2019-11-27 | 2020-11-03 | 马上消费金融股份有限公司 | Living body detection, image classification and model training method, device, equipment and medium |
| CN111937005A (en) * | 2020-06-30 | 2020-11-13 | 北京小米移动软件有限公司 | Biological feature recognition method, device, equipment and storage medium |
| CN112016524A (en) * | 2020-09-25 | 2020-12-01 | 北京百度网讯科技有限公司 | Model training method, face recognition device, face recognition equipment and medium |
| CN112149635A (en) * | 2020-10-23 | 2020-12-29 | 北京百度网讯科技有限公司 | Cross-modal face recognition model training method, device, equipment and storage medium |
| CN112464741A (en) * | 2020-11-05 | 2021-03-09 | 马上消费金融股份有限公司 | Face classification method, model training method, electronic device and storage medium |
| CN112926557A (en) * | 2021-05-11 | 2021-06-08 | 北京的卢深视科技有限公司 | Method for training multi-mode face recognition model and multi-mode face recognition method |
| CN113378984A (en) * | 2021-07-05 | 2021-09-10 | 国药(武汉)医学实验室有限公司 | Medical image classification method, system, terminal and storage medium |
| CN113903053A (en) * | 2021-09-26 | 2022-01-07 | 厦门大学 | Cross-modal pedestrian re-identification method based on unified intermediate modality |
| CN114627431A (en) * | 2022-02-22 | 2022-06-14 | 安徽新识智能科技有限公司 | Intelligent environment monitoring method and system based on Internet of things |
| CN114724041A (en) * | 2022-06-02 | 2022-07-08 | 浙江天铂云科光电股份有限公司 | Power equipment infrared chart identification method and system based on deep learning |
| CN118552794A (en) * | 2024-07-25 | 2024-08-27 | 湖南军芃科技股份有限公司 | Ore sorting identification method based on multichannel training and ore sorting machine |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530648A (en) * | 2013-10-14 | 2014-01-22 | 四川空港知觉科技有限公司 | Face recognition method based on multi-frame images |
| CN104966093A (en) * | 2015-05-25 | 2015-10-07 | 苏州珂锐铁电气科技有限公司 | Dynamic texture identification method based on deep neural networks |
| CN105913025A (en) * | 2016-04-12 | 2016-08-31 | 湖北工业大学 | Deep learning face identification method based on multiple-characteristic fusion |
| CN106339702A (en) * | 2016-11-03 | 2017-01-18 | 北京星宇联合投资管理有限公司 | Multi-feature fusion based face identification method |
| CN106446754A (en) * | 2015-08-11 | 2017-02-22 | 阿里巴巴集团控股有限公司 | Image identification method, metric learning method, image source identification method and devices |
-
2017
- 2017-03-02 CN CN201710122193.1A patent/CN106909905B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530648A (en) * | 2013-10-14 | 2014-01-22 | 四川空港知觉科技有限公司 | Face recognition method based on multi-frame images |
| CN104966093A (en) * | 2015-05-25 | 2015-10-07 | 苏州珂锐铁电气科技有限公司 | Dynamic texture identification method based on deep neural networks |
| CN106446754A (en) * | 2015-08-11 | 2017-02-22 | 阿里巴巴集团控股有限公司 | Image identification method, metric learning method, image source identification method and devices |
| CN105913025A (en) * | 2016-04-12 | 2016-08-31 | 湖北工业大学 | Deep learning face identification method based on multiple-characteristic fusion |
| CN106339702A (en) * | 2016-11-03 | 2017-01-18 | 北京星宇联合投资管理有限公司 | Multi-feature fusion based face identification method |
Non-Patent Citations (2)
| Title |
|---|
| MENGYI LIU 等: "Exploiting Feature Hierarchies with Convolutional Neural Networks for Cultural Event Recognition", 《2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS》 * |
| SHIQING ZHANG 等: "Multimodal Deep Convolutional Neural Network for Multimodal Deep Convolutional Neural Network for Audio-Visual Emotion Recognition", 《ICMR"16》 * |
Cited By (58)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107273872B (en) * | 2017-07-13 | 2020-05-05 | 北京大学深圳研究生院 | Depth discrimination network model method for re-identification of pedestrians in image or video |
| CN107273872A (en) * | 2017-07-13 | 2017-10-20 | 北京大学深圳研究生院 | The depth discrimination net model methodology recognized again for pedestrian in image or video |
| WO2019010950A1 (en) * | 2017-07-13 | 2019-01-17 | 北京大学深圳研究生院 | Depth discrimination network model method for pedestrian re-recognition in image or video |
| CN107423701A (en) * | 2017-07-17 | 2017-12-01 | 北京智慧眼科技股份有限公司 | The non-supervisory feature learning method and device of face based on production confrontation network |
| CN107480206A (en) * | 2017-07-25 | 2017-12-15 | 杭州电子科技大学 | A kind of picture material answering method based on multi-modal low-rank bilinearity pond |
| CN109299639A (en) * | 2017-07-25 | 2019-02-01 | 虹软(杭州)多媒体信息技术有限公司 | A kind of method and apparatus for Expression Recognition |
| CN112861760A (en) * | 2017-07-25 | 2021-05-28 | 虹软科技股份有限公司 | Method and device for facial expression recognition |
| US11023715B2 (en) * | 2017-07-25 | 2021-06-01 | Arcsoft Corporation Limited | Method and apparatus for expression recognition |
| CN107480206B (en) * | 2017-07-25 | 2020-06-12 | 杭州电子科技大学 | Multi-mode low-rank bilinear pooling-based image content question-answering method |
| CN107506822A (en) * | 2017-07-26 | 2017-12-22 | 天津大学 | A kind of deep neural network method based on Space integration pond |
| CN107506822B (en) * | 2017-07-26 | 2021-02-19 | 天津大学 | Deep neural network method based on space fusion pooling |
| CN108229497A (en) * | 2017-07-28 | 2018-06-29 | 北京市商汤科技开发有限公司 | Image processing method, device, storage medium, computer program and electronic equipment |
| CN107844744A (en) * | 2017-10-09 | 2018-03-27 | 平安科技（深圳）有限公司 | With reference to the face identification method, device and storage medium of depth information |
| CN111316290B (en) * | 2017-11-03 | 2024-01-12 | 通用电气公司 | System and method for interactive representation learning migration through deep learning of feature ontologies |
| CN111316290A (en) * | 2017-11-03 | 2020-06-19 | 通用电气公司 | System and method for interactive representation learning migration through deep learning of feature ontologies |
| CN108197587B (en) * | 2018-01-18 | 2021-08-03 | 中科视拓（北京）科技有限公司 | Method for performing multi-mode face recognition through face depth prediction |
| CN108197587A (en) * | 2018-01-18 | 2018-06-22 | 中科视拓（北京）科技有限公司 | A kind of method that multi-modal recognition of face is carried out by face depth prediction |
| CN108229440A (en) * | 2018-02-06 | 2018-06-29 | 北京奥开信息科技有限公司 | One kind is based on Multi-sensor Fusion indoor human body gesture recognition method |
| US11417147B2 (en) | 2018-03-09 | 2022-08-16 | South China University Of Technology | Angle interference resistant and occlusion interference resistant fast face recognition method |
| WO2019169942A1 (en) * | 2018-03-09 | 2019-09-12 | 华南理工大学 | Anti-angle and occlusion interference fast face recognition method |
| CN108875777B (en) * | 2018-05-03 | 2022-03-15 | 浙江大学 | Method for identifying fiber types and blending proportion in textile fabric based on double-path neural network |
| CN108875777A (en) * | 2018-05-03 | 2018-11-23 | 浙江大学 | Kinds of fibers and blending rate recognition methods in textile fabric based on two-way neural network |
| CN110110120B (en) * | 2018-06-11 | 2021-05-25 | 北方工业大学 | Image retrieval method and device based on deep learning |
| CN110110120A (en) * | 2018-06-11 | 2019-08-09 | 北方工业大学 | A kind of image search method and device based on deep learning |
| CN108960337A (en) * | 2018-07-18 | 2018-12-07 | 浙江大学 | A kind of multi-modal complicated activity recognition method based on deep learning model |
| CN108960337B (en) * | 2018-07-18 | 2020-07-17 | 浙江大学 | Multi-modal complex activity recognition method based on deep learning model |
| CN109284597A (en) * | 2018-11-22 | 2019-01-29 | 北京旷视科技有限公司 | A kind of face unlocking method, device, electronic equipment and computer-readable medium |
| CN109583387A (en) * | 2018-11-30 | 2019-04-05 | 龙马智芯（珠海横琴）科技有限公司 | Identity identifying method and device |
| CN111369567A (en) * | 2018-12-26 | 2020-07-03 | 腾讯科技（深圳）有限公司 | Method and device for segmenting target object in three-dimensional image and electronic equipment |
| CN111401107A (en) * | 2019-01-02 | 2020-07-10 | 上海大学 | Multi-mode face recognition method based on feature fusion neural network |
| CN111401107B (en) * | 2019-01-02 | 2023-08-18 | 上海大学 | Multi-mode face recognition method based on feature fusion neural network |
| CN109815965B (en) * | 2019-02-13 | 2021-07-06 | 腾讯科技（深圳）有限公司 | Image filtering method and device and storage medium |
| CN109815965A (en) * | 2019-02-13 | 2019-05-28 | 腾讯科技（深圳）有限公司 | A kind of image filtering method, device and storage medium |
| CN110009003A (en) * | 2019-03-14 | 2019-07-12 | 北京旷视科技有限公司 | Training method, the device and system of image procossing and image comparison model |
| CN110046551A (en) * | 2019-03-18 | 2019-07-23 | 中国科学院深圳先进技术研究院 | A kind of generation method and equipment of human face recognition model |
| WO2020192112A1 (en) * | 2019-03-22 | 2020-10-01 | 北京市商汤科技开发有限公司 | Facial recognition method and apparatus |
| CN109934198A (en) * | 2019-03-22 | 2019-06-25 | 北京市商汤科技开发有限公司 | Face identification method and device |
| CN109934198B (en) * | 2019-03-22 | 2021-05-14 | 北京市商汤科技开发有限公司 | Face recognition method and device |
| CN110674677A (en) * | 2019-08-06 | 2020-01-10 | 厦门大学 | Multi-mode multi-layer fusion deep neural network for anti-spoofing of human face |
| CN110458828B (en) * | 2019-08-12 | 2023-02-10 | 广东工业大学 | Laser welding defect identification method and device based on multi-mode fusion network |
| CN110458828A (en) * | 2019-08-12 | 2019-11-15 | 广东工业大学 | A kind of laser welding defect identification method and device based on multi-modal fusion network |
| CN111881706A (en) * | 2019-11-27 | 2020-11-03 | 马上消费金融股份有限公司 | Living body detection, image classification and model training method, device, equipment and medium |
| CN111611909A (en) * | 2020-05-18 | 2020-09-01 | 桂林电子科技大学 | Multi-subspace-domain self-adaptive face recognition method |
| WO2022000334A1 (en) * | 2020-06-30 | 2022-01-06 | 北京小米移动软件有限公司 | Biological feature recognition method and apparatus, and device and storage medium |
| CN111937005A (en) * | 2020-06-30 | 2020-11-13 | 北京小米移动软件有限公司 | Biological feature recognition method, device, equipment and storage medium |
| CN112016524A (en) * | 2020-09-25 | 2020-12-01 | 北京百度网讯科技有限公司 | Model training method, face recognition device, face recognition equipment and medium |
| CN112016524B (en) * | 2020-09-25 | 2023-08-08 | 北京百度网讯科技有限公司 | Model training method, face recognition device, equipment and medium |
| CN112149635A (en) * | 2020-10-23 | 2020-12-29 | 北京百度网讯科技有限公司 | Cross-modal face recognition model training method, device, equipment and storage medium |
| CN112464741B (en) * | 2020-11-05 | 2021-11-26 | 马上消费金融股份有限公司 | Face classification method, model training method, electronic device and storage medium |
| CN112464741A (en) * | 2020-11-05 | 2021-03-09 | 马上消费金融股份有限公司 | Face classification method, model training method, electronic device and storage medium |
| CN112926557A (en) * | 2021-05-11 | 2021-06-08 | 北京的卢深视科技有限公司 | Method for training multi-mode face recognition model and multi-mode face recognition method |
| CN113378984B (en) * | 2021-07-05 | 2023-05-02 | 国药(武汉)医学实验室有限公司 | Medical image classification method, system, terminal and storage medium |
| CN113378984A (en) * | 2021-07-05 | 2021-09-10 | 国药(武汉)医学实验室有限公司 | Medical image classification method, system, terminal and storage medium |
| CN113903053A (en) * | 2021-09-26 | 2022-01-07 | 厦门大学 | Cross-modal pedestrian re-identification method based on unified intermediate modality |
| CN113903053B (en) * | 2021-09-26 | 2024-06-07 | 厦门大学 | Cross-mode pedestrian re-identification method based on unified intermediate mode |
| CN114627431A (en) * | 2022-02-22 | 2022-06-14 | 安徽新识智能科技有限公司 | Intelligent environment monitoring method and system based on Internet of things |
| CN114724041A (en) * | 2022-06-02 | 2022-07-08 | 浙江天铂云科光电股份有限公司 | Power equipment infrared chart identification method and system based on deep learning |
| CN118552794A (en) * | 2024-07-25 | 2024-08-27 | 湖南军芃科技股份有限公司 | Ore sorting identification method based on multichannel training and ore sorting machine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106909905B (en) | 2020-02-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106909905A (en) | A kind of multi-modal face identification method based on deep learning | |
| CN110111340A (en) | The Weakly supervised example dividing method cut based on multichannel | |
| JP6891351B2 (en) | How to generate a human hairstyle based on multi-feature search and deformation | |
| CN109145939B (en) | Semantic segmentation method for small-target sensitive dual-channel convolutional neural network | |
| CN113657349B (en) | Human behavior recognition method based on multi-scale space-time diagram convolutional neural network | |
| CN107766850B (en) | Face recognition method based on combination of face attribute information | |
| CN108182441B (en) | Parallel multichannel convolutional neural network, construction method and image feature extraction method | |
| CN105069400B (en) | Facial image gender identifying system based on the sparse own coding of stack | |
| CN109711281A (en) | A kind of pedestrian based on deep learning identifies again identifies fusion method with feature | |
| CN111310668B (en) | Gait recognition method based on skeleton information | |
| CN105095870B (en) | Pedestrian based on transfer learning recognition methods again | |
| CN105373777A (en) | Face recognition method and device | |
| CN113297955B (en) | Sign language word recognition method based on multi-mode hierarchical information fusion | |
| CN111291604A (en) | Face attribute identification method, device, storage medium and processor | |
| CN113255728A (en) | Depression classification method based on map embedding and multi-modal brain network | |
| CN109815826A (en) | The generation method and device of face character model | |
| CN109299701A (en) | Expand the face age estimation method that more ethnic group features cooperate with selection based on GAN | |
| KR20200075114A (en) | System and Method for Matching Similarity between Image and Text | |
| CN107424161B (en) | Coarse-to-fine indoor scene image layout estimation method | |
| CN110046550A (en) | Pedestrian's Attribute Recognition system and method based on multilayer feature study | |
| CN109409240A (en) | A kind of SegNet remote sensing images semantic segmentation method of combination random walk | |
| CN110321862B (en) | Pedestrian re-identification method based on compact ternary loss | |
| CN109558902A (en) | A kind of fast target detection method | |
| CN104240256A (en) | Image salient detecting method based on layering sparse modeling | |
| CN114613013A (en) | End-to-end human behavior recognition method and model based on skeleton nodes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |