CN109886072A - Face character categorizing system based on two-way Ladder structure - Google Patents
Face character categorizing system based on two-way Ladder structure Download PDFInfo
- Publication number
- CN109886072A CN109886072A CN201811595193.4A CN201811595193A CN109886072A CN 109886072 A CN109886072 A CN 109886072A CN 201811595193 A CN201811595193 A CN 201811595193A CN 109886072 A CN109886072 A CN 109886072A
- Authority
- CN
- China
- Prior art keywords
- scoring
- module
- face character
- submodule
- indicate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000004927 fusion Effects 0.000 claims abstract description 76
- 230000001815 facial effect Effects 0.000 claims abstract description 45
- 230000003044 adaptive effect Effects 0.000 claims abstract description 39
- 239000000284 extract Substances 0.000 claims description 42
- 238000000605 extraction Methods 0.000 claims description 28
- 239000011159 matrix material Substances 0.000 claims description 14
- 238000000034 method Methods 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 6
- 230000015572 biosynthetic process Effects 0.000 claims description 5
- 238000003786 synthesis reaction Methods 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 21
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000037308 hair color Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000000465 moulding Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
- 230000017260 vegetative to reproductive phase transition of meristem Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Abstract
The present invention relates to technical field of computer vision, more particularly to a kind of face character categorizing system based on two-way Ladder structure, the feature for how making full use of different levels in depth network aimed to solve the problem that, and the corresponding relationship between different levels feature and different faces attribute, to improve the accuracy of face character classification.For this purpose, the face character categorizing system provided by the invention based on two-way Ladder structure includes two-way Ladder self-encoding encoder module, adaptive attention power module and the Fusion Module that adaptively scores；Two-way Ladder self-encoding encoder module includes coder module and decoder module；It is adaptive to notice that power module includes multiple attention submodules；Adaptive scoring Fusion Module is configured to obtain the face character classification results of facial image to be measured according to the result of output result and attention the submodule output of coder module.The coding characteristic and decoding feature that different levels can be made full use of based on above structure, improve the accuracy of face character classification.
Description
Technical field
The present invention relates to technical field of computer vision, and in particular to a kind of face category based on two-way Ladder structure
Property categorizing system.
Background technique
Face character classification method be it is a kind of judge in given facial image whether include certain face character method,
It is widely used in the fields such as recognition of face, face retrieval, face verification.Currently, face character classification is carried out using depth network,
Classification performance can be significantly promoted, but depth network is primarily upon effect of the further feature to attributive classification, shallow-layer is special
Sign is not sufficiently excavated and is utilized.In addition, pair between the feature and different faces attribute of depth network different levels
It should be related to, also not be utilized sufficiently, so significantly limit the accuracy of face character classification.
Correspondingly, this field needs a kind of new face character categorizing system to solve the above problems.
Summary of the invention
In order to solve the above problem in the prior art, in order to solve how to make full use of different levels in depth network
Feature and different levels feature and different faces attribute between corresponding relationship, with improve face character classification it is accurate
Degree, the present invention provides a kind of face character categorizing system based on two-way Ladder structure, the face character categorizing systems
Include:
The face character categorizing system includes two-way Ladder self-encoding encoder module, adaptive pays attention to power module and adaptive
Should score Fusion Module；
The two-way Ladder self-encoding encoder module includes coder module and decoder module, the coder module packet
Multiple sequentially connected coding characteristic extract layers are included, the decoder module includes multiple sequentially connected decoding feature extractions
Layer, the outlet side of the coder module are connect with the input side of the decoder module；Wherein, the coding characteristic extract layer
It is equal with the decoding quantity of feature extraction layer；
The adaptive attention power module includes multiple attention submodules, the attention submodule respectively with it is described double
Into Ladder self-encoding encoder module, the corresponding coding characteristic extract layer of same data characteristics layer and decoding feature extraction layer connect
It connects；The attention submodule is configured to be mentioned according to the coding characteristic that the coding characteristic extract layer extracts with the decoding feature
The decoding feature for taking layer to extract carries out Fusion Features；
The adaptive scoring Fusion Module is configured to output result and each note according to the coder module
The Fusion Features result of meaning power submodule output obtains the face character classification results of the facial image to be measured.
Further, a preferred embodiment provided by the invention are as follows:
The adaptive scoring Fusion Module includes scoring fusion submodule and attributive classification device submodule, the attribute point
Class device submodule includes the first attributive classification device and multiple second attributive classification devices, the quantity of the second attributive classification device and institute
The quantity for stating attention submodule is equal；
The input side of the first attributive classification device is connect with the outlet side of the coder module, first attribute point
The outlet side of class device merges submodule connection with the scoring；The first attributive classification device is configured to according to the encoder mould
The coding characteristic of block output obtains corresponding first scoring of preset multiple face characters；
The input side of the multiple second attributive classification device is connect with the multiple attention submodule respectively, the multiple
The outlet side of second attributive classification device merges submodule connection with the scoring；The second attributive classification device is configured to basis
The fusion feature of the attention submodule output obtains corresponding second scoring of the multiple face character；
Scoring fusion submodule be configured to score to first scoring with described second be weighted merge and
The face character classification results of the facial image to be measured are obtained according to Weighted Fusion result.
Further, a preferred embodiment provided by the invention are as follows:
The attention submodule be further configured to according to the following formula shown in method according to the coding characteristic extract layer
The decoding feature that the coding characteristic of extraction and the decoding feature extraction layer extract carries out Fusion Features:
Wherein, the OutiIndicate the fusion feature of i-th of attention submodule output, the A (i) indicate described i-th
The corresponding mapping function of attention submodule andIt is describedDescribed in expressionWith institute
It statesMatrix parallel operation, it is describedIndicate that the corresponding coding characteristic extract layer of i-th of attention submodule mentions
The coding characteristic taken, it is describedIndicate the decoding that the corresponding decoding feature extraction layer of i-th of attention submodule extracts
Feature.
Further, a preferred embodiment provided by the invention are as follows:
The scoring fusion submodule is further configured to execute following operation:
Method shown according to the following formula scores to be weighted and merge and according to adding to first scoring with described second
Power fusion results obtain the face character classification results of the facial image to be measured:
Wherein, the P indicate Weighted Fusion result andAny one groupIt is all satisfiedIt is described
M indicates preset face character number, describedIndicate that the facial image to be measured belongs to the general of default kth kind face character
Rate, it is describedIndicate that the facial image to be measured is not belonging to the probability of default kth kind face character, the WsIndicate preset power
Weight matrix, the sjIndicate the face character classification scoring of j-th of particular community classifier output, the particular community classifier
For the first attributive classification device or the second attributive classification device, the people when the particular community classifier is the first attributive classification device
The scoring of face attributive classification is the first scoring, the face character point when the particular community classifier is the second attributive classification device
Class scoring is the second scoring, the ∑ sjIndicate the sum of the face character classification scoring of all particular community classifier outputs, institute
State " (sj,∑sj) " indicate the sjWith the ∑ sjMatrix parallel operation, it is describedRepresenting matrix multiplication operations；
Described in selectionWithClassification results of the corresponding face character of middle the larger value as kth kind face character.
Further, a preferred embodiment provided by the invention are as follows:
The two-way Ladder self-encoding encoder module, adaptive attention power module and adaptive scoring Fusion Module are bases
In neural network building module, the system also includes training module, the training module be configured to according to the following formula shown in
Loss function pays attention to power module and adaptive scoring Fusion Module to the two-way Ladder self-encoding encoder module, adaptively simultaneously
It is trained:
Wherein, describedIndicate the corresponding loss function of j-th of particular community classifier, the particular community point
Class device is the first attributive classification device or the second attributive classification device, and the n indicates the quantity of the particular community classifier, described
lFusionIndicate the corresponding loss function of the scoring fusion submodule, the lReconstructIndicate the corresponding loss letter of the decoder module
Number, the α, β respectively indicate preset hyper parameter.
Further, a preferred embodiment provided by the invention are as follows:
The corresponding loss function of the j particular community classifierIt is shown below:
Wherein, the M indicates preset face character number, pkIndicate the kth of the j particular community classifier output
The corresponding face character classification scoring of kind face character, the ykIndicate true kth kind face character label.
Further, a preferred embodiment provided by the invention are as follows:
The corresponding loss function l of submodule is merged in the scoringFusionIt is shown below:
Wherein, the tkIndicate that first scoring corresponding to kth kind face character, all second scorings and scoring are total
With the scoring vector being operated in parallel, the scoring summation be kth kind face character it is corresponding it is described first scoring with
The sum of all second scorings.
Further, a preferred embodiment provided by the invention are as follows:
The corresponding loss function l of the decoder moduleReconstructIt is shown below:
Wherein, describedIndicate L2Square of norm, it is describedIndicate the two-way Ladder self-encoding encoder module
The facial image synthesized according to facial image x to be measured.
Compared with the nearest prior art, above-mentioned technical method is at least had the following beneficial effects:
Face character categorizing system provided by the invention based on two-way Ladder structure mainly comprises the following structure: two-way
Ladder self-encoding encoder module, adaptive attention power module and the Fusion Module that adaptively scores；Two-way Ladder self-encoding encoder mould
Block includes coder module and decoder module, and coder module includes multiple sequentially connected coding characteristic extract layers, decoding
Device module includes multiple sequentially connected decoding feature extraction layers, and the outlet side of coder module is defeated with the decoder module
Enter side connection；Wherein, coding characteristic extract layer is equal with the decoding quantity of feature extraction layer；It is adaptive to notice that power module includes more
A attention submodule, attention submodule are right with the same data characteristics layer in two-way Ladder self-encoding encoder module respectively
The coding characteristic extract layer answered is connect with decoding feature extraction layer；Attention submodule is configured to be mentioned according to coding characteristic extract layer
The decoding feature that the coding characteristic and decoding feature extraction layer taken extracts carries out Fusion Features；Adaptive scoring Fusion Module configuration
To obtain face figure to be measured according to the Fusion Features result of the output result of coder module and the output of each attention submodule
The face character classification results of picture.Based on above structure can make full use of different levels coding characteristic and decoding feature and
Corresponding relationship between coding characteristic and decoding feature, improves the precision of face character classification.
Further, adaptively scoring Fusion Module includes scoring fusion submodule and attributive classification device submodule, attribute
Classifier submodule includes the first attributive classification device and multiple second attributive classification devices, the quantity and attention of the second attributive classification device
The quantity of power submodule is equal；The input side of first attributive classification device and the outlet side of coder module connect, the first attribute point
The outlet side of class device merges submodule connection with scoring；First attributive classification device is configured to the coding exported according to coder module
Feature obtains corresponding first scoring of preset multiple face characters；The input side of second attributive classification device is sub with attention respectively
Module connection, the outlet side of the second attributive classification device merge submodule connection with scoring；Second attributive classification device is configured to root
Corresponding second scoring of multiple face characters is obtained according to the fusion feature of attention submodule output；Scoring fusion submodule configuration
To be weighted the face for merging and obtaining facial image to be measured according to Weighted Fusion result with the second scoring to the first scoring
Attributive classification result.Based on above structure, precise score can be carried out to different levels feature is come from and be merged, further
Improve the accuracy of face character classification in ground.
Detailed description of the invention
Fig. 1 is the main knot of face character categorizing system of one of the embodiment of the present invention based on two-way Ladder structure
Structure schematic diagram；
Fig. 2 is a kind of the main of face character categorizing system based on two-way Ladder structure of another embodiment of the present invention
Structural schematic diagram.
Specific embodiment
The preferred embodiment of the present invention described with reference to the accompanying drawings.It will be apparent to a skilled person that this
A little embodiments are used only for explaining technical principle of the invention, it is not intended that limit the scope of the invention.
Face has contained a large amount of attribute information as important biological characteristic, such as sex, race, age, expression, hair
Individual event or multinomial face may be implemented in color etc., the face character categorizing system provided by the invention based on two-way Ladder structure
The prediction of attributive classification task.Specifically, the face character categorizing system based on two-way Ladder structure, which can be extracted, to be had not
The facial characteristics of same level improves the accuracy of facial image attributive classification.The system uses modularized design, between each module
Can be independent mutually, facilitate the building and update of model.Face character categorizing system based on two-way Ladder structure can adopt
It is constructed with depth convolutional neural networks, the local feature and global characteristics of different levels is extracted using convolutional neural networks, thus
Prediction part and global face character respectively.By adaptively paying attention to power module Weighted Fusion coding characteristic and decoding feature,
It is indicated with the level characteristics for learning richer.Adaptive scoring Fusion Module predicts different layers using multiple attributive classification devices respectively
Secondary facial attribute, and carry out the accuracy that scoring fusion further promotes face character classification.With reference to the accompanying drawing to base
It is described in detail in the face character categorizing system of two-way Ladder structure.
Refering to attached drawing 1, Fig. 1 illustrates the main knot of the face character categorizing system based on two-way Ladder structure
Structure schematic diagram, the face character categorizing system as shown in Figure 1 based on two-way Ladder structure may include such as flowering structure: two-way
Ladder self-encoding encoder module, adaptive attention power module and the Fusion Module that adaptively scores.
Two-way Ladder self-encoding encoder module includes coder module and decoder module, and coder module includes multiple suitable
The coding characteristic extract layer of secondary connection, decoder module include multiple sequentially connected decoding feature extraction layers, coder module
Outlet side connect with the input side of the decoder module；Wherein, the number of coding characteristic extract layer and decoding feature extraction layer
It measures equal.In some embodiments, two-way Ladder self-encoding encoder module is constructed based on depth convolutional neural networks, encoder mould
Block includes sequentially connected coding characteristic extract layer to extract the coding characteristic of different layers, and decoder module includes multiple sequentially connecting
The decoding feature extraction layer connect, to extract the decoding feature of different levels.Two-way Ladder self-encoding encoder module passes through encoder
Facial image to be measured is mapped to feature space by module, decoder module based on the coding characteristic that coder module exports restore to
Survey the facial image that facial image is synthesized.During synthesizing facial image, decoder can be in the decoding of different levels
Feature extraction layer learns the face character information to different levels, so two-way Ladder self-encoding encoder module is in order to obtain more
The character representation for adding different levels abundant when extracting the coding characteristic of different levels, while extracting the decoding of different levels
Feature.In addition, with incremental, the data characteristics tool that shallow-layer network extracts of the network number of plies in two-way Ladder self-encoding encoder module
There is locality, it includes the information of the details such as hair color, face contour, eye color in facial image, can be right
Answer the prediction of local facial attribute；Deep layer network extract data characteristics have it is of overall importance, it includes the property in facial image
Not, beautiful contour level semantic information, can correspond to the prediction of Global Face attribute.
Adaptive to notice that power module includes multiple attention submodules, attention submodule is self-editing with two-way Ladder respectively
The corresponding coding characteristic extract layer of same data characteristics layer is connect with feature extraction layer is decoded in code device module；Attention submodule
It is special that block is configured to the decoding feature progress that the coding characteristic extracted according to coding characteristic extract layer and decoding feature extraction layer extract
Sign fusion.Specifically, the quantity of attention submodule is less than or equal to the quantity of coding characteristic extract layer, the connection of attention submodule
Coding characteristic extract layer and decoding feature extraction layer, the corresponding coding characteristic of the two and decode the feature hierarchy of feature it is identical.
Each attention submodule can adaptive mode learn merge coding characteristic and decode feature.In some embodiments,
Attention submodule is further configured to the coding characteristic extracted according to method shown in formula (1) according to coding characteristic extract layer
The decoding feature extracted with decoding feature extraction layer carries out Fusion Features:
Wherein, OutiIndicate the fusion feature of i-th of attention submodule output, A(i)Indicate i-th of attention submodule
Corresponding mapping function and It indicatesWithMatrix parallel operation,Indicate the coding characteristic that the corresponding coding characteristic extract layer of i-th of attention submodule extracts,Indicate i-th of attention
The decoding feature that the corresponding decoding feature extraction layer of power submodule extracts.It should be noted that the adaptive attention mould of the present invention
It the quantity of block and can be adjusted according to the number of plies of network in the demand and two-way Ladder self-encoding encoder of user, it is right here
The adaptive quantity for paying attention to power module is not construed as limiting.
Adaptive scoring Fusion Module is configured to defeated according to the output result of coder module and each attention submodule
Fusion Features result out obtains the face character classification results of facial image to be measured.
Specifically, adaptively scoring Fusion Module includes scoring fusion submodule and attributive classification device submodule, attribute point
Class device submodule includes the first attributive classification device and multiple second attributive classification devices, the quantity and attention of the second attributive classification device
The quantity of submodule is equal.The input side of first attributive classification device and the outlet side of coder module connect, the first attributive classification
The outlet side of device merges submodule connection with scoring；It is special that first attributive classification device is configured to the coding exported according to coder module
Sign obtains corresponding first scoring of preset multiple face characters；The input side of multiple second attributive classification devices respectively with multiple notes
Power submodule of anticipating connects, and the outlet side of multiple second attributive classification devices merges submodule connection with scoring；Second attributive classification
Device is configured to obtain corresponding second scoring of multiple face characters according to the fusion feature that attention submodule exports；
Scoring fusion submodule is configured to be weighted the first scoring with the second scoring and merge and according to Weighted Fusion
As a result the face character classification results of facial image to be measured are obtained, it specifically, can be according to method shown in formula (2) to first
It scores to be weighted with the second scoring and merges and classified according to the face character that Weighted Fusion result obtains facial image to be measured
As a result:
Wherein, P indicate Weighted Fusion result andAppoint
It anticipates one groupIt is all satisfiedM indicates preset face character number, wherein default face character is root
According to the face character that face character classification task defines, such as gender, age, hair color be one such or multiple combinations,Indicate that facial image to be measured belongs to the probability of default kth kind face character,Indicate that facial image to be measured is not belonging to default the
The probability of k kind face character, WsIndicate preset weight matrix, in the present embodiment, weight matrix WsIt can be scoring fusion submodule
Study obtains after the completion of block training, sjIndicate the face character classification scoring of j-th of particular community classifier output, particular community
Classifier is the first attributive classification device or the second attributive classification device, the face when particular community classifier is the first attributive classification device
Attributive classification scoring is the first scoring, and when particular community classifier is the second attributive classification device, face character classification scoring is the
Two scorings, ∑ sjIndicate the sum of the face character classification scoring of all particular community classifier outputs, " (sj,∑sj) " indicate sj
With ∑ sjMatrix parallel operation,Representing matrix multiplication operations；
It choosesWithClassification results of the corresponding face character of middle the larger value as kth kind face character.It needs to illustrate
, Weighted Fusion result P is two classification to each face character, (for example, " 1 " indicates to belong to preset face character,
" 0 " indicates to be not belonging to preset face character), two probability values corresponding for each face characterWithAccording toWithThe result that middle the larger value is classified as this kind of face character.
In some embodiments, two-way Ladder self-encoding encoder module, adaptive attention power module and adaptive scoring are melted
Molding block is all based on the module of neural network building, and system further includes training module, and training module is configured to according to formula (3)
Shown in loss function to two-way Ladder self-encoding encoder module, adaptive pay attention to power module and the Fusion Module that adaptively scores
It is trained simultaneously:
Wherein,Indicate that the corresponding loss function of j-th of particular community classifier, particular community classifier are the first category
Property classifier or the second attributive classification device, n indicates the quantity of particular community classifier, lFusionIndicate that scoring fusion submodule is corresponding
Loss function, lReconstructIndicate the corresponding loss function of decoder module, α, β respectively indicate preset hyper parameter.
Wherein, the corresponding loss function of above-mentioned j-th of particular community classifierAs shown in formula (4):
Wherein, M indicates preset face character number, pkIndicate the kth kind face of j-th of particular community classifier output
The corresponding face character classification scoring of attribute, yk indicate true kth kind face character label.
The corresponding loss function l of scoring fusion submoduleFusionAs shown in formula (5):
Wherein, tkIndicate that the first scoring corresponding to kth kind face character, all second scorings and scoring summation carry out simultaneously
The scoring vector that connection operation obtains, scoring summation are the sum of corresponding first scoring of kth kind face character and all second scorings.
The corresponding loss function l of decoder moduleReconstructAs shown in formula (6):
Wherein,Indicate L2Square of norm,Indicate two-way Ladder self-encoding encoder module according to people to be measured
The facial image of face image x synthesis,The output of decoder module in as two-way Ladder self-encoding encoder module.Decoder
The corresponding loss function l of moduleReconstructDecoder module can be made to retain face character information as much as possible, so that decoder mould
The facial image to be measured that the facial image of block synthesis is corresponding as far as possible is approximate.
In conclusion the face character categorizing system provided by the invention based on two-way Ladder structure can be excavated sufficiently
The local feature and global characteristics of facial image to be measured, using the corresponding relationship between local feature and local facial attribute, with
And the corresponding relationship between global characteristics and Global Face attribute, promote the accuracy of face character classification.
The face character based on two-way Ladder structure point provided below with reference to another embodiment the present invention is described in detail
Class system.
Refering to attached drawing 2, Fig. 2 illustrates a kind of face category based on two-way Ladder structure of another embodiment
The primary structure of property categorizing system, as shown in Fig. 2, the face character categorizing system based on two-way Ladder structure includes two-way
Ladder self-encoding encoder module, adaptive attention power module and the Fusion Module that adaptively scores.Two-way Ladder self-encoding encoder mould
Block includes coder module and decoder module, and adaptive coder module includes 7 sequentially connected coding characteristic extract layers,
Decoder module includes 7 sequentially connected decoding feature extraction layers.It is adaptive to notice that power module includes 4 attention submodules
Block is to pay attention to power module respectively1, pay attention to power module2, pay attention to power module3, pay attention to power module4, attention submodule respectively with it is double
Into Ladder self-encoding encoder module, the corresponding coding characteristic extract layer of same data characteristics layer and decoding feature extraction layer connect
It connects, the adaptive Fusion Module that scores includes scoring fusion submodule and attributive classification device submodule, attributive classification device submodule packet
Include the first attributive classification device (i.e. classifier5) and 4 the second attributive classification device (i.e. classifiers1, classifier2, classifier3And classification
Device4).The input side of first attributive classification device and the outlet side of coder module connect, the outlet side of the first attributive classification device with
Scoring fusion submodule connection.That is, classifier5Input side and coder module outlet side connect, classifier5Outlet side
Submodule connection is merged with scoring.The input side of second attributive classification device is connect with attention submodule, i.e. classifier4Input
Side and attention power module4Connection, classifier3Input side and pay attention to power module3Connection, classifier2Input side and attention mould
Block2Connection, classifier1Input side and pay attention to power module1Connection, classifier1, classifier2, classifier3And classifier4Output
Submodule connection is merged with scoring in side.Two-way Ladder self-encoding encoder module can be according to original facial image (people i.e. to be measured
Face image) carry out the facial image that coding-decoding operation is synthesized.
With continued reference to attached drawing 2, the face character categorizing system based on two-way Ladder structure carries out original facial image
The step of face character is classified is as follows:
S1, original facial image is inputted into two-way Ladder self-encoding encoder module, extracts coding characteristic respectively and decoding is special
Sign.
Two-way Ladder self-encoding encoder module is encoded using coder module original facial image being mapped to feature space,
And original facial image is restored by decoder module and obtains synthesis facial image.In the mistake of decoder module synthesis facial image
Cheng Zhong, while extracting coding characteristic and decoding feature.As the number of plies of two-way Ladder self-encoding encoder is deepened, it is located at two-way
The feature of different levels has different characteristics in Ladder self-encoding encoder.Specifically, in two-way Ladder self-encoding encoder module
The feature extracted of shallow-layer there is locality, local face character prediction can be corresponded to；Have in the feature that deep layer is extracted complete
Office's property can correspond to global face character prediction.Therefore, by extracting the part of different levels and the character representation of the overall situation,
It may be implemented respectively to the classification with part and the face character of global property.
S2, the coding characteristic of two-way Ladder self-encoding encoder module output and decoding feature are sent into adaptive attention mould
Block carries out Fusion Features.
It is adaptive to notice that power module is that the output feature of coder module obtained in step S1 and decoder module is same
Shi Zuowei input carries out Fusion Features.The present embodiment is the data using four levels of two-way Ladder self-encoding encoder module
Feature, and construct four attention submodules that structure is identical but parameter is different.Each attention submodule is with certainly
The mode of adaptation learns to merge two kinds of features of coder module and decoder module output.To any one attention submodule
I (i=1,2,3,4) is enabledIndicate the coding characteristic that the corresponding coding characteristic extract layer of i-th of attention submodule extracts,Indicate the decoding feature that the corresponding decoding feature extraction layer of i-th of attention submodule extracts, attention submodule is first
Adaptively learn a mapping function.The mapping function can be followed by a Sigmoid letter by one 1 × 1 convolutional layer
Number obtains.Therefore, corresponding attention mapping can be expressed asFinally, attention submodule i
Output such as formula (1) shown in.
S3, it will adaptively notice that the fused feature of power module is sent into adaptive scoring Fusion Module, and carry out attributive classification
Face character classification results are obtained with scoring fusion.
By the volume of four attention submodule fusion features exported and two-way Ladder self-encoding encoder in step S2
The coding characteristic of code device end of module output, is respectively fed to five different attributive classification device submodules, to realize two-way
Five feature hierarchies while multiple face characters of classifying in Ladder self-encoding encoder.Specifically, each attributive classification device submodule is enabled
The face character classification scoring of block j output is sj(j=1,2,3,4,5) is obtaining the basis of five attributive classification device submodules
After scoring, the scoring of five attributive classification device submodules is summed to obtain another basic score ∑ s firstj, and by its with
The scoring of five attributive classification device submodules is in parallel, common to be sent into scoring fusion submodule.Scoring fusion submodule with it is a kind of from
The mode of adaptation learns a weight matrix Ws, each base categories appraisal result is weighted, final face is obtained
Shown in attributive classification result such as formula (2).
Those skilled in the art should be able to recognize that, mould described in conjunction with the examples disclosed in the embodiments of the present disclosure
Block, system, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate electronic hardware
With the interchangeability of software, each exemplary composition and step are generally described according to function in the above description.This
A little functions are executed actually with electronic hardware or software mode, the specific application and design constraint item depending on technical solution
Part.Those skilled in the art can use different methods to achieve the described function each specific application, but this
Kind is realized and be should not be considered as beyond the scope of the present invention.
Term " first ", " second " etc. are to be used to distinguish similar objects, rather than be used to describe or indicate specific suitable
Sequence or precedence.
Term " includes " or any other like term are intended to cover non-exclusive inclusion, so that including a system
Process, method, article or equipment/device of column element not only includes those elements, but also including being not explicitly listed
Other elements, or further include the intrinsic element of these process, method, article or equipment/devices.
So far, it has been combined preferred embodiment shown in the drawings and describes technical solution of the present invention, still, this field
Technical staff is it is easily understood that protection scope of the present invention is expressly not limited to these specific embodiments.Without departing from this
Under the premise of the principle of invention, those skilled in the art can make equivalent change or replacement to the relevant technologies feature, these
Technical solution after change or replacement will fall within the scope of protection of the present invention.
Claims (8)
1. a kind of face character categorizing system based on two-way Ladder structure, which is characterized in that face character classification system
System includes two-way Ladder self-encoding encoder module, adaptive attention power module and the Fusion Module that adaptively scores；
The two-way Ladder self-encoding encoder module includes coder module and decoder module, and the coder module includes more
A sequentially connected coding characteristic extract layer, the decoder module include multiple sequentially connected decoding feature extraction layers, institute
The outlet side for stating coder module is connect with the input side of the decoder module；Wherein, the coding characteristic extract layer reconciliation
The quantity of code feature extraction layer is equal；
The adaptive attention power module includes multiple attention submodules, the attention submodule respectively with it is described two-way
The corresponding coding characteristic extract layer of same data characteristics layer is connect with decoding feature extraction layer in Ladder self-encoding encoder module；
The attention submodule is configured to the coding characteristic extracted according to the coding characteristic extract layer and the decoding feature extraction
The decoding feature that layer extracts carries out Fusion Features；
The adaptive scoring Fusion Module is configured to output result and each attention according to the coder module
The Fusion Features result of submodule output obtains the face character classification results of the facial image to be measured.
2. the face character categorizing system according to claim 1 based on two-way Ladder structure, which is characterized in that described
Adaptive scoring Fusion Module includes scoring fusion submodule and attributive classification device submodule, the attributive classification device submodule packet
Include the first attributive classification device and multiple second attributive classification devices, the quantity of the second attributive classification device and the attention submodule
The quantity of block is equal；
The input side of the first attributive classification device is connect with the outlet side of the coder module, the first attributive classification device
Outlet side with it is described scoring merge submodule connect；The first attributive classification device is configured to defeated according to the coder module
Coding characteristic out obtains corresponding first scoring of preset multiple face characters；
The input side of the multiple second attributive classification device is connect with the multiple attention submodule respectively, and the multiple second
The outlet side of attributive classification device merges submodule connection with the scoring；The second attributive classification device is configured to according to
The fusion feature of attention submodule output obtains corresponding second scoring of the multiple face character；
Scoring fusion submodule be configured to score to first scoring with described second be weighted merge and according to
Weighted Fusion result obtains the face character classification results of the facial image to be measured.
3. the face character categorizing system according to claim 2 based on two-way Ladder structure, which is characterized in that described
Attention submodule be further configured to according to the following formula shown in the coding that is extracted according to the coding characteristic extract layer of method it is special
The decoding feature that sign is extracted with the decoding feature extraction layer carries out Fusion Features:
Wherein, the OutiIndicate the fusion feature of i-th of attention submodule output, the A(i)Indicate i-th of attention
The corresponding mapping function of power submodule andIt is describedDescribed in expressionWith it is describedMatrix parallel operation, it is describedIndicate what the corresponding coding characteristic extract layer of i-th of attention submodule extracted
Coding characteristic, it is describedIndicate the decoding feature that the corresponding decoding feature extraction layer of i-th of attention submodule extracts.
4. the face character categorizing system according to claim 3 based on two-way Ladder structure, which is characterized in that described
Scoring fusion submodule is further configured to execute following operation:
Method shown according to the following formula is weighted first scoring with second scoring and merges and melted according to weighting
Close the face character classification results that result obtains the facial image to be measured:
Wherein, the P indicate Weighted Fusion result andAppoint
It anticipates one groupIt is all satisfiedThe M indicates preset face character number, describedIndicate it is described to
The probability that facial image belongs to default kth kind face character is surveyed, it is describedIndicate that the facial image to be measured is not belonging to default kth
The probability of kind face character, the WsIndicate preset weight matrix, the sjIndicate j-th of particular community classifier output
Face character classification scoring, the particular community classifier is the first attributive classification device or the second attributive classification device, as the spy
Determining face character classification scoring when attributive classification device is the first attributive classification device is the first scoring, when the particular community point
Class device when being the second attributive classification device face character classification scoring be the second scoring, the ∑ sjIndicate all particular communities
The sum of face character classification scoring of classifier output, the " (sj,∑sj) " indicate the sjWith the ∑ sjMatrix it is in parallel
Operation, it is describedRepresenting matrix multiplication operations；
Described in selectionWithClassification results of the corresponding face character of middle the larger value as kth kind face character.
5. the face character categorizing system according to claim 4 based on two-way Ladder structure, which is characterized in that described
Two-way Ladder self-encoding encoder module, adaptive attention power module and adaptive scoring Fusion Module are all based on neural network structure
The module built, the system also includes training module, the training module be configured to according to the following formula shown in loss function to institute
It states two-way Ladder self-encoding encoder module, adaptive attention power module and adaptive scoring Fusion Module while being trained:
Wherein, describedIndicate that the corresponding loss function of the j particular community classifier, the particular community classifier are
First attributive classification device or the second attributive classification device, the n indicate the quantity of the particular community classifier, the lFusionIt indicates
The corresponding loss function of submodule, the l are merged in the scoringReconstructIndicate the corresponding loss function of the decoder module, it is described
α, β respectively indicate preset hyper parameter.
6. the face character categorizing system according to claim 5 based on two-way Ladder structure, which is characterized in that described
The corresponding loss function of j-th of particular community classifierIt is shown below:
Wherein, the M indicates preset face character number, pkIndicate the kth kind of j-th of particular community classifier output
The corresponding face character classification scoring of face character, the ykIndicate true kth kind face character label.
7. the face character categorizing system according to claim 6 based on two-way Ladder structure, which is characterized in that described
The corresponding loss function l of scoring fusion submoduleFusionIt is shown below:
Wherein, the tkIndicate it is corresponding to kth kind face character it is described first scoring, it is all second scoring and scoring summation into
The scoring vector that row parallel operation obtains, the scoring summation are corresponding first scorings of kth kind face character and own
The sum of second scoring.
8. the face character categorizing system according to claim 7 based on two-way Ladder structure, which is characterized in that described
The corresponding loss function l of decoder moduleReconstructIt is shown below:
Wherein, describedIndicate L2Square of norm, it is describedIndicate the two-way Ladder self-encoding encoder module according to
The facial image of facial image x synthesis to be measured.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811595193.4A CN109886072B (en) | 2018-12-25 | 2018-12-25 | Face attribute classification system based on bidirectional Ladder structure |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811595193.4A CN109886072B (en) | 2018-12-25 | 2018-12-25 | Face attribute classification system based on bidirectional Ladder structure |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109886072A true CN109886072A (en) | 2019-06-14 |
| CN109886072B CN109886072B (en) | 2021-02-26 |
Family
ID=66925169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811595193.4A Active CN109886072B (en) | 2018-12-25 | 2018-12-25 | Face attribute classification system based on bidirectional Ladder structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109886072B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110232672A (en) * | 2019-06-20 | 2019-09-13 | 合肥工业大学 | A kind of denoising method and system of exercise data |
| CN110826056A (en) * | 2019-11-11 | 2020-02-21 | 南京工业大学 | Recommendation system attack detection method based on attention convolution self-encoder |
| CN111104874A (en) * | 2019-12-03 | 2020-05-05 | 北京金山云网络技术有限公司 | Face age prediction method, training method and device of model and electronic equipment |
| CN111209916A (en) * | 2019-12-31 | 2020-05-29 | 中国科学技术大学 | Focus identification method and system and identification equipment |
| CN111652158A (en) * | 2020-06-04 | 2020-09-11 | 浙江大华技术股份有限公司 | Target object detection method and device, storage medium and electronic device |
| CN112434702A (en) * | 2019-08-26 | 2021-03-02 | 阿里巴巴集团控股有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| WO2021057426A1 (en) * | 2019-09-29 | 2021-04-01 | 腾讯科技（深圳）有限公司 | Method and apparatus for training image fusion processing model, device, and storage medium |
| CN116168040A (en) * | 2023-04-26 | 2023-05-26 | 四川元智谷科技有限公司 | Component direction detection method and device, electronic equipment and readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104143079A (en) * | 2013-05-10 | 2014-11-12 | 腾讯科技（深圳）有限公司 | Method and system for face attribute recognition |
| CN105404877A (en) * | 2015-12-08 | 2016-03-16 | 商汤集团有限公司 | Human face attribute prediction method and apparatus based on deep study and multi-task study |
| US20160328606A1 (en) * | 2013-11-15 | 2016-11-10 | Facebook, Inc. | Pose-aligned networks for deep attribute modeling |
| CN106682628A (en) * | 2016-12-30 | 2017-05-17 | 佳都新太科技股份有限公司 | Face attribute classification method based on multilayer depth feature information |
| CN106951825A (en) * | 2017-02-13 | 2017-07-14 | 北京飞搜科技有限公司 | A kind of quality of human face image assessment system and implementation method |
| CN107729909A (en) * | 2016-08-11 | 2018-02-23 | 中国移动通信集团湖南有限公司 | A kind of application process and device of attributive classification device |
| CN108229267A (en) * | 2016-12-29 | 2018-06-29 | 北京市商汤科技开发有限公司 | Object properties detection, neural metwork training, method for detecting area and device |
-
2018
- 2018-12-25 CN CN201811595193.4A patent/CN109886072B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104143079A (en) * | 2013-05-10 | 2014-11-12 | 腾讯科技（深圳）有限公司 | Method and system for face attribute recognition |
| US20160328606A1 (en) * | 2013-11-15 | 2016-11-10 | Facebook, Inc. | Pose-aligned networks for deep attribute modeling |
| CN105404877A (en) * | 2015-12-08 | 2016-03-16 | 商汤集团有限公司 | Human face attribute prediction method and apparatus based on deep study and multi-task study |
| CN107729909A (en) * | 2016-08-11 | 2018-02-23 | 中国移动通信集团湖南有限公司 | A kind of application process and device of attributive classification device |
| CN108229267A (en) * | 2016-12-29 | 2018-06-29 | 北京市商汤科技开发有限公司 | Object properties detection, neural metwork training, method for detecting area and device |
| CN106682628A (en) * | 2016-12-30 | 2017-05-17 | 佳都新太科技股份有限公司 | Face attribute classification method based on multilayer depth feature information |
| CN106951825A (en) * | 2017-02-13 | 2017-07-14 | 北京飞搜科技有限公司 | A kind of quality of human face image assessment system and implementation method |
Non-Patent Citations (3)
| Title |
|---|
| XIN ZHEN ET AL.: "A Survey of Deep Facial Attribute Analysis", 《ARXIV》 * |
| 邵蔚元 等: "多任务学习及卷积神经网络在人脸识别中的应用", 《计算机工程与应用》 * |
| 黄怀波: "基于生成模型的人脸图像合成与分析", 《中国博士学位论文全文数据库》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110232672A (en) * | 2019-06-20 | 2019-09-13 | 合肥工业大学 | A kind of denoising method and system of exercise data |
| CN112434702A (en) * | 2019-08-26 | 2021-03-02 | 阿里巴巴集团控股有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| WO2021057426A1 (en) * | 2019-09-29 | 2021-04-01 | 腾讯科技（深圳）有限公司 | Method and apparatus for training image fusion processing model, device, and storage medium |
| US11526712B2 (en) | 2019-09-29 | 2022-12-13 | Tencent Technology (Shenzhen) Company Limited | Training method and apparatus for image fusion processing model, device, and storage medium |
| CN110826056A (en) * | 2019-11-11 | 2020-02-21 | 南京工业大学 | Recommendation system attack detection method based on attention convolution self-encoder |
| CN110826056B (en) * | 2019-11-11 | 2024-01-30 | 南京工业大学 | Recommended system attack detection method based on attention convolution self-encoder |
| CN111104874A (en) * | 2019-12-03 | 2020-05-05 | 北京金山云网络技术有限公司 | Face age prediction method, training method and device of model and electronic equipment |
| CN111104874B (en) * | 2019-12-03 | 2024-02-20 | 北京金山云网络技术有限公司 | Face age prediction method, training method and training device for model, and electronic equipment |
| CN111209916A (en) * | 2019-12-31 | 2020-05-29 | 中国科学技术大学 | Focus identification method and system and identification equipment |
| CN111209916B (en) * | 2019-12-31 | 2024-01-23 | 中国科学技术大学 | Focus identification method and system and identification equipment |
| CN111652158A (en) * | 2020-06-04 | 2020-09-11 | 浙江大华技术股份有限公司 | Target object detection method and device, storage medium and electronic device |
| CN116168040A (en) * | 2023-04-26 | 2023-05-26 | 四川元智谷科技有限公司 | Component direction detection method and device, electronic equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109886072B (en) | 2021-02-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109886072A (en) | Face character categorizing system based on two-way Ladder structure | |
| CN110298037B (en) | Convolutional neural network matching text recognition method based on enhanced attention mechanism | |
| CN107330109B (en) | A kind of trade mark inquiry result degree of approximation evaluation and sort method, device | |
| CN106021364B (en) | Foundation, image searching method and the device of picture searching dependency prediction model | |
| CN110442684A (en) | A kind of class case recommended method based on content of text | |
| CN108920544A (en) | A kind of personalized position recommended method of knowledge based map | |
| CN109460737A (en) | A kind of multi-modal speech-emotion recognition method based on enhanced residual error neural network | |
| CN109902293A (en) | A kind of file classification method based on part with global mutually attention mechanism | |
| CN110134771A (en) | A kind of implementation method based on more attention mechanism converged network question answering systems | |
| CN109523463A (en) | A kind of face aging method generating confrontation network based on condition | |
| CN108280233A (en) | A kind of VideoGIS data retrieval method based on deep learning | |
| CN108920720A (en) | The large-scale image search method accelerated based on depth Hash and GPU | |
| CN108804677A (en) | In conjunction with the deep learning question classification method and system of multi-layer attention mechanism | |
| CN105718960A (en) | Image ordering model based on convolutional neural network and spatial pyramid matching | |
| CN110209789A (en) | A kind of multi-modal dialog system and method for user's attention guidance | |
| CN111159367B (en) | Information processing method and related equipment | |
| CN111127146A (en) | Information recommendation method and system based on convolutional neural network and noise reduction self-encoder | |
| CN109086405A (en) | Remote sensing image retrieval method and system based on conspicuousness and convolutional neural networks | |
| CN107491729B (en) | Handwritten digit recognition method based on cosine similarity activated convolutional neural network | |
| CN110175248A (en) | A kind of Research on face image retrieval and device encoded based on deep learning and Hash | |
| CN111444367A (en) | Image title generation method based on global and local attention mechanism | |
| CN110516145B (en) | Information searching method based on sentence vector coding | |
| CN109710768A (en) | A kind of taxpayer's industry two rank classification method based on MIMO recurrent neural network | |
| CN108256968A (en) | A kind of electric business platform commodity comment of experts generation method | |
| CN114693397A (en) | Multi-view multi-modal commodity recommendation method based on attention neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |