CN111460880B - Multimode biological feature fusion method and system - Google Patents

Multimode biological feature fusion method and system Download PDF

Info

Publication number
CN111460880B
CN111460880B CN202010023577.XA CN202010023577A CN111460880B CN 111460880 B CN111460880 B CN 111460880B CN 202010023577 A CN202010023577 A CN 202010023577A CN 111460880 B CN111460880 B CN 111460880B
Authority
CN
China
Prior art keywords
data
biological data
feature
mode
biological
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010023577.XA
Other languages
Chinese (zh)
Other versions
CN111460880A (en
Inventor
杨明辉
吴亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Simimage Technology Co ltd
Original Assignee
Hangzhou Simimage Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Simimage Technology Co ltd filed Critical Hangzhou Simimage Technology Co ltd
Publication of CN111460880A publication Critical patent/CN111460880A/en
Application granted granted Critical
Publication of CN111460880B publication Critical patent/CN111460880B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/70Multimodal biometrics, e.g. combining information from different biometric modalities
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • G06F18/251Fusion techniques of input or preprocessed data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • G06F18/253Fusion techniques of extracted features

Abstract

The embodiment of the invention discloses a multimode biological feature fusion method and a multimode biological feature fusion system, wherein the method comprises the following steps: feature comparison is carried out on feature data corresponding to each piece of biological data in the multi-mode biological data, a score vector of the multi-mode biological data is generated, piecewise linear classification processing is carried out on the score vector, a decision value corresponding to the score vector is generated, and identity information corresponding to the multi-mode biological data is identified according to the decision value. By adopting the invention, a plurality of biological characteristics are fused through the linear classifier, so that the useful information of each biological characteristic can be reserved, the identity information can be extracted from the high-dimensional characteristic space, and the accuracy of identity identification can be ensured.

Description

Multimode biological feature fusion method and system
Technical Field
The invention relates to the technical field of millimeter wave security inspection imaging identity recognition, in particular to a multimode biological feature fusion method and system.
Background
Each person has its own biometric features, such as fingerprint, face, ear profile, figure skeleton, iris, gait, etc., each of which can be used as an identification for identifying a person. But when special conditions occur (e.g., finger injury, mask wear, etc.), neither the corresponding fingerprint nor facial recognition accurately recognizes their corresponding identity.
Disclosure of Invention
The embodiment of the invention provides a multimode biological feature fusion method and a multimode biological feature fusion system, which are used for fusing multiple biological features through a linear classifier, so that the useful information of each biological feature can be reserved, the identity information is extracted from a high-dimensional feature space, and the accuracy of identity recognition can be ensured.
An embodiment of a first aspect of the present invention provides a multimode biometric fusion method, which may include:
feature comparison is carried out on feature data corresponding to each biological data in the multi-modal biological data, a score vector of the multi-modal biological data is generated, and the multi-modal biological data comprises at least two millimeter wave biological data collected based on an all-electronic sparse array;
performing piecewise linear classification processing on the score vector to generate a decision value corresponding to the score vector;
and identifying the identity information corresponding to the multi-mode biological data according to the decision value.
Further, the multimode biological feature fusion method further comprises the following steps:
and carrying out data normalization processing on the input multi-mode biological data.
Further, the multimode biological feature fusion method further comprises the following steps:
and extracting the characteristic data of each biological data after normalization processing by adopting a characteristic extraction algorithm matched with each biological data type in the multi-mode biological data.
Further, when feature comparison is performed on feature data corresponding to each piece of biological data in the multi-mode biological data to generate a score vector of the multi-mode biological data, the multi-mode biological feature fusion method further comprises:
dividing the characteristic data corresponding to each biological data in the multi-mode biological data into a plurality of non-overlapping sub-characteristic data;
and carrying out feature comparison on all the sub-feature data, and connecting the comparison scores of the feature comparison to form score vectors of the multi-mode biological data.
Further:
the multi-mode biological data comprises millimeter wave face image data, millimeter wave gait image data and gait electromagnetic echo data.
Embodiments of the second aspect of the present invention provide a multimode biometric fusion system, which may comprise:
the vector generation module is used for carrying out feature comparison on feature data corresponding to each piece of biological data in the multi-mode biological data to generate a score vector of the multi-mode biological data, wherein the multi-mode biological data comprises at least two pieces of millimeter wave biological data acquired based on an all-electronic sparse array;
the decision value generation module is used for carrying out piecewise linear classification processing on the score vector to generate a decision value corresponding to the score vector;
and the identity recognition module is used for recognizing the identity information corresponding to the multi-mode biological data according to the decision value.
Further, the multimode biometric fusion system further comprises:
and the data normalization module is used for performing data normalization processing on the input multi-mode biological data.
Further, the multimode biometric fusion system further comprises:
the characteristic extraction module is used for extracting the characteristic data of each biological data after normalization processing by adopting a characteristic extraction algorithm matched with each biological data type in the multi-mode biological data.
Further, the vector generation module includes:
a sub-data dividing unit for dividing the characteristic data corresponding to each biological data in the multi-mode biological data into a plurality of non-overlapping sub-characteristic data;
and the vector generation unit is used for carrying out feature comparison on all the sub-feature data, and connecting the comparison scores of the feature comparison to form the score vector of the multi-mode biological data.
Further:
the multi-mode biological data comprises millimeter wave face image data, millimeter wave gait image data and gait electromagnetic echo data.
In the embodiment of the invention, a plurality of biological characteristics are fused through the linear classifier, the useful information of each biological characteristic is reserved, the identity information is extracted in the high-dimensional characteristic space, and the accuracy of identity identification is ensured.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a multimode biological feature fusion method according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a multimode biometric fusion system according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a vector generation module according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by a person of ordinary skill in the art without inventive effort, are intended to be within the scope of the present invention, based on the embodiments of the present invention.
In the embodiment of the invention, the multi-mode biological feature fusion system can be simply called as a fusion system hereinafter.
The multimode biological feature fusion method provided by the embodiment of the invention will be described in detail with reference to fig. 1.
Referring to fig. 1, a flow chart of a multimode biological feature fusion method is provided in an embodiment of the invention. As shown in fig. 1, the method according to the embodiment of the present invention may include the following steps S101 to S103.
S101, performing feature comparison on feature data corresponding to each piece of biological data in the multi-mode biological data to generate a score vector of the multi-mode biological data.
It is understood that the fusion system may acquire input multi-modal biometric data, which may include at least two types of millimeter wave biometric data acquired based on an all-electronic sparse array, for example, millimeter wave face image data, millimeter wave gait image data, and gait electromagnetic echo data. Furthermore, the fusion system can perform data normalization processing on the multi-mode biological data. Furthermore, the fusion system may extract the feature data of each biological data after normalization processing by using a feature extraction algorithm matched with each biological data type in the multi-mode biological data, and optionally, the same feature extraction method may also be used to extract features of each biological data.
In the embodiment of the invention, the fusion system can perform feature comparison on the feature data corresponding to each piece of biological data in the multi-mode biological data to generate the score vector of the multi-mode biological data. It is understood that the score vector may be implicit with feature data useful for each biometric data.
In an alternative embodiment, the fusion system may divide the feature data corresponding to each biometric data in the multi-modal biometric data into a plurality of non-overlapping sub-feature data, and further, may perform feature comparison on all the sub-feature data, and connect the comparison scores of the feature comparison to form a score vector of the multi-modal biometric data.
S102, performing piecewise linear classification processing on the score vector to generate a decision value corresponding to the score vector.
In the embodiment of the invention, the fusion system can adopt a hierarchical fusion algorithm of a piecewise linear classifier to perform piecewise linear classification processing on the score vector, so as to generate a decision value corresponding to the score vector. It will be appreciated that the above described piecewise linear classification process may be a process of mapping the score vector to a higher dimensional feature space in which decision values are extracted that are implicitly included in the score vector to indicate identity information.
And S103, identifying identity information corresponding to the multi-mode biological data according to the decision value.
Specifically, the fusion system can identify identity information corresponding to the multi-mode biological data according to the decision value. It can be understood that the decision value can be an information value which is obtained after training and learning in the linear classifier and can comprehensively reflect the identity information.
In an alternative embodiment, the fusion system may train the classification model of the linear classifier according to a set of multimodal biological data collected in advance, and in the identification stage, the decision value may be input into the trained classification model to identify the identity information corresponding to the multimodal biological data.
In the embodiment of the invention, a plurality of biological characteristics are fused through the linear classifier, the useful information of each biological characteristic is reserved, the identity information is extracted in the high-dimensional characteristic space, and the accuracy of identity identification is ensured.
The following describes in detail the multimode biometric fusion system provided in the embodiment of the present invention with reference to fig. 2 and 3. It should be noted that, in the fusion system shown in fig. 2, for performing the method of the embodiment of fig. 1 of the present invention, only the portions relevant to the embodiment of the present invention are shown for convenience of description, and specific technical details are not disclosed, please refer to the embodiment of fig. 1 of the present invention.
As shown in fig. 2, the fusion system 10 according to the embodiment of the present invention may include: a vector generation module 101, a decision value generation module 102, an identification module 103, a data normalization module 104 and a feature extraction module 105.
The vector generation module 101 is configured to perform feature comparison on feature data corresponding to each piece of biological data in the multi-modal biological data, and generate a score vector of the multi-modal biological data.
It will be appreciated that the fusion system 10 may acquire input multi-modality biometric data that may include at least two millimeter wave biometric data acquired based on an all-electronic sparse array, for example, millimeter wave face image data, millimeter wave gait image data, and gait electromagnetic echo data. Further, the data normalization module 104 may perform data normalization processing on the multi-modal biological data. Further, the feature extraction module 105 may extract feature data of each biological data after normalization processing by using a feature extraction algorithm matched with each biological data type in the multi-mode biological data, or alternatively, may perform feature extraction on each biological data by using the same feature extraction method.
In the embodiment of the present invention, the vector generation module 101 may perform feature comparison on feature data corresponding to each piece of biological data in the multi-modal biological data, to generate a score vector of the multi-modal biological data. It is understood that the score vector may be implicit with feature data useful for each biometric data.
Optionally, referring to fig. 3 together, the vector generation module may include a sub-data dividing unit 1011 and a vector generation unit 1012.
The sub-data dividing unit 1011 may divide the feature data corresponding to each of the multi-modal biological data into a plurality of non-overlapping sub-feature data, and the vector generating unit 1012 may perform feature comparison on all the sub-feature data, and connect the comparison scores of the feature comparison to form a score vector of the multi-modal biological data.
The decision value generating module 102 is configured to perform piecewise linear classification processing on the score vector, and generate a decision value corresponding to the score vector.
In the embodiment of the present invention, the decision value generating module 102 may perform piecewise linear classification processing on the above-mentioned score vector by using a hierarchical fusion algorithm of a piecewise linear classifier, so as to generate a decision value corresponding to the score vector. It will be appreciated that the above described piecewise linear classification process may be a process of mapping the score vector to a higher dimensional feature space in which decision values are extracted that are implicitly included in the score vector to indicate identity information.
The identity recognition module 103 is configured to recognize identity information corresponding to the multi-mode biological data according to the decision value.
In a specific implementation, the identity recognition module 103 can recognize identity information corresponding to the multi-mode biological data according to the decision value. It can be understood that the decision value can be an information value which is obtained after training and learning in the linear classifier and can comprehensively reflect the identity information.
In an alternative embodiment, the fusion system 10 may train the classification model of the linear classifier according to a set of multimodal biological data collected in advance, and in the identification stage, the identity recognition module 103 may input the decision value into the trained classification model to recognize the identity information corresponding to the multimodal biological data.
In the embodiment of the invention, a plurality of biological characteristics are fused through the linear classifier, the useful information of each biological characteristic is reserved, the identity information is extracted in the high-dimensional characteristic space, and the accuracy of identity identification is ensured.
It should be understood that the execution of the steps of the method is only a preferred execution sequence, and the execution sequence may be adjusted according to actual requirements in the implementation process.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program comprising instructions for the relevant hardware, and the program may be stored on a computer-readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random-access Memory (Random Access Memory, RAM), or the like.
The foregoing disclosure is illustrative of the present invention and is not to be construed as limiting the scope of the invention, which is defined by the appended claims.

Claims (2)

1. A method of multimode biometric fusion, the method comprising: feature comparison is carried out on feature data corresponding to each biological data in the multi-modal biological data, a score vector of the multi-modal biological data is generated, and the multi-modal biological data comprises at least two millimeter wave biological data collected based on an all-electronic sparse array; performing piecewise linear classification processing on the score vector to generate a decision value corresponding to the score vector; identifying identity information corresponding to the multi-mode biological data according to the decision value;
the method further comprises the steps of:
carrying out data normalization processing on the input multi-mode biological data;
extracting the characteristic data of each biological data after normalization treatment by adopting a characteristic extraction algorithm matched with each biological data type in the multi-mode biological data;
when feature comparison is carried out on feature data corresponding to each piece of biological data in the multi-mode biological data, and score vectors of the multi-mode biological data are generated: dividing the characteristic data corresponding to each biological data in the multi-mode biological data into a plurality of non-overlapping sub-characteristic data; feature comparison is carried out on all the sub-feature data, and the comparison scores of the feature comparison are connected to form score vectors of the multi-mode biological data;
the multi-mode biological data comprises millimeter wave face image data, millimeter wave gait image data and gait electromagnetic echo data.
2. A multi-modal biometric fusion system, the system comprising: the vector generation module is used for carrying out feature comparison on feature data corresponding to each piece of biological data in the multi-mode biological data to generate a score vector of the multi-mode biological data, wherein the multi-mode biological data comprises at least two pieces of millimeter wave biological data acquired based on an all-electronic sparse array; the decision value generation module is used for carrying out piecewise linear classification processing on the score vector to generate a decision value corresponding to the score vector; the identity recognition module is used for recognizing the identity information corresponding to the multi-mode biological data according to the decision value;
the system further comprises: the data normalization module is used for performing data normalization processing on the input multi-mode biological data;
the system further comprises: the feature extraction module is used for extracting feature data of each biological data after normalization processing by adopting a feature extraction algorithm matched with each biological data type in the multi-mode biological data;
the vector generation module includes: a sub-data dividing unit for dividing the characteristic data corresponding to each biological data in the multi-mode biological data into a plurality of non-overlapping sub-characteristic data; the vector generation unit is used for carrying out feature comparison on all the sub-feature data, and connecting the comparison scores of the feature comparison to form score vectors of the multi-mode biological data;
the multi-mode biological data comprises millimeter wave face image data, millimeter wave gait image data and gait electromagnetic echo data.
CN202010023577.XA 2019-02-28 2020-01-09 Multimode biological feature fusion method and system Active CN111460880B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN2019101534070 2019-02-28
CN201910153407 2019-02-28

Publications (2)

Publication Number Publication Date
CN111460880A CN111460880A (en) 2020-07-28
CN111460880B true CN111460880B (en) 2024-03-05

Family

ID=71679928

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010023577.XA Active CN111460880B (en) 2019-02-28 2020-01-09 Multimode biological feature fusion method and system

Country Status (1)

Country Link
CN (1) CN111460880B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116434289A (en) * 2021-12-30 2023-07-14 同方威视技术股份有限公司 Millimeter wave or terahertz wave identity verification method and device, millimeter wave or terahertz wave security inspection equipment and electronic equipment

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000242781A (en) * 1999-02-18 2000-09-08 Nippon Telegr & Teleph Corp <Ntt> Structure information restoring method, recording medium and structure information restoring device
CN1794266A (en) * 2005-12-31 2006-06-28 清华大学 Biocharacteristics fusioned identity distinguishing and identification method
CN103902964A (en) * 2012-12-31 2014-07-02 深圳先进技术研究院 Face recognition method
CN203984590U (en) * 2014-08-21 2014-12-03 成都易默生汽车技术有限公司 The vehicle-mounted safe examination system of a kind of bus based on mm-wave imaging technology
CN105224849A (en) * 2015-10-20 2016-01-06 广州广电运通金融电子股份有限公司 A kind of multi-biological characteristic merges authentication identifying method and device
WO2016201679A1 (en) * 2015-06-18 2016-12-22 华为技术有限公司 Feature extraction method, lip-reading classification method, device and apparatus
CN107153811A (en) * 2017-03-20 2017-09-12 中国建设银行股份有限公司 Know method for distinguishing, apparatus and system for multi-modal biological characteristic
CN107294730A (en) * 2017-08-24 2017-10-24 北京无线电计量测试研究所 A kind of multi-modal biological characteristic identity identifying method, apparatus and system
CN108681746A (en) * 2018-05-10 2018-10-19 北京迈格威科技有限公司 A kind of image-recognizing method, device, electronic equipment and computer-readable medium
CN108960173A (en) * 2018-07-12 2018-12-07 芜湖博高光电科技股份有限公司 A kind of millimeter wave and camera merge face identification method

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5010905B2 (en) * 2006-12-13 2012-08-29 パナソニック株式会社 Face recognition device
CN102844766B (en) * 2011-04-20 2014-12-24 中国科学院自动化研究所 Human eyes images based multi-feature fusion identification method

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000242781A (en) * 1999-02-18 2000-09-08 Nippon Telegr & Teleph Corp <Ntt> Structure information restoring method, recording medium and structure information restoring device
CN1794266A (en) * 2005-12-31 2006-06-28 清华大学 Biocharacteristics fusioned identity distinguishing and identification method
CN103902964A (en) * 2012-12-31 2014-07-02 深圳先进技术研究院 Face recognition method
CN203984590U (en) * 2014-08-21 2014-12-03 成都易默生汽车技术有限公司 The vehicle-mounted safe examination system of a kind of bus based on mm-wave imaging technology
WO2016201679A1 (en) * 2015-06-18 2016-12-22 华为技术有限公司 Feature extraction method, lip-reading classification method, device and apparatus
CN105224849A (en) * 2015-10-20 2016-01-06 广州广电运通金融电子股份有限公司 A kind of multi-biological characteristic merges authentication identifying method and device
CN107153811A (en) * 2017-03-20 2017-09-12 中国建设银行股份有限公司 Know method for distinguishing, apparatus and system for multi-modal biological characteristic
CN107294730A (en) * 2017-08-24 2017-10-24 北京无线电计量测试研究所 A kind of multi-modal biological characteristic identity identifying method, apparatus and system
CN108681746A (en) * 2018-05-10 2018-10-19 北京迈格威科技有限公司 A kind of image-recognizing method, device, electronic equipment and computer-readable medium
CN108960173A (en) * 2018-07-12 2018-12-07 芜湖博高光电科技股份有限公司 A kind of millimeter wave and camera merge face identification method

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Moving Human Full-Body and Body-Parts Detection, Tracking, and Applications on Human Activity Estimation, Walking Pattern and Face Recognition;Hai-Wen Chen et al.;《SPIE Defense + Security》;20161231;全文 *
Xiaoli Zhou et al..Feature Fusion of Face and Gait for Human Recognition at a Distance in Video.《ICPR'06》.2006,全文. *
基于红外/毫米波双模融合的目标识别方法;陈锐;刘小英;刘德明;陈琛;;光学与光电技术;20060830(04);全文 *
多元数据融合的非干扰步态身份识别方法研究;于佃存;《中国优秀硕士学位论文全文数据库 信息科技辑》;20180415;全文 *

Also Published As

Publication number Publication date
CN111460880A (en) 2020-07-28

Similar Documents

Publication Publication Date Title
Wang et al. Towards complete and accurate iris segmentation using deep multi-task attention network for non-cooperative iris recognition
US10776470B2 (en) Verifying identity based on facial dynamics
Li et al. Facial expression recognition using deep neural networks
CN113033438B (en) Data feature learning method for modal imperfect alignment
Waheed et al. Person identification using vascular and non-vascular retinal features
Zakariah et al. Sign language recognition for Arabic alphabets using transfer learning technique
Alshamsi et al. Automated facial expression and speech emotion recognition app development on smart phones using cloud computing
Shekhar et al. Joint sparsity-based robust multimodal biometrics recognition
Pham et al. Offline handwritten signature verification using local and global features
Khurana et al. Static hand gestures recognition system using shape based features
Chang et al. Effectiveness evaluation of iris segmentation by using geodesic active contour (GAC)
CN111460880B (en) Multimode biological feature fusion method and system
Aslam et al. Gender classification based on isolated facial features and foggy faces using jointly trained deep convolutional neural network
Khryashchev et al. Bimodal person identification using voice data and face images
CN110738985A (en) Cross-modal biometric feature recognition method and system based on voice signals
Bansal et al. Multimodal biometrics by fusion for security using genetic algorithm
CN111428670B (en) Face detection method, face detection device, storage medium and equipment
CN102663457A (en) Method and system of identity authentication based on radio frequency identification and heart-sound technology
CN113657364A (en) Method, device, equipment and storage medium for recognizing character mark
Alirezazadeh et al. Effect of purposeful feature extraction in high-dimensional kinship verification problem
Trivikram et al. EVALUATION OF HYBRID FACE AND VOICE RECOGNITION SYSTEMS FOR BIOMETRIC IDENTIFICATION IN AREAS REQUIRING HIGH SECURITY.
Ebrahim Classification of Arabic Autograph as Genuine‎ And Forged through a Combination of New‎ Attribute Extraction Techniques
Jeon et al. Performance evaluation of teeth image recognition system based on difference image entropy
Khan et al. Face Acknowledgment using Principle Component Analysis (PCA) of Eigenfaces
Mahmood et al. A proposed model for fingerprint recognition based on convolutional neural networks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant