CN101404060B - Human face recognition method based on visible light and near-infrared Gabor information amalgamation - Google Patents

Abstract

The invention discloses a human face recognition method which is based on Gabor information fusion of visible light and near-infrared light. The method comprises: human face images under a visible light source and a near-infrared light source are respectively collected, Gabor features of the two images are respectively extracted to be fused at a feature layer; an AdaBoost algorithm is adopted to carry out the feature selection on the feature after the fusion, and the nearest neighbor classification is adopted to carry out the calculation and the classification on the similarity thereof. The human face recognition method has very high accuracy rate and excellent robustness of the impacts of light illumination on the human face recognition; in addition, compared with other methods, the human face recognition method has the advantages of small number of the used features, rapid classification speed, etc.

Description

A kind of face identification method based on visible light and near-infrared Gabor information fusion

Technical field

The present invention relates to a kind of face identification method, belong to intelligent monitoring technology in computer vision, particularly face recognition technology based on visible light and near-infrared Gabor information fusion.

Background technology

People's face is a pattern the most general in the human vision, and people's the visual information that face showed has important function and significance in social interactions and contacts.Recognition of face (Face Recognition) mainly is to study automatically algorithm and the theory that people's face of searching out in the image and facial characteristics point are distinguished.The research of recognition of face attempts to make computing machine to have human face as recognition capability.At present, the handled object of recognition of face mainly comprises the dynamic human face image in static facial image of single width and the video.Usually, people exist broad sense and two kinds of understandings of narrow sense to " recognition of face ".Broadly, recognition of face research comprises multinomial contents such as face tracking, the detection of people's face, the detection of facial characteristics point, human face recognition, facial expression analysis and face synthesize.Narrowly, recognition of face only refers to human face recognition.The invention belongs to the category of narrow sense recognition of face.

Recognition of face research relies on multiple subjects such as pattern-recognition, image understanding, computer vision and artificial intelligence, and subjects such as while and cognitive science, Neuscience, physiological psychology have the contact of countless ties.Although the mankind can carry out recognition of face fast and accurately like a dream, yet realize that the computing machine Automatic face recognition remains the huge challenge to the researcher.This is because the function of the human powerful cognitive ability that oneself is had is understood deficiency, thereby is difficult to simulate fully the recognition of face mechanism of human brain.At present, recognition of face research has obtained success in limited range, and has obtained application in certain field.

The range of application of recognition of face has contained a plurality of fields such as police criminal detection, judicial expertise, access control, financial payment, medical application, vision monitoring and video conference.

At first, face similarly is an important biomolecule feature a kind of high ubiquity, can contactless collection, and recognition of face has a wide range of applications in identity is differentiated.Recognition of face is a kind of of bio-identification (Biometrics) technology.Bio-identification mainly is meant according to the observation, analyzes and measure people's physical trait or people's behavioural characteristic, but extract everyone exclusive measurement features, thereby realize the method for identity identification and authentication, wherein physical trait comprises facial characteristics, details in fingerprint and iris texture etc.; And behavioural characteristic comprises gait, person's handwriting and signature etc.Desirable biometric feature should have characteristics such as stability, uniqueness and convenience.At present, the main research object of bio-identification comprises physical trait or behavioural characteristics such as fingerprint (Fingerprint), iris (Iris), retina (Retinal), person's handwriting (Handwriting), speaker (Speaker), gait (Gait), hand shape (Hand Geometry), palmmprint (Hand Vein), signature (Signature) and dna sequence dna.With traditional comparing based on special articles such as I.D., credit card, keys with based on the authentication identifying method of specific knowledge such as password, password, code word, people's biological characteristic does not have advantages such as can forgeing, can not lose and can not pretend to be.Therefore, biological identification technology has obtained swift and violent development in recent years, and progressively become a kind of be applied to security fields, the rising high tech industry.

And include the needed enough information of person identification of carrying out successfully in human high reliability that in recognition of face, shows and the high robustness explanation facial image.With respect to the biological characteristic of other kinds, facial image can obtain by direct, natural mode.Therefore, recognition of face is direct, friendly, convenient, belongs to the active identification of non-infringement, is easy to be accepted by the user.Recognition of face can be used in multiple different security fields: additional clause authentications such as driving license and passport; The security control of building turnover; Safety detection and monitoring in the important place such as airport; Authentication in the smart card.Face recognition technology also has huge potential using value at information security field.Along with network technology is come into daily life day by day, increasing user can access internet, and increasing information is placed into the internet.Because the convenience of network information access, the security control of network becomes a urgent day by day major issue.Utilize face recognition technology, can carry out computing machine land safely that control, application security are used, database is by the security control of full visit, file encryption, LAN (Local Area Network) and wide area network, can also protect safety of electrical business.

Secondly, face recognition technology can be used to create the man-machine interaction mode of friendly nature, is one of important content of intelligent computer area research.The abundant information that people's face is comprised plays crucial effect in social interaction and information interaction.Recognition of face need not special collecting device, and system cost is low relatively, does not disturb the user, does not invade user's privacy.The intelligent computer that can obtain knowledge such as identity, expression from user's face picture can provide convenient service for the user, in addition make between people and the computing machine alternately as interpersonal free and relaxed alternately.

In addition, face recognition technology also can be used for the image library retrieval, the retrieval face picture identical or close with thumbnail in large-scale facial image database.The face picture of given unknown human can utilize recognition of face to retrieve one or more immediate face pictures from database.For example: public security department can utilize face recognition technology to carry out management and the inquiry of criminal's face as the storehouse, ATM (automatic teller machine) (Automatic Teller Machine abbreviates ATM as) can be from database retrieval user fast.

Because the face recognition technology application prospect is extensive, recognition of face as one independently research topic attracted numerous researchers of international and domestic academia.

The mankind can trace back to before the several centuries the earliest to the exploration of face identification method.At numerous areas such as graphics art, anthropology, psychology and medical jurisprudence, the research of people's face all occupies a tiny space.In nearly 30 years, face recognition technology has obtained paying close attention to widely and studying in computer application field.As far back as the 60 to 70's of twentieth century, utilize computing machine to carry out the strong interest that Automatic face recognition has promptly caused the researcher, started the climax first time to recognition of face research.Enter nineteen nineties, recognition of face research is studied climax for the second time along with the develop rapidly of electronic computer technology has entered, and continues to remain so far the research focus in the field.

Many scientific research institutions have all set up the particular study group to be engaged in the research of recognition of face in the world wide.In the world, famous recognition of face research institution comprises (the Carnegie Mellon University of U.S. Ka Naijimeilong university, abbreviate CMU as) robot research institute, (the Massachusetts Institute ofTechnology of masschusetts, u.s.a Polytechnics, abbreviate MIT as) Media Lab and Artificial Intelligence Laboratory, Surrey university (University of Surrey) vision, voice and signal Processing research centre (Center for Vision, Speech and Signal Processing), Beckman research institute of university of Illinois, US university (University of Illinois), France INRIA (French National Institute for Research inComputer Science and Control) research institute, Switzerland IDIAP (Dalle Molle Institute forPerceptual Artificial Intelligence) research institute, Japan ART (AdvancedTelecommunications Research Institute International) research institute etc.The research of China aspect recognition of face is started in the climax initial stage for the second time, the domestic famous colleges and universities such as Tsing-Hua University, Peking University, Institutes Of Technology Of Nanjing, Northern Transportation University, Chinese University of Science and Technology, Harbin Institute of Technology, Shanghai Communications University and Zhongshan University that mainly contain, the Chinese Academy of Sciences calculate scientific research institutions such as institute, Institute of Automation, Chinese Academy of sociences and acoustics institute of the Chinese Academy of Sciences and have carried out correlative study.The special international conference that exchanges and inquire into face recognition technology has automatic face picture and gesture identification international conference (International Conference on Automatic Face and GestureRecognition, abbreviate AFGR as) and differentiate international conference (InternationalConference on Audio and Video Based Person Authentication abbreviates AVBPA as) based on the identity of audio frequency, video.Some pattern-recognitions, the famous international conference of computer vision field, as international computer visual conference (International Conference on Computer Vision, abbreviate ICCV as), international computer vision and pattern-recognition meeting (International Conference on Computer Vision andPattern Recognition, abbreviate CVPR as), international pattern-recognition meeting (InternationalConference on Pattern Recognition, abbreviate ICPR as), Europe computer vision meeting (European Conference on Computer Vision, abbreviate ECCV as) etc. and well-known international periodical, as IEEE Transactions on Pattern Analysis and Machine Intelligence (abbreviating PAMI as), Pattern Recognition (abbreviating PR as), Image and Vision Computing (abbreviating IVC as) etc. have set up the recognition of face special topic one after another.

In application, rely on the existing recognition of face achievement in research of scientific research circle, many scientific ﹠ technical corporation have pushed face recognition technology to application.The commercial recognition of face software of comparative maturity has the FaceIt of U.S. Visionics company and the recognition of face groupware of Viisage company etc.Chengdu, Sichuan silver Webex Communications Inc. in morning of China and mould knowledge company of middle section etc. are also carrying out related work.

At present, face recognition algorithms mainly is applicable under the qualification environment, limits the application under the categorical measure condition, also can not give play to the advantage that facial image can obtain under field conditions (factors) and discern far away.The variation of external factor such as illumination, attitude and expression is producing violent influence to recognition of face.Especially illumination problem is the stumbling-block that The Study of Interference person obtains desirable achievement always.

In order to solve the violent influence that illumination produces recognition of face, the researcher has carried out a lot of trials:

In list of references [1]: Georghiades A.S., Belhumeur P.N., and Kriegman D.J.From Few to Many:Illumination Cone Models for Face Recognition underVariable Lighting and Pose[J] .IEEE Trans.Pattern Analysis and MachineIntelligence, 2001,23 (9): 643-660,2001, people such as Georghiades have proved under different photoenvironments, the facial image of all identical attitudes forms a protruding vertebra (Convex Cone) model, and this model is called as illumination vertebra model; Ramamoorthi is in list of references [2]: Ramamoorthi R.Analytic PCA Construction for Theoretical Analysis of Lighting Variability inImages of a Lambertian Object[J] .IEEE Trans.Pattern Analysis andMachine Intelligence, 2002,25 (10): 1322-1333, Basri and Jacobs are in list of references [3]: Basri R.and Jacobs D.W.Lambertian Reflectance and LinearSubspaces[J] .IEEE Trans.Pattern Analysis and Machine Intelligence, 2003,25 (2): propose respectively among the 218-233 to adopt spheric harmonic function (Spherical Harmonic) representation that the low dimension facial image under the different light is made an explanation; Nayar, people such as Bolle and Jacobs proposes to adopt the Lambertian model that does not have shade to extract the intrinsic characteristic of facial image, see list of references [4] for details: Nayar S.K.and Bolle R.M.Reflectance Based Object Recognition[J] .Int ' 1 J.Computer Vision, 1996,17 (3): 219-240, and list of references [5]: Jacobs D., Belhumeur P.and Basri R.Comparing Images under Variable Illumination[A] .Proc.IEEE CS Conf.Computer Vision and Pattern Recognition[C] .1998:610-617; Shashua and Raviv are in list of references [6]: Shashua A.and Raviv T.R.TheQuotient Image:Class Based Re-Rendering and Recognition with VaryingIlluminations[J] .IEEE Trans.Pattern Analysis and Machine Intelligence, 2001,23 (2): the illumination invariant representation that has proposed a simple and practical quotient images (Quotient Image) algorithm extraction image among the 129-139, though these methods have all improved the accuracy rate of identification to a certain extent, do not obtain any for the constant face identification method of illumination.

Other methods that overcome illumination effect also comprise the method that adopts three-dimensional data and use the far infrared image; Wherein, under many circumstances, be actually the data that adopt 2.5 dimensions.List of references [7]: Bowyer K.W., Chang K.I.and Flynn P.J.A Survey of 3D and MultiModal 3D+2D Face Recognition[A] .Proc.Int ' 1 Conf.Pattern Recognition[C] data capture that obtains by laser scanner or 3D vision method among the .2004:358-361 geometric configuration of people's face, and this system seldom is subjected to the influence that ambient lighting changes; And be to the illumination variation robust, and can identify camouflage people face based on the advantage of the method for far infrared image.

Adopting near-infrared image to carry out recognition of face is a kind of effective means equally, compare with above method, can give prominence to the advantage based on near-infrared method: at first, near-infrared image keeps robust for the variation of ambient light in certain limit, can reach the requirement of illumination unchangeability; Secondly, more economy, cost lack this method, calculated amount is little than the method for employing three-dimensional data; At last, this method is not easy to be subjected to the influence of environment temperature, people's mood and health status, than adopting far method that stronger stability is arranged.

Consider the These characteristics of near infrared facial image, little by little caused related work person's attention based on the recognition of face of near-infrared image, more wide applications makes it to become the focus of Research on Face Recognition Technology in recent years in addition, by day all can operate as normal with night, be one to have widespread use and be worth and challenging problem.

The characteristics of contrast near-infrared image and visible images, and be summarised in the control test of carrying out on visible light and the near infrared facial image, the detail textures information that can find visible images is very abundant, for example, can see the freckle of people face, details such as acne, and these information have been lost in the gatherer process of near infrared facial image, the near infrared facial image can not show enough detailed information in other words, but the sharpest edges that are based on the recognition of face of near-infrared image are exactly the variation maintenance robust for illumination condition, and, under the violent condition of illumination variation, almost lost efficacy unlike recognition of face based on visible images.

Used method in the recognition of face, can be divided into two classes substantially, one class is with principal component analysis (PCA) (Principal Component Analysis, abbreviate PCA as) for the method based on global characteristics of representative, another kind of then is with elastic bunch graph coupling (Elastic Bunch Graph Matching, abbreviate EBGM as) and local binary pattern (Local Binary Pattern abbreviates LBP as) be the method based on local feature of representative.

List of references [8]: Turk M, Pentland A.Eigenfaces for recognition[A] .Journalof Cognitive Neuroscience[J] .1991,3 (1): 71-86 is described in detail PCA, is also referred to as eigenface method (Eigenface) based on the face identification method of PCA.This method is launched a formed high dimension vector with facial image by row (row) and is regarded a kind of random vector as, therefore can adopt Karhunen-Loeve transformation to obtain its quadrature K-L substrate. have shape with human face similarity corresponding to the substrate of big eigenwert wherein, so be called eigenface. utilize less relatively Eigenface collection to describe people's face, every like this width of cloth facial image is just corresponding to a weight vector that dimension is lower, therefore, recognition of face can be carried out on the space behind the dimensionality reduction.

EBGM has utilized Gabor Wavelet wave filter to carry out feature extraction, just said Gabor feature is carried out recognition of face, the correlation technique details is referring to list of references [9]: L.Wiskott, J.M.Fellous, N.Kruger, and C.Von der Malsburg.Face Recognition by Elastic Bunch GraphMatching[A] .IEEE Trans.Pattern Analysis and Machine Intelligence[J], 1997,19 (7): 775-779.Gabor Wavelet can well simulate the unicellular profile of experiencing the visual field in the cerebral cortex, catches outstanding perceptual property, for example space orientation, direction selection etc.Particularly GaborWavelet can extract multiple dimensioned in the image specific region, and multi-direction spatial frequency feature is amplified grey scale change as microscope.Like this, the eyes in the facial image, nose and mouth and some other local feature are exaggerated.Therefore, adopt Gabor Wavelet to extract the information of people's face, can strengthen some key features, to distinguish different facial images.

Since LBP in 2004 at list of references [10]: A.Timo, H.Abdenour, and P.Matti.Face Recognition with Local Binary Patterns[A] .In Proceedings of EuropeanConference on Computer Vision[C], by since recognition of face, be subjected to people's attention gradually among the 2004:469-481 based on the face identification method of LBP.Its reason mainly is for a lot of recognition of face utility systems, often can not obtain a plurality of samples of people to be identified, and this just means and can not effectively train them, and as the non-statistical learning method, LBP has demonstrated its distinctive advantage.This method derives from the texture analysis field. and it is each pixel and the order relation of its local neighborhood point in brightness in the computed image at first, then the two-value order relation is encoded and form local binary pattern, adopt the feature description of multizone histogram at last as image.What the LBP method was extracted in essence is localized variation features such as image border, angle point, and they are very important for distinguishing different people's faces.

Studies show that in recent years, be better than face identification method greatly based on the performance of the face identification method of local feature based on global characteristics, yet need extract the feature of higher dimension based on the face identification method of local feature, increased the difficulty of computation complexity and application real-time, be used for recognition of face so will in the multidimensional feature of extracting, select more most representative features, on the basis of the other accuracy rate of underwriter, reduce computation complexity.

Boosting is any given learning algorithm method of accuracy of a kind of raising.Its thought originates from Probably Approximately Correct (the abbreviating PAC as) learning model that Valiant proposes, see list of references [11] for details: Valiant L.G.A Theory of the Learnable[A] .Communications ofthe ACM[J], 1984,27 (11): 1134-1142.Valiant and Kearns have proposed the notion of weak study and strong study, and the identification error rate also is that accuracy rate is only than guessing that at random slightly high learning algorithm is called weak learning algorithm less than 1/2; Recognition accuracy finish in polynomial time very much learning algorithm of Gao Bingneng is called strong learning algorithm.Simultaneously, Valiant and Kearns are first in list of references [12]: Kearns M., Valiant L.G.Learning Boolean Formulae or Factoring[T] .Technical Report TR-1488, Cambridge, MA:Havard University Aiken Computation Laboratory, the equivalence question of learning algorithm and strong learning algorithm a little less than having proposed in the PAC learning model in 1988, promptly given arbitrarily only than guessing slightly good weak learning algorithm at random, it can be promoted and be strong learning algorithm? if the two equivalence, so only need find one just it can be promoted than the weak learning algorithm that conjecture is slightly good at random and to be strong learning algorithm, and needn't seek the strong learning algorithm that is difficult to acquisition.Nineteen ninety, Schapire is in list of references [13]: Schapire R.E.TheStrength of Weak Learnability[A] .Machine Learning[J], 1990,5 (2): the algorithm that constructs a kind of polynomial expression level among the 197-227 at first, this problem has been done sure proof, Here it is initial Boosting algorithm.After 1 year, Freund is in list of references [14]: Freund Y.Boosting aWeak Learning Algorithm By majority[A] .Information and Computation[J], 1995,121 (2): proposed the higher Boosting algorithm of a kind of efficient among the 256-285.But there is the defective in the common practice in these two kinds of algorithms, and that is exactly the lower limit that all requires to know in advance weak learning algorithm study accuracy.Nineteen ninety-five, Freund and Schapire have improved the Boosting algorithm, AdaBoost (Adaptive Boosting) algorithm has been proposed, see list of references [15] for details: Freund Y., Schapire R.E.A.A., Decision-Theoretic Generalization ofline Learning and an Application toBoosting[A] .Journal of Computer and System Sciences[J], 1997,55 (1): 119-139, the Boosting algorithm that this efficiency of algorithm and Freund proposed in 1991 much at one, but without any need for priori, thereby easier being applied in the middle of the practical problems about weak learner.Afterwards, Freund and Schapire have further proposed to change the AdaBoost.M1 of Boosting ballot weight, and the AdaBoost.M2 scheduling algorithm has received great concern in the machine learning field.

Summary of the invention

Consider above-mentioned reason, the present invention is merged the recognition of face of visible light and near-infrared image under certain strategy, to improve the effect of recognition of face.Main now convergence strategy has: data Layer merges, characteristic layer merges and decision-making level merges or the like, is optimum by the syncretizing effect of relatively finding characteristic layer.

The objective of the invention is under the application conditions indoor, that the user cooperates with on one's own initiative, merge the Gabor feature of visible light facial image and near infrared facial image effectively, provide a kind of computation complexity lower, and the face identification method of higher recognition correct rate is arranged.

For achieving the above object, the invention provides a kind of face identification method based near infrared and visible light facial image Gabor Feature Fusion, this method comprises:

Step 1: gather visible light facial image and near infrared facial image;

Build the software and hardware equipment of gathering visible images and near-infrared image;

Step 2: visible light facial image and near infrared facial image background removal, normalization;

Visible light facial image and near infrared facial image are carried out background removal and normalized;

Step 3: the Gabor feature of extracting visible light facial image and near infrared facial image;

Make up visible light and near infrared facial image to (Image Couple), and extract the Gabor feature of visible light facial image and near infrared facial image respectively;

Step 4: visible light people face Gabor feature and near infrared people face Gabor feature merge at characteristic layer;

The Gabor feature of visible light facial image and near infrared facial image is merged on characteristic layer;

Step 5: adopt the AdaBoost algorithm that the Gabor feature after merging is carried out feature selecting;

Use the AdaBoost algorithm that the Gabor feature after merging is carried out feature selecting;

Step 6: the employing nearest neighbor classifier calculates human face similarity degree and classifies;

Adopt the arest neighbors method to carry out the calculating and the classification of human face similarity degree.

In the technique scheme, guarantee the synchronous acquisition of visible light and near infrared facial image, so that it is right to set up facial image.Facial image is to carrying out the expression of people's face as the basic operation unit in the face recognition algorithms, replaced traditional with the let others have a look at method of face of single image table.Each facial image is to being made up of 2 images, one is visible images, another is a near-infrared image, these two images are in BUAA-IRIP facial image database building process, at one time respectively by colour TV camera and near infrared camera acquisition, so these two images are except the physical characteristics difference of image, in the attitude of people's face, aspects such as expression do not have evident difference.If carefully identification has the people probably near-infrared image to be regarded as visible images and carries out result after gray scale is handled.With such facial image his-and-hers watches face of leting others have a look at, can provide more horn of plenty, more comprehensive information.Wherein, BHU-IRIP facial image database is the facial image database that gather in BJ University of Aeronautics ﹠ Astronautics's Intelligent Recognition and Flame Image Process laboratory.

In the technique scheme, the Gabor Feature Extraction is to adopt 5 yardsticks, and the Gabor function and the image of 8 directions carry out convolution operation.

In the technique scheme, the AdaBoost algorithm is only selected preceding 500 processes that the most effective feature is used to classify in feature selection process.

The invention has the advantages that:

(1) influence that illumination in the recognition of face is produced provides the good restraining effect;

(2) has very high discrimination;

(3) used characteristic number is less, and computing velocity is fast.

Description of drawings

Fig. 1 is the process flow diagram of a kind of face identification method based on the fusion of visible light and near-infrared Gabor information of the present invention;

Fig. 2 is visible light and the right part sample of near infrared facial image composition diagram picture that collects;

Fig. 3 is the hardware synoptic diagram of image collecting device;

Fig. 4 is image collecting device and the locus synoptic diagram of being gathered the people;

Fig. 5 is the synoptic diagram of Gabor function;

Fig. 6 is the Gabor feature of the visible light facial image of employing function extraction shown in Figure 5;

Fig. 7 is the Gabor feature of the near infrared facial image of employing function extraction shown in Figure 5.

Embodiment

Below in conjunction with the drawings and specific embodiments the present invention is described in further details.

Fig. 1 shows the process flow diagram of a kind of face identification method that merges based on visible light and near-infrared Gabor information of the present invention.As shown in Figure 1:

Step 1: gather visible light facial image and near infrared facial image.

In order to control the light conditions in the images acquired process, and the applied environment of considering present face identification system is still based on indoor (Indoor) environment, and need people's cooperate with on one's own initiative (Cooperative), in order to obtain the higher near infrared facial image of quality, the present invention has adopted following method:

At first, settle initiatively near-infrared light source on the near infrared video camera, purpose is to strengthen positive near infrared intensity, shields the influence that the environment illumination condition changes as far as possible;

Then, reduce the negative effect of the variation of ambient lighting condition to the active near-infrared light source as far as possible.In addition,, require the infrared intensity of its generation to be controlled accurately, not only can support near infrared video camera shooting quality front face image preferably, and can not produce interference people's eyes for the active near-infrared light source.

For selecting initiatively near-infrared light source, the present invention selects the near infrared light spectral coverage of whole spectrum medium wavelength between 780nm and 1100nm as the active near-infrared light source.In engineering was used, initiatively the exposure intensity of near-infrared light source was relevant with the power of light source---and exposure intensity is big more, and the power of required active near-infrared light source is also big more.The exposure intensity of required active near-infrared light source and the power of light source are directly proportional.Under the test condition, people's face apart from camera and light source in 3 meters, initiatively near-infrared light source can be by 60～100 wavelength light emitting diode (Light Emitting Diode that is 850nm, abbreviate LED as) form, this wavelength almost is sightless for human eye, but most charge-coupled image sensor (Charge Coupled Device, abbreviate CCD as) or complementary metal oxide semiconductor (CMOS) (Complementary Metal Oxide Semiconductor, abbreviating CMOS as) sensor but is very responsive on this wavelength points, and the near infrared intensity that these diodes produce is enough to support indoor purposes.

In combination active near-infrared light source and near infrared video camera, the present invention is approximate to guarantee that light emitting diode and near infrared video camera are coaxial, can obtain the most suitable front direct light like this and shine.The experiment proved that this setting is all better than other any settings.And, after light emitting diode and camera are placed in together,, can control it very easily by means of a circuit box.Pass through meticulous adjustment afterwards, make that light source is very uniform when acting on people's face, as shown in Figure 3 light emitting diode putting position in the near infrared camera plane.

When the distance of finding near infrared video camera and people's face in experiment is 500cm～800cm, the user is very easily, this distance is 750cm among the present invention, as shown in Figure 4, and initiatively the bulk strength of near-infrared light source should be able to make when the distance of near infrared video camera and people's face changes that the near infrared facial image of generation has a good signal to noise ratio (S/N ratio) (S/N) in this scope.Article one, aim makes near infrared intensity strong as much as possible exactly, should be eager to excel than the ambient lighting intensity of estimating at least, and can not make that sensor is saturated.Another main points of being paid close attention to promptly are such infrared intensities for people's eyes safety whether, in general, when the working sensor of near infrared video camera under normal unsaturation pattern, the safety of human eye is can be guaranteed.

For the collection of near infrared facial image, the present invention is placed in an optical filter on the camera lens of near infrared video camera, and purpose is the incident that allows the incident of near infrared light and forbid visible light.The optical filter that the present invention adopts is 720nm in wavelength points, 800nm, and 850nm, the light transmittance during 880nm is respectively 0%, 50%, and 88%, 99%.Like this, can guarantee when the most wavelength of permission are the near infrared light incident of 850nm, to have stoped the incident of wavelength less than the luminous ray of 700nm.

Collection for the visible light facial image, the present invention uses a visible light camera to aim at people face position and gathers, and a visible light camera and near infrared video camera that is connected on the same image pick-up card carried out synchro control by image capture software, so that guarantee the simultaneity of image acquisition, carry out Feature Fusion and control experiment better more easily.The sample that collects as shown in Figure 2.

Step 2: visible light facial image and near infrared facial image background removal, normalization.

Background removal mainly adopts the method for human eye location, and pupil is positioned, and carries out background removal in conjunction with operations such as yardstick normalization, direction normalization and unitary of illumination.

Step 3: the Gabor feature of extracting visible light facial image and near infrared facial image.

The visible light and the near infrared facial image of facial image centering are carried out convolution operation with the Gabor function respectively, extract corresponding Gabor feature.

Gabor changes owing to have good spatial locality and directional selectivity, can describe the spatial frequency and the architectural feature of a plurality of directions in the image local zone, is a kind of good character description method.

The Gabor wave filter is defined as follows:

${\mathrm{\ψ}}_{\mathrm{\μ},\mathrm{\ν}}\left(z\right)=\frac{{\left|\right|k_{\mathrm{\μ},\mathrm{\ν}}\left|\right|}^2}{{\mathrm{\σ}}^2}\exp \left(\frac{-{\left|\right|k_{\mathrm{\μ},\mathrm{\ν}}\left|\right|}^2{\left|\right|z\left|\right|}^2}{2{\mathrm{\σ}}^2}\right)(\exp \left({\mathrm{ik}}_{\mathrm{\μ},\mathrm{\ν}}z\right)-\exp (-\frac{{\mathrm{\σ}}^2}{2}))$

Wherein (x y) is the spatial domain variable to z=, and ‖ ‖ represents modulo operation, and σ is a variance, k _{μ, v}Be the yardstick of decision Gabor wave filter and the frequency domain variable of direction, k _{μ, v}Be defined as follows:

$k_{\mathrm{\μ},\mathrm{\ν}}=k_{\mathrm{\ν}}{e}^{i{\mathrm{\φ}}_{\mathrm{\μ}}}$

And k is arranged _v=k _Max/ f _v, φ _μ=π μ/8.k _MaxBe maximum frequency, f is the factor coefficient between the frequency domain Gabor wave filter.μ, v are the yardstick and the directions of Gabor wave filter.

The Gabor image of piece image is that the Gabor wave filter of this image and a series of different scale and direction carries out convolution and obtains.Suppose that (x y) is the intensity profile of piece image to function I=, image I and Gabor wave filter ψ _{μ, v}The formula that carries out convolution is as follows:

O _μ，v(z)＝I(z)*ψ _μ，v(z)

* be the convolution algorithm symbol, o _{μ, v}(z) be image I and corresponding Gabor wave filter ψ _{μ, v}Convolution results on direction μ and yardstick v.

This paper use the Gabor wave filter 5 yardstick v ∈ (0 ..., 4), 8 direction μ ∈ (0 ..., 7), other parameters σ=2 π, k _Max=pi/2, $f=\sqrt{2},$ Fig. 5 has provided the real part form of 40 Gabor functions under above parameter condition.

Step 4: visible light people face Gabor feature and near infrared people face Gabor feature merge at characteristic layer.

The visible light that extracts and the Gabor feature of near infrared facial image are merged at characteristic layer.Fig. 6 and Fig. 7 have provided the Gabor filtering of a width of cloth visible images and a width of cloth near infrared facial image respectively and have represented.

The present invention has selected visible images and near-infrared image are carried out the fusion of characteristic layer.Propose two kinds at the fusion feature of characteristic layer and compare, the explanation based on the feature selection approach of AdaBoost that wherein relates to sees step 5 for details.

First kind is the Gabor feature of two kinds of images, extracts the Gabor proper vector χ of two kinds of images respectively _vAnd χ _n, and it is joined end to end into a new Gabor proper vector ρ _{V﹠amp; n}The facial image of expression correspondence is right, promptly

χ _v＝(χ _v1，χ _v2...χ _vs)；

χ _n＝(χ _n1，χ _n2...χ _nt)；

χ _v&n＝(χ _v，χ _n)＝(χ _v1，χ _v2...χ _vs，χ _n1，χ _n2...χ _nt)；

Adopt the AdaBoost algorithm to ρ afterwards _{V﹠amp; n}Carry out feature selecting, obtain being used for discriminant classification through the proper vector ρ that selects, this method is called as AdaBoost and selects to merge (AdaBoost Fusion abbreviates AF as).

Another kind is a resulting Euclidean distance when adopting addition principle (Sum Rule) to merge Gabor feature calculation image similarity, and the computing method of Euclidean distance are as follows:

Vector ρ ₁=(x ₁, x ₂... x _n) and vectorial ρ ₂=(y ₁, y ₂... y _n) between Euclidean distance be

$d({\mathrm{\ρ}}_1,{\mathrm{\ρ}}_2)=\sqrt{{(x_1-y_1)}^2+{(x_2-y_2)}^2+\·\·\·+{(x_n-y_n)}^2}=\sqrt{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{(x_i-y_i)}^2}$

To a kind of method is similar before, at first obtain the Gabor proper vector χ of facial image centering visible light and near-infrared image _vAnd χ _n,, obtain the proper vector ρ that two kinds of images are used to classify through feature selecting _vAnd ρ _n, adopt minimax normalization (Min-Max normalization) method, with the matching value that adopts nearest neighbor method NN to obtain according to following formula,

$N\left(d\right)=\frac{d-d_{\min }}{d_{\max }-d_{\min }}$

With d _vAnd d _nBe transformed in [0,1] interval N (d) the expression value after the normalization of d minimax of adjusting the distance wherein, d _MaxAnd d _MinBe maximal value and the minimum value in the Euclidean distance value set that calculates, and the results added that obtains sued for peace as new criteria for classification that this method is called as the addition rule and merges (Sum Rule Fusion abbreviates SR as).

Step 5: adopt the AdaBoost algorithm that the Gabor feature after merging is carried out feature selecting.

Along with the extensive and successful Application at people's face detection range, the Boosting algorithm has demonstrated its powerful ability on solution two classification problems.In order to adopt this method to carry out feature selecting, we must transform into one two classification problem with these many classification problems of recognition of face.Briefly, the recognition of face problem is a multiclass problem, but the variation of learning between the different samples of same body by the method for statistics forms space (Intra-personal Space) in the class, and the variation between the different samples of study Different Individual forms space (Extra-personal Space) between class.Therefore, classification problem more than also just has been converted to one two classification problem.This in the class and the estimation that distributes between class must be based on a hypothesis: distribute in the class and satisfy Gaussian distribution model.

Among the present invention, be to define like this in the class and between class: χ _iBe the Gabor proper vector of a width of cloth near infrared facial image, subscript i represents that this width of cloth image belongs to the individuality that is numbered i; χ _jIt is the Gabor proper vector of an other width of cloth near infrared facial image.D (χ)=|| χ _i-χ _j|| represent the difference between two vectors, i.e. difference between the Gabor proper vector of two width of cloth near infrared facial images.If i=j, D (χ) belong to space in the class, and are used as positive sample in training process; On the contrary, if i ≠ j, D (χ) belongs to space between class, is used as negative sample in training process.

AdaBoost is a version of Boosting algorithm, and often is used to differentiate two such classification problems of space between interior space of class and class.The AdaBoost basic idea is that one group of Weak Classifier is carried out strong classifier of the final formation of linear combination.

Weak Classifier can be one only by a simple feature f _j(χ) the simple threshold values function h of Zu Chenging _j(χ):

$h_j\left(x\right)=\left\{\begin{array}{cc}1& \mathrm{if}{p}_j{f}_j\left(x\right)<p_j{\mathrm{\&lambda;}}_j\\ 0& \mathrm{otherwise}\end{array}\right.$

λ wherein _jBe a threshold value (threshold), p _jBe the symbol (parity) of expression inequality direction, threshold value can be determined by positive sample average and negative sample average:

${\mathrm{\&lambda;}}_j=\frac{1}{2}[\frac{1}{m}\underset{p=1}{\overset{m}{\mathrm{\&Sigma;}}}{f}_j\left(x_p\right|y_p=1)+\frac{1}{l}\underset{n=1}{\overset{l}{\mathrm{\&Sigma;}}}{f}_j\left(x_n\right|y_n=0)]$

Each Weak Classifier is used for selecting a feature in all classification set by training.When these Weak Classifiers were combined strong classifier of formation, this sorter was far superior to a single Weak Classifier.The AdaBoost algorithm promotes the expressive force of those difficult classification modes in follow-up training set.

Step 6: the employing nearest neighbor classifier calculates human face similarity degree and classifies.

After feature extraction and two steps of feature selecting based on the AdaBoost algorithm based on the Gabor wave filter, the most expressive T feature that the present invention has obtained in the big measure feature is carried out sort operation.What sorter was selected is nearest neighbor algorithm (Nearest Neighbor abbreviates NN as), rather than the final strong classifier of AdaBoost that is made of several Weak Classifiers.Because the Weak Classifier of AdaBoost often is used to carry out two classification problems such as people's face checking, and for the so many classification problems of recognition of face, it is convenient that nearest neighbor algorithm NN then seems.

Nearest neighbor algorithm NN, relative for a certain people's face figure in the test set (Probe), the feature that is used to classify

Calculate respectively

With registered set (Gallery) in n classification face images to feature

Between Euclidean distance, wherein with

The registration sample of Euclidean distance minimum

Affiliated classification is

Affiliated classification.

$d_{\min }=d({\mathrm{\&chi;}}_{p_i},{\mathrm{\&chi;}}_{g_j})=\mathrm{Min}\left(\mathrm{Euclidean}({\mathrm{\&chi;}}_{p_i},{\mathrm{\&chi;}}_{g_k})\right),$ k＝1，2，…，n

$C\left({\mathrm{\&chi;}}_{p_i}\right)=C\left({\mathrm{\&chi;}}_{g_j}\right)$

Wherein C (χ) represents the affiliated classification of vectorial χ.

In order to prove the validity based on the algorithm that merges visible light and near infrared facial image Gabor of the present invention, the present invention tests on BUAA-IRIP facial image database, wherein the recognition of face of the recognition of face of visible images and near-infrared image is tested in contrast, the outstanding advantage that merges the recognition of face of two kinds of images, and by extracting PCA, features such as LBP compare, be intended to the stability of outstanding Gabor feature, simultaneously, effect for the feature selecting that proves AdaBoost, the present invention has adopted even down-sampling to carry out the dimensionality reduction of proper vector equally, and this mode is called as direct fusion (Direct Fusion abbreviates DF as).

Purpose according to test, BUAA-IRIP facial image data for experiment, the present invention has carried out reorganization and has been divided into two parts: first (experiment I) comprises 135 individualities, it is right that each individuality contains 10 facial images, wherein 5 pairs are used for training, other 5 pairs are used for test, are the validity of proof convergence strategy and Gabor feature based on the test fundamental purpose of this partial data; Second portion (experiment II) comprises 20 individualities, it is right that each individuality contains 5 facial images, these images are to all gathering after BUAA-IRIP facial image database makes up one month, this a part of all images all is used for test, is the robustness of this blending algorithm of proof for the illumination variation influence based on the experiment fundamental purpose of this partial data.

All view data samples are all normalized to 80 * 80 pixel.The image that obtains after every image and 40 the Gabor wave filter convolution is 64 * 64 pixels, therefore the right Gabor proper vector of each facial image comprises 2 * 64 * 64 * 5 * 8=327680 dimensional feature, select optimum representational 500 features by the AdaBoost algorithm, these features account for 0.2% of feature sum; And in the test of contrast with it, even down-sampling technology of the present invention, by 64 * 64 dimensionality reductions to 8 * 8, the right Gabor proper vector of each facial image has just comprised 2 * 8 * 8 * 5 * 8=5120 dimensional feature like this with the Gabor image.

Table 1

Table 1 has provided among the experiment I discrimination based on the blending algorithm of different characteristic.As can be seen from Table 1:

(1) adopts convergence strategy can improve the result of recognition of face to a great extent, be far superior to only adopt the recognition of face of single kind image;

(2) based on the blending algorithm of outperforming of the blending algorithm of local features such as Gabor and LBP based on global characteristics (PCA feature);

(3) blending algorithm based on the Gabor feature just can reach discrimination a higher level when selecting the feature of 500 dimensions, is about 97%, and only has 76% based on the discrimination of algorithm when selecting 500 dimensional features of LBP feature, and both differ greatly.

This explanation convergence strategy can be in the same place the advantages of visible images and near-infrared image effectively, and the feature of extracting with the Gabor wave filter is more stable, is more suitable in this convergence strategy.

The result of experiment II provides in table 2, owing to the second portion data were gathered in the different periods, so obvious variation has taken place illumination condition.The convergence strategy that is adopted in the experiment is identical with experiment I, as can be seen from Table 2, is insensitive based on the blending algorithm of Gabor feature for the variation of illumination, is far superior to LBP and PCA feature.

Table 2

Claims (4)

1. face identification method that merges based on visible light and near-infrared Gabor information is characterized in that this method may further comprise the steps:

Step 1: gather visible light facial image and near infrared facial image;

Near infrared man face image acquiring method is as follows:

At first, on the near infrared video camera, settle initiatively near-infrared light source;

Wherein, select the near infrared light spectral coverage of whole spectrum medium wavelength between 780nm and 1100nm as the active near-infrared light source, the LED that the active near-infrared light source is 850nm by N wavelength is formed, and wherein N is a natural number, N ∈ [60,100];

When combination active light source and near infrared video camera, with LED and the coaxial arrangement of near infrared video camera;

The putting position of meticulous adjustment light emitting diode in the near infrared camera plane, with the light source stepless action in people's face;

Then, an optical filter is placed on the camera lens of above-mentioned near infrared video camera;

The optical filter that adopts is 720nm in wavelength points, 800nm, and 850nm, the light transmittance during 880nm is respectively 0%, 50%, and 88%, 99%;

Collection for the visible light facial image, using a visible light camera to aim at people face position gathers, and above-mentioned visible light camera and the above-mentioned near infrared video camera that is connected on the same image pick-up card carried out synchro control by image capture software, image acquisition has simultaneity;

Step 2: visible light facial image and near infrared facial image background removal, normalization;

Background removal mainly adopts the method for human eye location, pupil is positioned, in conjunction with yardstick normalization, direction normalization and unitary of illumination operation carrying out background removal;

Step 3: the Gabor feature of extracting visible light facial image and near infrared facial image;

The visible light and the near infrared facial image of facial image centering are carried out convolution operation with the Gabor function respectively, extract corresponding Gabor feature;

Step 4: visible light people face Gabor feature and near infrared people face Gabor feature merge at characteristic layer;

The visible light that extracts and the Gabor feature of near infrared facial image are merged at characteristic layer; Gabor feature and these two kinds of fusion features at characteristic layer of Euclidean distance are proposed:

First kind is the Gabor feature of visible light and near infrared facial image,

Extract the Gabor proper vector χ of visible light and near infrared facial image at first, respectively _vAnd χ _n, and with proper vector χ _vAnd χ _nJoin end to end into a new Gabor proper vector ρ _{V﹠amp; n}The facial image of expression correspondence is right, promptly

χ _v＝(χ _v1，χ _v2…χ _vs)；

χ _n＝(χ _n1，χ _n2…χ _nt)；

χ _v&n＝(χ _v，χ _n)＝(χ _v1，χ _v2…χ _vs，χ _n1，χ _n2…χ _nt)；

Then, adopt the AdaBoost algorithm to ρ _{V﹠amp; n}Carry out feature selecting, obtain being used for discriminant classification through the proper vector ρ that selects;

Second kind is resulting Euclidean distance when adopting the addition principle to merge Gabor feature calculation image similarity, and the computing method of Euclidean distance are as follows:

Vector ρ ₁=(x ₁, x ₂X _n) and vectorial ρ ₂=(y ₁, y ₂Y _n) between Euclidean distance be:

$d({\mathrm{\&rho;}}_1,{\mathrm{\&rho;}}_2)=\sqrt{{(x_1-y_1)}^2+{(x_2-y_2)}^2+...+{(x_n-y_n)}^2}=\sqrt{\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{(x_i-y_i)}^2}$

With the Gabor feature similarity of above-mentioned first kind of visible light and near infrared facial image,

At first, obtain the Gabor proper vector χ of facial image centering visible light and near-infrared image _vAnd χ _n,, obtain the proper vector ρ that visible light and near infrared facial image are used to classify through feature selecting _vAnd ρ _n

Then, adopt the minimax method for normalizing, with the matching value that adopts nearest neighbor method to obtain according to formula

With d _vAnd d _nBe transformed in [0,1] interval;

Wherein, N (d) the expression value after the normalization of d minimax of adjusting the distance, d _MaxAnd d _MinBe maximal value and the minimum value in the Euclidean distance value set that calculates;

At last, the value addition after the d minimax normalization of adjusting the distance that obtains is sued for peace as new criteria for classification;

Step 5: adopt the AdaBoost algorithm that the Gabor feature after merging is carried out feature selecting;

χ _iBe the Gabor proper vector of a width of cloth near infrared facial image, subscript i represents that this width of cloth image belongs to the individuality that is numbered i; χ _jIt is the Gabor proper vector of an other width of cloth near infrared facial image;

D (χ)=|| χ _i-χ _j|| represent the difference between two vectors, i.e. difference between the Gabor proper vector of two width of cloth near infrared facial images;

If i=j, D (χ) belong to space in the class, and are used as positive sample in training process; On the contrary, if i ≠ j, D (χ) belongs to space between class, is used as negative sample in training process;

Step 6: the employing nearest neighbor classifier calculates human face similarity degree and classifies.

2. a kind of face identification method based on visible light and near-infrared Gabor information fusion according to claim 1 is characterized in that the distance of the described near infrared video camera of step 1 and people's face is 500cm～800cm.

3. a kind of face identification method based on visible light and near-infrared Gabor information fusion according to claim 1 is characterized in that the ambient lighting intensity that the strength ratio of the described active near-infrared light source of step 1 is estimated is strong.

4. a kind of face identification method based on visible light and near-infrared Gabor information fusion according to claim 1 is characterized in that the working sensor of the described near infrared video camera of step 1 is under normal unsaturation pattern.

Images