US7925060B2 - Authentication system and registration system related to facial feature information - Google Patents

Authentication system and registration system related to facial feature information Download PDF

Info

Publication number
US7925060B2
US7925060B2 US11/588,025 US58802506A US7925060B2 US 7925060 B2 US7925060 B2 US 7925060B2 US 58802506 A US58802506 A US 58802506A US 7925060 B2 US7925060 B2 US 7925060B2
Authority
US
United States
Prior art keywords
information
face
person
model
perfection level
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US11/588,025
Other languages
English (en)
Other versions
US20070098230A1 (en
Inventor
Toshio Norita
Yuichi Kawakami
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Konica Minolta Inc
Original Assignee
Konica Minolta Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Konica Minolta Inc filed Critical Konica Minolta Inc
Assigned to KONICA MINOLTA HOLDINGS, INC. reassignment KONICA MINOLTA HOLDINGS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KAWAKAMI, YUICHI, NORITA, TOSHIO
Publication of US20070098230A1 publication Critical patent/US20070098230A1/en
Application granted granted Critical
Publication of US7925060B2 publication Critical patent/US7925060B2/en
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G06V20/653—Three-dimensional objects by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces

Definitions

  • the present invention relates to a face authentication technique.
  • the face authentication technique as one of the biometric authentication techniques is a non-face-to-face authentication method and is expected to be applied to various fields of security with a monitor camera, an image database using faces as keys, and the like.
  • the method however, has a problem such that since the obtained face images are used as they are for authentication without evaluating the reliability, the recognition accuracy is not sufficiently high.
  • an authentication system includes: a generating part for generating face information including at least one of three-dimensional shape information and two-dimensional information in the face of a first person to be authenticated on the basis of measurement information of the first person; a model modifying part for modifying a standard model of a human face by using the face information, thereby generating an individual model of the face of the first person; a calculating part for calculating a first model perfection level as a perfection level of the individual model on the basis of reliability of the face information; an extracting part for extracting first feature information as feature information of the first person from the individual model; an obtaining part for obtaining second feature information as feature information of a second person to be compared which is pre-registered; and an authenticating part for performing an authenticating operation on the first person by using the first model perfection level in addition to similarity between the first feature information and the second
  • face information of a person to be authenticated is generated on the basis of measurement information.
  • an individual model of the face of the person to be authenticated is created.
  • a perfection level (first model perfection level) of the individual model is calculated on the basis of the reliability of the face information, and the authenticating operation is performed by using the feature information extracted from the individual model and the first model perfection level.
  • a registration system for registering information for face authentication includes: a generating part for generating face information including at least one of three-dimensional shape information and two-dimensional information in the face of a first person to be registered on the basis of measurement information of the first person; a model modifying part for modifying a standard model of a human face by using the face information, thereby generating an individual model of the face of the first person; a calculating part for calculating a model perfection level as a perfection level of the individual model on the basis of reliability of the face information; an extracting part for extracting feature information of the first person from the individual model; and a registering part for registering the model perfection level and the feature information as the information for face authentication.
  • face information of a person to be authenticated is generated on the basis of measurement information.
  • face information By using the face information, an individual model of the face of the person to be authenticated is created.
  • a model perfection level is calculated on the basis of the reliability of the face information, and the feature information extracted from the individual model and the model perfection level are registered as the information for face authentication.
  • the invention is also directed to a computer software program.
  • FIG. 1 is a diagram showing general operation of a face authentication system according to a first embodiment of the invention.
  • FIG. 2 is a configuration diagram showing a face verification system according to the first embodiment of the invention.
  • FIG. 3 is a diagram showing a configuration outline of a controller in the face verification system.
  • FIG. 4 is a block diagram showing various functions of the controller in the face verification system.
  • FIG. 5 is a block diagram showing a detailed functional configuration of a personal authenticating part.
  • FIG. 6 is a diagram showing various functions of a controller in a face registration system.
  • FIG. 7 is a block diagram showing a detailed functional configuration of a person registering part.
  • FIG. 8 is a flowchart showing operation of a controller in the face verification system.
  • FIG. 9 is a flowchart showing operation of the controller in the face verification system.
  • FIG. 10 is a diagram in which images continuously obtained from two cameras are displayed in time series.
  • FIG. 11 is a diagram showing feature points of a feature part in a face image.
  • FIG. 12 is a schematic diagram showing a state of calculating three-dimensional coordinates by using the principle of triangulation from feature points in a two-dimensional image.
  • FIG. 13 is a diagram showing a predetermined region around a feature point Q 20 as a center.
  • FIG. 14 is a diagram showing a standard model of a three-dimensional face.
  • FIG. 15 is a diagram showing texture information.
  • FIG. 16 is a diagram showing individual control points of a feature region after normalization.
  • FIG. 17 is a flowchart showing generation operation of a controller in the face registration system.
  • FIG. 18 is a flowchart showing operation of a controller in a face authentication system of a second embodiment.
  • FIG. 19 is a flowchart showing operation of the controller in the face authentication system of the second embodiment.
  • FIG. 20 is a flowchart showing model fitting in which semi-control point is considered.
  • FIG. 21 is a diagram showing semi-control points on a standard model.
  • FIG. 22 is a diagram showing correspondence between a three-dimensional position in an individual model of semi-control point and two-dimensional positions of semi-control point on two images.
  • FIG. 23 is a flowchart showing operation of the controller in a face registration system of the second embodiment.
  • FIG. 24 is a flowchart showing replacing operation in a face part detecting step.
  • FIG. 25 is a diagram showing corresponding feature points in obtained two images.
  • FIG. 26 is a diagram showing a three-dimensional shape measuring device constructed by a laser beam emitter and a camera.
  • FIG. 1 is a diagram showing general operation of a face authentication system 1 A according to a first embodiment of the present invention.
  • the face authentication system 1 A has a face registration system SYS 1 and a face verification system SYS 2 as two subsystems.
  • face information EB 1 of a person HMa to be registered is generated on the basis of stereo images EA 1 of the person HMa to be registered which are captured repeatedly (continuously) at different timings (time points), and an individual model GM 1 is generated by modifying a standard model on the basis of the face information EB 1 .
  • Predetermined processes such as feature extracting process and model perfection level computing process
  • corresponding to the individual model GM 1 and the face information EB 1 are executed, thereby obtaining feature information and model perfection level of the individual model GM 1 , respectively.
  • the feature information of the person HMa to be registered and the model perfection level is registered as registrant information EC 1 of the person HMa to be registered into a person information storage 34 which will be described later.
  • face verification system SYS 2 first, processes similar to those performed in the face registration system SYS 1 are executed on the basis of stereo images EA 2 (measurement information) of a person HMb to be authenticated which are captured repeatedly (continuously) at different timings (time points).
  • face information EB 2 of the person HMb to be authenticated is generated on the basis of the stereo images EA 2 (measurement information) of the person HMb to be authenticated which are captured repeatedly at different timings
  • an individual model GM 2 is generated by modifying a standard model on the basis of the face information EB 2 .
  • Predetermined processes (such as feature extracting process and model perfection level computing process) corresponding to the individual model GM 2 and the face information EB 2 are executed, thereby obtaining feature information of the person HMb to be authenticated and model perfection level of the individual model GM 2 , as information EC 2 of the person to be authenticated, respectively.
  • authenticating operation is performed, for comparing the information EC 2 of the person HMb to be authenticated with the registrant information EC 1 (feature information and the like) of the person HMa to be registered which is registered in the person information storage 34 ( FIG. 5 ).
  • the model perfection level of the individual model GM 2 is used.
  • the model perfection level is calculated on the basis of reliability of the face information EB 2 of the person HMb to be authenticated. Consequently, authentication in which the reliability of the face information EB 2 is reflected can be realized.
  • such a face authentication system 1 A can be applied to, for example, a copying machine.
  • a situation is assumed such that a person (user) to use a copying machine is specified and an operation panel dedicated to the user is displayed.
  • operation of capturing face images (verification images) of the user starts and is continued at predetermined timings until the user reaches an operation panel of the copying machine.
  • the person is specified (authenticated), and the operation panel of the copying machine changes to a panel dedicated to the user. Consequently, the operation content often used by the user is selectively displayed in the screen of the operation panel, and the working efficiency can be improved.
  • the specification of a person is performed to improve operability. Even if the specification of a person is performed erroneously, it is sufficient to make an error correction by entering a password of the operator or the like and log in the system again. That is, the authentication accuracy required for the face authentication system 1 A for selecting a panel state is relatively low.
  • the face authentication system 1 A repeatedly obtains face images of the user from the beginning of approach of the user until reliable information necessary for the required authentication accuracy is generated. Face images obtained for a relatively long period may include various face images in which the face orientation of the user changes, so that face authentication using various information can be performed, and high-accuracy authentication can be realized. Specifically, the face authentication system 1 A evaluates the reliability (quality and/or quantity) of the obtained data as model perfection level, and uses the evaluation value for face image capturing and/or authenticating operation. With the configuration, in the case where the authentication accuracy is requested more than higher response speed of authentication from the viewpoint of confidential document protection, proper authentication can be performed.
  • the face authentication system 1 A can perform proper authentication according to the required authentication accuracy (authentication level).
  • FIG. 2 is a configuration diagram showing the face verification system SYS 2 according to a first embodiment of the invention.
  • the face verification system SYS 2 is constructed by a controller 10 b and two image capturing cameras (hereinbelow, also simply called “cameras”) CA 1 and CA 2 .
  • the cameras CA 1 and CA 2 are disposed so as to be able to capture images of the face of the person HMb to be authenticated from different positions.
  • face images of the person HMb to be authenticated are captured by the cameras CA 1 and CA 2
  • appearance information specifically, two face images of the person HMb to be authenticated captured by the image capturing operation is transmitted to the controller 10 b via a communication line.
  • the communication system for image data between the cameras and the controller 10 b is not limited to a wired system but may be a wireless system.
  • FIG. 3 is a diagram showing a schematic configuration of the controller 10 b .
  • the controller 10 b is a general computer such as a personal computer including a CPU 2 , a storage 3 , a media drive 4 , a display 5 such as a liquid crystal display, an input part 6 such as a keyboard 6 a and a mouse 6 b as a pointing device, and a communication part 7 such as a network card.
  • the storage 3 has a plurality of storage media, concretely, a hard disk drive (HDD) 3 a and a RAM (semiconductor memory) 3 b capable of performing processes at a higher speed than the HDD 3 a .
  • HDD hard disk drive
  • RAM semiconductor memory
  • the media drive 4 can read information recorded on a portable recording medium 8 such as CD-ROM, DVD (Digital Versatile Disk), flexible disk, or memory card.
  • a portable recording medium 8 such as CD-ROM, DVD (Digital Versatile Disk), flexible disk, or memory card.
  • the information supplied to the controller 10 b is not limited to information supplied via the recording medium 8 but may be information supplied via a network such as LAN or the Internet.
  • FIG. 4 is a block diagram showing the various functions of the controller 10 b .
  • FIG. 5 is a block diagram showing a detailed functional configuration of a personal authenticating part 14 .
  • the various functions of the controller 10 b are conceptual functions realized by executing a predetermined software program (hereinbelow, also simply called “program”) with various kinds of hardware such as the CPU in the controller 10 b.
  • program a predetermined software program
  • the controller 10 b has an image input part 11 , a face area retrieving part 12 , a face part detector 13 , the personal authenticating part 14 , and an output part 15 .
  • the image input part 11 has the function of inputting two images (stereo images) captured by the cameras CA 1 and CA 2 to the controller 10 b.
  • the face area retrieving part 12 has the function of specifying a face part in an input face image.
  • the face part detector 13 has the function of detecting the positions of feature parts (for example, eyes, eyebrows, nose, mouth, and the like) in the specified face area.
  • the personal authenticating part 14 is constructed to mainly authenticate a face and has the function of authenticating a person by using a face image. The details of the personal authenticating part 14 will be described later.
  • the output part 15 has the function of outputting an authentication result obtained by the personal authenticating part 14 .
  • the personal authenticating part 14 has a three-dimensional reconstructing part 21 , a face information generating part 22 , an optimizing part 23 , a correcting part 24 , a model perfection level computing part 25 , a feature extracting part 26 , an information compressing part 27 , and a comparing part 28 .
  • the three-dimensional reconstructing part 21 has the function of calculating coordinates in three dimensions of each feature point on the basis of two-dimensional coordinates of the feature point which is set for a feature part of a face obtained from input images.
  • the three-dimensional coordinate calculating function is realized by using camera information stored in a camera parameter storage 31 .
  • the face information generating part 22 has the function of sequentially obtaining three-dimensional coordinates of each of feature points of a face obtained from stereo images (two images) repeatedly (continuously) input to the three-dimensional reconstructing part 21 , and generating the face information EB 2 (hereinbelow, also called “correction face information”) of the person HMb to be authenticated.
  • the face information generating part 22 has the function of executing predetermined statistic process using reliability of the three-dimensional coordinates of each feature point to the sequentially obtained three-dimensional coordinates of each feature point, and correcting the three-dimensional coordinates of each feature point.
  • the three-dimensional coordinates of each feature point corrected are also called “corrected three-dimensional coordinates” or “corrected three-dimensional position”.
  • the optimizing part 23 has the function of generating the individual model GM 2 by modifying a standard stereoscopic model (also simply called “standard model”) of a human face stored in a three-dimensional model database 32 based on the face information generated by the face information generating part 22 .
  • a standard stereoscopic model also simply called “standard model”
  • the correcting part 24 has the function of correcting the generated individual model GM 2 .
  • the processing parts 21 to 24 information of the person HMb to be authenticated is normalized and converted into forms which can be easily compared with each other.
  • the individual model generated by the functions of the processing parts includes both three-dimensional information and two-dimensional information of the person HMb to be authenticated.
  • the “three-dimensional information” is information related to a stereoscopic configuration constructed by three-dimensional coordinate values or the like.
  • the “two-dimensional information” is information related to a plane configuration constructed by surface information (texture information) and/or information of positions in a plane or the like.
  • the model perfection level computing part 25 has the function of computing the perfection level of the generated individual model on the basis of the reliability of each of the feature points.
  • the feature extracting part 26 has a feature extracting function of extracting the three-dimensional information and two-dimensional information from the individual model generated by the processing parts 21 to 24 .
  • the information compressing part 27 has the function of compressing the three-dimensional information and the two-dimensional information used for face authentication by converting each of the three-dimensional information and the two-dimensional information extracted by the feature extracting part 26 to a proper face feature amount (feature information) for face authentication.
  • the information compressing function is realized by using information stored in a compressed information storage 33 , and the like.
  • the comparing part 28 has the function of calculating similarity between a face feature amount of the registered person (person to be compared) which is pre-registered in a person information storage 34 and a face feature amount of the person HMb to be authenticated, which is obtained by the above-described function parts, thereby authenticating the face.
  • a face feature amount of the registered person person to be compared
  • a face feature amount of the person HMb to be authenticated which is obtained by the above-described function parts, thereby authenticating the face.
  • perfection level of an individual model calculated by the model perfection level computing part 25 is also used for authentication.
  • the face registration system SYS 1 has a configuration similar to that of the face verification system SYS 2 shown in FIG. 2 . Specifically, the face registration system SYS 1 has the controller 10 a and the two cameras CA 1 and CA 2 . Appearance information, that is, two face images of the person HMa to be registered, obtained by the two cameras can be input to the controller 10 a.
  • controller 10 a Various functions of the controller 10 a will now be described.
  • FIG. 6 is a diagram showing various functions of the controller 10 a
  • FIG. 7 is a block diagram showing a detailed function configuration of a personal registering part 41 .
  • the controller 10 a has a hardware configuration similar to that of the controller 10 a shown in FIG. 3 .
  • the various functions of the controller 10 a are conceptual functions realized by executing a predetermined software program (hereinbelow, also simply called “program”) with various kinds of hardware such as the CPU in the controller 10 a.
  • program a predetermined software program
  • the controller 10 a has, in addition to the functions (image input part 11 , face area retrieving part 12 , and face part detector 13 ) of the controller 10 b , the personal registering part 41 .
  • the controller 10 a having the functions two images of the person HMa to be registered obtained by the two cameras (CA 1 and CA 2 ) are input. Processes using the function parts are performed on the input information (measurement information).
  • the personal registering part 41 is constructed to mainly register feature information of a face and has the function of generating the registrant information EC 1 from the face images of persons. The details of the personal registering part 41 will now be described.
  • the detailed configuration of the personal registering part 41 will be described with reference to FIG. 7 .
  • the personal registering part 41 has a registering part 51 in addition to the three-dimensional reconstructing part 21 , face information generating part 22 , optimizing part 23 , correcting part 24 , model perfection level computing part 25 , feature extracting part 26 , and information compressing part 27 .
  • the three-dimensional reconstructing part 21 , face information generating part 22 , optimizing part 23 , correcting part 24 , model perfection level computing part 25 , feature extracting part 26 , and information compressing part 27 have functions similar to the corresponding functions in the controller 10 b.
  • the registering part 51 has the function of registering, as the registrant information EC 1 of the person HMa to be registered, the three-dimensional and two-dimensional face feature amounts (feature information) compressed and generated by the information compressing part 27 and the perfection level of an individual model into the person information storage 34 .
  • information registered in the person information storage 34 in the controller 10 a is also transferred and stored to the person information storage 34 in the controller 10 b by proper synchronous process (overwriting process or the like). Operation of Face Verification System SYS 2
  • the authenticating operation realized by the face verification system SYS 2 will be described.
  • the case of actually authenticating a predetermined person photographed by the cameras CA 1 and CA 2 as the person HMb to be authenticated will be described.
  • Three-dimensional shape information measured on the basis of the principle of triangulation using images captured by the cameras CA 1 and CA 2 is used as the three-dimensional information, and texture (brightness) information is used as the two-dimensional information.
  • FIGS. 8 and 9 are a flowchart of the general operation of the controller 10 b .
  • FIG. 10 is a diagram in which images continuously captured by the two cameras are displayed in time series.
  • FIG. 11 is a diagram showing feature points of a feature part in a face image.
  • FIG. 12 is a schematic diagram showing a state where three-dimensional coordinates are calculated by using the principle of triangulation from feature points in two-dimensional images.
  • Reference numeral G 1 in FIG. 12 indicates an image G 1 captured by the camera CA 1 and input to the controller 10 b .
  • Reference numeral G 2 indicates an image G 2 captured by the camera CA 2 and input to the controller 10 b .
  • Points Q 20 in the images G 1 and G 2 correspond to a point at the right end of a mouth in FIG. 11 .
  • FIG. 13 is a diagram showing a predetermined region RF around the feature point Q 20 as a center.
  • the face information EB 2 of the person HMb to be authenticated and model perfection level of the individual model is generated (updated) on the basis of stereo images of the person HMb to be authenticated sequentially (repeatedly) captured at different time points.
  • an individual model is generated on the basis of the face information EB 2 , and the face feature amount of the person HMb to be authenticated and model perfection level of the individual model are generated as the information EC 2 of the person to be authenticated.
  • the face authentication of the person HMb to be authenticated is performed using the information EC 2 of the person to be authenticated.
  • the processes in the controller 10 b will be described in detail hereinbelow.
  • steps SP 1 to SP 8 also called “face information generating process”.
  • face information generating process steps SP 1 to SP 8
  • the face information EB 2 of the person HMb to be authenticated used for model fitting which will be described later is generated on the basis of the face images of the person HMb to be authenticated repeatedly captured at different timings.
  • the face information generating process (steps SP 1 to SP 8 ) is a loop process sequentially and repeatedly executed every stereo images (two images) input in time series at different timings until “end” is determined in a predetermined determining process (step SP 8 ) which will be described later.
  • two face images G 1 (ti+1) and G 2 (ti+1) are newly captured and input to the controller 10 b .
  • face information corrected face information
  • face information is generated using the face information at the time T(ti) and the face information based on the images captured at the time T(ti+1).
  • face information (face information based on the images captured at time T(ti+1)) is generated on the basis of the two face images G 1 (ti+1) and G 2 (ti+1) captured at the time T(ti+1).
  • face information at the time T(ti+1) is generated.
  • the face information at the time T(ti+1) is generated by reflecting the face information at the time T(ti) in the face information based on the images captured at the time T(ti+1).
  • face information (face information based on the images captured at the time T(ti+2)) is generated on the basis of the two face images G 1 (ti+2) and G 2 (ti+2) captured at the time T(ti+2).
  • face information at the time T(ti+2) is generated.
  • the face information at the time T(ti+2) is generated by reflecting the face information at the time T(ti+1) in the face information based on the images captured at the time T(ti+2).
  • each time new face images are captured by sequentially updating face information using face information generated in the past and face information generated from the new images, very reliable face information (corrected face information) can be generated.
  • step SP 1 face images (G 1 and G 2 ) of a predetermined person (person to be registered) captured by the cameras CA 1 and CA 2 are input to the controller 10 b via a communication line.
  • Each of the cameras CA 1 and CA 2 for capturing face images is a general image capturing apparatus capable of capturing a two-dimensional image.
  • the camera parameter Bi will be described later.
  • step SP 2 an area in which the face exists is detected from each of the two images (G 1 and G 2 ) input from the cameras CA 1 and CA 2 .
  • a face area detecting method for example, a method of detecting a face area from each of the two images by template matching using a prepared standard face image can be employed.
  • step SP 3 the position of a feature part in the face is detected from the face area image detected in step SP 2 .
  • the feature parts in the face are eyes, eyebrows, nose, mouth, and the like.
  • step SP 3 the coordinates of feature points Q 1 to Q 23 of the parts as shown in FIG. 11 are calculated.
  • a feature part can be detected by template matching using a standard template of the feature part.
  • the coordinates of a feature point calculated are expressed as coordinates on the images G 1 and G 2 input from the cameras. For example, with respect to the feature point Q 20 corresponding to the right end of the mouth in FIG. 11 , as shown in FIG. 12 , coordinate values in the two images G 1 and G 2 are calculated, respectively.
  • a brightness value of each of pixels in an area using, as an apex point, a feature point in an input image is obtained as information of the area (hereinbelow, also called “texture information”).
  • the texture information in each area is pasted (mapped) to a modified individual model in step SP 9 or the like which will be described later.
  • the number of input images is two, so that an average brightness value in corresponding pixels in corresponding areas in the images is used as the texture information of the area.
  • step SP 4 three-dimensional reconstructing process
  • m denotes the number of feature points.
  • a camera parameter matrix Bi indicates values peculiar to each camera, which are obtained by capturing an image of an object whose three-dimensional coordinates are known, and is expressed by a projection matrix of 3 ⁇ 4.
  • Equation (2) shows the relation between coordinates (x1, y1) at the feature point Q 20 on the image G 1 and three-dimensional coordinates (x, y, z) when the feature point Q 20 is expressed in a three-dimensional space.
  • Equation (3) shows the relation between the coordinates (x2, y2) at the feature point Q 20 on the image G 2 and the three-dimensional coordinates (x, y, z) when the feature point Q 20 is expressed in a three-dimensional space.
  • Unknown parameters in Equations (2) and (3) are five parameters in total; two parameters ⁇ 1 and ⁇ 2 and three component values x, y, and z of three-dimensional coordinates Ms (20) .
  • the number of equalities included in Equations (2) and (3) is six, so that each of the unknown parameters, that is, three-dimensional coordinates (x, y, z) at the feature point Q 20 can be calculated.
  • three-dimensional coordinates Ms (j) at all of feature points Qj can be obtained.
  • step SP 5 the reliability of three-dimensional coordinates (face information) at each feature point is calculated.
  • a correlation value between corresponding areas (regions) in two images, each of the corresponding areas having corresponding feature points as a center is used as an evaluation value H (j) of reliability.
  • H (j) of reliability For example, in the case of calculating the reliability of the feature point Q 20 , as shown in FIG. 13 , the correlation value F (20) between predetermined areas RF (region having 5 ⁇ 5 pixels) each using the feature point Q 20 as a center in two images is calculated.
  • Equation (4) the differential absolute value of pixel signals (luminance information) of corresponding two pixels in each of 25 pairs is computed, and the inverse of the average value of 25 differential absolute values can be used as a correlation value F (j) .
  • u denotes the number of pixels
  • G 1 and G 2 express luminance values of corresponding pixels in each image.
  • the small letter “r” indicates the r-th pixel.
  • Such a correlation value F (j) becomes a large value in the case where corresponding predetermined areas RF include the same part of the subject, and becomes a small value in the case where the predetermined areas RF are different parts. That is, when the correlation value F (j) is large, the reliability of coordinate information of the feature point is high. When the correlation value F (j) is small, the reliability of coordinate information of the feature point is low.
  • steps SP 1 to SP 5 the face information (the three-dimensional coordinates Ms (j) at each of the feature points Qj in the face) of the person HMb to be authenticated and the reliability of the face information is generated on the basis of two images input.
  • the face information generating process (steps SP 1 to SP 8 ) is a loop process executed on every stereo images (two images) repeatedly input at different timings.
  • the process (steps SP 1 to SP 5 ) is executed for each input stereo image, and the face information and the reliability of the face information is generated for each input stereo image.
  • steps SP 6 to SP 8 are repeatedly executed on the basis of the face information and the reliability of the face information generated for each input stereo image.
  • step SP 6 predetermined statistic process is executed on the three-dimensional coordinates Ms (j) of each feature point Qj of each stereo image calculated in step SP 4 and the reliability at each feature point of each stereo image calculated in step SP 5 , thereby generating corrected face information (corrected three-dimensional coordinates Mm (j) at each of the feature points in the face) of the person HMb to be authenticated.
  • the corrected three-dimensional coordinates Mm (j) at each feature point are obtained by calculating weighted mean of the three-dimensional coordinates Ms (j) at the feature points of the stereo images by using the reliability at the feature points of the stereo images.
  • step SP 6 texture information is obtained sequentially so as to compensate a region which has not obtained texture information among regions each having, as an apex, a feature point of a face by using the texture information obtained in step SP 3 for each of the stereo images repeatedly input.
  • texture information obtained from a stereo image which was input in the past is held for each region, in the case where texture information of a region (unobtained region) which has not obtained texture information yet from a stereo image which was input in the past is newly obtained from a newly input stereo image in step SP 3 , the texture information is held as texture information of the region.
  • model perfection level Hp of an individual model is calculated on the basis of the reliability evaluation value H (j) indicative of reliability of the face information (the three-dimensional coordinates at each feature point in the face) calculated in step SP 5 .
  • the model perfection level Hp of an individual model is calculated by accumulating the reliability evaluation values H (j) at the feature points in the stereo images in time series as shown by Equation (6), and further adding the reliability evaluation values at all of feature points.
  • the model perfection level H (j) calculated on the basis of the reliability evaluation values H (j) of three-dimensional coordinates at the feature points can be also expressed as an evaluation value indicative of reliability of three-dimensional shape information extracted from an individual model in step SP 11 which will be described later.
  • step SP 8 whether the face information generating process is finished and the model fitting process as the following step and similarity calculating process as subsequent process are started (executed) or not is determined.
  • a determining method a method of determining whether the model perfection level Hp exceeds a preset threshold TH 1 or not can be employed.
  • measurement information input information
  • face information corrected face information
  • Unstable authentication based on insufficient face information can be avoided.
  • the system immediately shifts to the following process, so that authentication with high time efficiency at the required authentication accuracy can be performed.
  • the very reliable face information (corrected face information) EB 2 can be generated.
  • step SP 9 to SP 12 feature information adapted to authentication (also called “authentication information”) is generated on the basis of the corrected face information EB 2 and the like.
  • model fitting is performed using the corrected face information EB 2 .
  • the “model fitting” is a process of generating an “individual model” in which measurement information of the face of the person HMb to be authenticated is reflected by modifying a “standard model (of a face)” as a prepared stereoscopic model of a general (standard) face by using the face information (corrected face information) of the person HMb to be authenticated. Concretely, a process of changing three-dimensional information of the standard model by using the calculated corrected three-dimensional coordinates Mm (j) and a process of changing two-dimensional information of the standard model by using the texture information are performed.
  • FIG. 14 is a diagram showing a standard model of a three-dimensional face.
  • FIG. 15 is a diagram showing texture information.
  • the standard model of a face shown in FIG. 14 is constructed by vertex data and polygon data and stored as the three-dimensional model database 32 ( FIG. 5 ) in the storage 3 or the like.
  • the vertex data is a collection of coordinates of a vertex (hereinbelow, also called “standard control point”) COj of a feature part in the standard model and corresponds to the corrected three-dimensional coordinates at each feature point Qj calculated in step SP 6 in a one-to-one correspondence manner.
  • the polygon data is obtained by dividing the surface of the standard model into small polygons (for example, triangles) and expressing the polygons as numerical value data.
  • FIG. 14 shows the case where the vertex of a polygon is constructed also by an intermediate point other than the standard control point COj. The coordinates at an intermediate point can be obtained by a proper interpolating method.
  • the vertex (standard control point COj) of each of feature parts of the standard model is moved to a feature point calculated in step SP 4 .
  • the corrected three-dimensional coordinate value at each feature point Qj is substituted as a three-dimensional coordinate value of the corresponding standard control point COj, thereby obtaining a standard control point (hereinbelow, also called “individual control point”) Cj after movement.
  • the standard model can be modified to an individual model expressed by the corrected three-dimensional coordinates Mm (j) .
  • a positional change of the individual model with respect to the standard model can be obtained by a deviation amount between a predetermined reference position in the standard model and a corresponding reference position in an individual model modified. From a deviation amount between a reference vector connecting predetermined two points in the standard model and a reference vector connecting points corresponding to the predetermined two points in the modified individual model, a change in the tilt and a scale change with respect to the standard model, of the individual model can be obtained.
  • the position of the individual model can be obtained. Further, by comparing the middle point QM with another feature point, the scale and tilt of the individual model can be calculated.
  • Equation (7) expresses a transformation parameter (vector) vt expressing the correspondence relation between the standard model and the individual model.
  • the transformation parameter (vector) vt is a vector having, as its elements, scale transformation index sz between the standard model and the individual model, transformation parameters (tx, ty, tz) indicative of translation displacements in three orthogonal axes, and translation parameters ( ⁇ , ⁇ , ⁇ ) indicative of a rotation displacement (tilt).
  • vt ( sz, ⁇ , ⁇ , ,tx,ty,tz ) T (7) where T expresses transposition.
  • T expresses transposition.
  • the process of changing the three-dimensional information of the standard model using the corrected three-dimensional coordinates Mm (j) related to the person HMb to be authenticated is performed.
  • process of changing the two-dimensional information of the standard model by using the texture information is performed.
  • the texture information of each region using, as a vertex, a feature point of the face obtained from a stereo image in step SP 3 and held in step SP 6 is pasted (mapped) to corresponding regions (polygons) on the three-dimensional individual model.
  • Each region (polygon) to which the texture information is pasted on a stereoscopic model (such as individual model) is also called a “patch”.
  • step SP 9 the model fitting process
  • step SP 10 the individual model is corrected on the basis of the standard model as a reference.
  • an alignment correction and a shading correction are made.
  • the alignment correction is a correcting process for three-dimensional information
  • the shading correction is a correcting process for two-dimensional information.
  • the alignment (face orientation) correction is performed on the basis of the scale, tilt, and position of the individual model using the standard model as a reference. More specifically, by performing coordinate conversion on the individual control point of the individual model using the conversion parameter vt (refer to Expression 7) indicative of the relation between the standard model as a reference and the individual model, a three-dimensional face model having the same posture as that of the standard model can be created. That is, by the alignment correction, the three-dimensional information of the person HMb to be authenticated can be properly normalized.
  • the shading correction is a process for correcting a brightness value (texture information (refer to FIG. 15 )) of each of the pixels in a patch mapped to the individual model.
  • the shading correction the difference in the texture information between the models (the standard model and the individual model) can be corrected, which occurs in the case where the positional relation between a light source and the subject at the time of capturing an image of a person for forming a standard model and that at the time of capturing an image of a person of the individual model (at the time of capturing an image of a person to be authenticated) are different from each other. That is, by the shading correction, the texture information as one of the two-dimensional information of the person to be authenticated can be normalized properly.
  • step SP 10 information of the person HMb to be authenticated is generated in a normalized state as an individual model including both three-dimensional information and two-dimensional information of the person HMb to be authenticated.
  • step SP 11 as information indicative of features of the person HMb to be authenticated, three-dimensional shape information (three-dimensional information) and texture information (two-dimensional information) is extracted.
  • a three-dimensional coordinate vector of m pieces of the individual control points Cj in the individual model is extracted.
  • h s ( X 1 , . . . , Xm,Y 1 , . . . , Ym,Z 1 , . . . , Zm ) T (8)
  • texture (brightness) information of a patch or a group (local area) of patches (hereinbelow, also called “local two-dimensional information”) near a feature part, that is, an individual control point in the face, which is important information for person authentication is extracted.
  • texture information local two-dimensional information
  • information mapped to the sub model is used.
  • the local two-dimensional information is comprised of, for example, brightness information of pixels of local areas such as an area constructed by a group GR in FIG. 16 indicative of individual control points of a feature part after normalization (a patch R 1 having, as vertexes, individual control points C 20 , C 22 , and C 23 and a patch R 2 having, as vertexes, individual control points C 21 , C 22 , and C 23 ), an area constructed only by a single patch, or the like.
  • L denotes the number of local areas
  • h (k) ( BR 1, . . . , BRn ) T (9)
  • step SP 11 the three-dimensional shape information (three-dimensional information) and the texture information (two-dimensional information) is extracted as information indicative of a feature of the individual model from the individual model.
  • step SP 12 information compressing process for converting the information extracted in step SP 11 to information adapted to authentication is performed.
  • the information compressing process is performed by using similar methods on the three-dimensional shape information h S and each local two-dimensional information h (k) .
  • the case of performing the information compressing process on the local two-dimensional information h (k) will be described in detail.
  • the local two-dimensional information h (k) can be expressed in a basis decomposition form as shown by Expression (10) using average information (vector) h ave (k) of the local area preliminarily obtained from a plurality of sample face images and a matrix P (k) (which will be described below) expressed by a set of eigenvectors of the local area preliminarily calculated by performing KL expansion on the plurality of sample face images.
  • P (k) which will be described below
  • local two-dimensional face information (vector) c (k) is obtained as compression information of the local two-dimensional information h (k) .
  • h (k) h ave (k) +P (k) c (k) (10)
  • the matrix P (k) in Expression (10) is calculated from a plurality of sample face images.
  • the matrix P (k) is calculated as a set of some eigenvectors (basis vectors) having large eigenvalues among a plurality of eigenvectors obtained by performing the KL expansion on the plurality of sample face images.
  • the basis vectors are stored in the compressed information storage 33 .
  • the case where local two-dimensional information h (GR) of a local area constructed by a group GR shown in FIG. 16 is expressed in a basis decomposition form will be considered.
  • the local two-dimensional information h (GR) is expressed as Expression (11) using average information h ave (GR) of the local area and three eigenvectors P 1 , P 2 , and P 3 .
  • the average information h ave is a vector obtained by averaging a plurality of pieces of local two-dimensional information (vectors) of various sample face images on each corresponding factor.
  • vectors the plurality of sample face images. It is sufficient to use a plurality of standard face images having proper variations.
  • h ( GR ) h avg ( GR ) + ( P ⁇ ⁇ 1 P ⁇ ⁇ 2 P ⁇ ⁇ 3 ) ⁇ ( c ⁇ ⁇ 1 c ⁇ ⁇ 2 c ⁇ ⁇ 3 ) ( 11 )
  • the face information c (GR) is information obtained by compressing the local two-dimensional information h (GR) of the local area constructed by the group GR.
  • the local two-dimensional face information c (GR) obtained as described above can be used as it is for authentication, in the embodiment, the information is further compressed.
  • a process of converting a feature space expressed by the local two-dimensional face information c (GR) to a subspace which increases the differences among persons is performed in addition. More specifically, a transformation matrix A is considered which reduces the local two-dimensional face information c (GR) of vector size “f” to the local two-dimensional feature amount (vector) d (GR) of vector size “g” as shown by Expression (12).
  • the feature space expressed by the local two-dimensional face information c (GR) can be converted to a subspace expressed by the local two-dimensional feature amount d (GR) .
  • d (GR) A T c (GR) (12)
  • the transformation matrix A is a matrix having the size of f ⁇ g. By selecting “g” pieces of main components having high ratio (F ratio) between within-class variance and between-class variance from the feature space by using multiple discriminant analysis (MDA), the transformation matrix A can be determined.
  • local two-dimensional face feature amounts d (k) of the local areas can be obtained.
  • a three-dimensional face feature amount ds can be obtained.
  • a face feature amount “d” obtained by combining the three-dimensional face feature amount d S and the local two-dimensional face feature amount d (k) calculated in the step SP 12 can be expressed in a vector form by Expression (13).
  • the face feature amount “d”, that is, the feature information of the person HMb to be authenticated is obtained from input face images of the person HMb to be authenticated.
  • face authentication of the person HMb to be authenticated is performed using the information EC 2 of the person to be authenticated (the face feature amount “d” or the like).
  • step SP 13 overall similarity Re as similarity between the person HMb to be authenticated and the person HMa to be compared (a person to be registered) is calculated (step SP 13 ).
  • step SP 14 a comparing operation between the person HMb to be authenticated and the person to be compared on the basis of the overall similarity Re is performed (step SP 14 ).
  • the overall similarity Re is calculated using weight factors specifying weights on three-dimensional similarity Re S and local two-dimensional similarity Re (k) (hereinbelow, also simply called “weight factors”) in addition to the three-dimensional similarity Re S calculated from the three-dimensional face feature amount d S and local two-dimensional similarity Re (k) calculated from the local two-dimensional face feature amount d (k) .
  • weight factors WT and WS predetermined values are used.
  • step SP 13 similarity evaluation is conducted between the face feature amount (comparison feature amount) of a person to be compared, which is pre-registered in the person information storage 34 and the face feature amount of the person HMb to be authenticated, which is calculated in the steps SP 1 to SP 12 .
  • the similarity calculation is executed between the registered face feature amount (comparison feature amount) (d SM and d (k)M ) and the face feature amount (d SI and d (k)I ) of the person HMb to be authenticated, and the three-dimensional similarity Re S and the local two-dimensional similarity Re (k) is calculated.
  • the face feature amount of the person to be compared (the person HMa to be registered) in face authentication is preliminarily obtained in the face registration system SYS 1 executed prior to the operation of the face verification system SYS 2 .
  • Re S ( d SI ⁇ d SM ) T ( d SI ⁇ d SM ) (14)
  • the local two-dimensional similarity Re (k) is obtained by calculating Euclidean distance Re (k) of each of vector components of the feature amounts in the corresponding local regions as shown by Equation (15).
  • Re (k) ( d (k)I ⁇ d (k)M ) T ( d (k)I ⁇ d (k)M ) (15)
  • Equation (16) the three-dimensional similarity Re S and the local two-dimensional similarity Re (k) are combined by using weight factors WT and WS. In such a manner, the overall similarity Re as similarity between the person HMb to be authenticated and the person to be compared (person HMa to be registered) is obtained.
  • step SP 14 authentication determination is performed on the basis of the overall similarity Re.
  • the authentication determining method varies between the case of face verification and the case of face identification as follows.
  • an input face (the face of the person HMb to be authenticated) is that of a specific registrant or not. Consequently, by comparing the overall similarity Re of the face feature amount of the specific registrant, that is, the person to be compared (comparison feature amount) with a predetermined threshold TH 2 , similarity between the person HMb to be authenticated and the person to be compared is determined. Specifically, when the overall similarity Re is smaller than the predetermined threshold TH 2 , the similarity between the person HMb to be authenticated and the person to be compared is high, and it is determined that the person HMb to be authenticated and the person to be compared are the same person.
  • the face identification is to identify a person as the owner of an input face (the face of the person HMb to be authenticated).
  • similarity between the face feature amount of each of the persons registered and the face feature amount of the person HMb to be authenticated is calculated, thereby determining coincidence between the person HMb to be authenticated and each of the persons to be compared.
  • the person to be compared having the highest coincidence among the plurality of persons to be compared is determined as the same person as the person HMb to be authenticated.
  • the person to be compared corresponding to the minimum similarity Re min among various similarities Re of the person HMb to be authenticated and a plurality of persons to be compared is determined as the same person as the person HMb to be authenticated.
  • step SP 14 the authentication determination is made on the basis of the overall similarity Re.
  • the controller 10 b in the embodiment executes the authenticating operation to determine whether the person to be compared is the same person as the person to be authenticated or not in consideration of both of the model perfection level Hp and the overall similarity Re in the processes of the steps SP 1 to SP 14 .
  • the model perfection level Hp in which reliability of face information (corrected face information) is reflected in addition to the overall similarity Re for the authenticating operation, accurate authentication in which the reliability of information used for authentication is also reflected can be performed.
  • the process does not proceed into step SP 9 and the subsequent steps.
  • the persons HMb and HMa are determined as the same person.
  • the model perfection level Hp is higher than the threshold TH 1 in step SP 8 and the overall similarity Re is lower than the threshold TH 2 in step SP 14 , the persons HMb and HMa are determined as the same person.
  • the model perfection level Hp and the overall similarity Re are considered as follows.
  • the condition that the model perfection level Hp exceeds the threshold TH 1 is used as the condition of executing the similarity calculating operation.
  • the similarity calculating operation (step SP 13 ) is executed. Consequently, accurate authenticating operation in which the reliability of the information used for authentication is also sufficiently reflected can be performed.
  • model perfection level is lower than the threshold TH 1 , the similarity calculating operation is not executed, so that unstable authentication can be avoided.
  • the operation of extracting the feature information of the person HMb to be authenticated from the individual model (step SP 11 ) is executed. Therefore, when the model perfection level Hp is less than the threshold TH 1 , the extracting operation is not executed, so that unstable authentication can be avoided.
  • the threshold TH 1 may be set according to an authentication level (authentication accuracy) required for the face authentication system 1 A.
  • an authentication level authentication accuracy
  • the predetermined threshold TH 1 is set to a relatively large value.
  • high-accuracy authentication can be achieved.
  • the authenticating operation executed to display an operation panel dedicated to a specific person in the copying machine high authentication accuracy is not requested for the face authentication system 1 A, so that the predetermined threshold TH 1 is set to a relatively small value.
  • the threshold TH 1 in accordance with the required authentication level, at the time of generating an individual model of the person HMb to be authenticated, face information is obtained repeatedly until an individual model corresponding to the requested authentication level (authentication accuracy) is generated. Consequently, information acquisition and authentication according to required authentication accuracy can be realized.
  • the authenticating operation can be performed according to various scenes such as the case where high authentication level is not required but high authentication speed is required or the case where high authentication speed is not required but high-accuracy authentication is required.
  • the controller 10 b in the embodiment updates and registers the feature information of the person HMb to be authenticated as feature information of the person HMa to be compared. That is, the controller 10 b updates the registrant information EC 1 by performing the processes (steps SP 15 to SP 18 ) as shown in FIG. 9 .
  • the model perfection level included in the registrant information EC 1 of the person to be compared determined as the same person as the person HMb to be authenticated in step SP 14 is compared with the model perfection level included in the information EC 2 of the person HMb to be authenticated, which is generated by the controller 10 b . Which one of the information is more proper as authentication information is determined, and the registrant information EC 1 is updated.
  • step SP 15 the process proceeds into step SP 16 (step SP 15 ).
  • step SP 16 the model perfection level Hp included in the information EC 2 of the person to be authenticated (that is, the model perfection level Hpb of the person HMb to be authenticated) is compared with the model perfection level Hp included in the registrant information EC 1 of the person to be compared who is determined as the same person (that is, the model perfection level Hpa of the person HMa to be authenticated).
  • step SP 17 When it is determined that the model perfection level Hpb included in the information EC 2 of the person to be authenticated is higher than the model perfection level Hpa included in the registrant information EC 1 (step SP 17 ), the registrant information EC 1 registered in the person information storage 34 is changed (updated) (step SP 18 ). That is, the process of changing (updating) the registrant information EC 1 registered in the person information storage 34 to the information EC 2 of the person to be authenticated is performed.
  • step SP 15 determines whether a person to be compared who is the same as the person to be authenticated does not exist, or in the case where it is determined in step SP 17 that the registrant information EC 1 is not updated, the registrant information EC 1 is not updated and is held as it is in the person storage 34 .
  • the registrant information used for the authentication is compared with the authenticator information, and the registrant information EC 1 is changed (updated). Consequently, each time the authenticating operation is performed, information of higher accuracy can be assured in the person information storage 34 .
  • the operation of the face registration system SYS 1 will be described. Concretely, the case of registering a predetermined person photographed by the cameras CA 1 and CA 2 as the person HMa to be registered will be described. Three-dimensional shape information measured on the basis of the principle of triangulation using images captured by the cameras CA 1 and CA 2 is used as the three-dimensional information, and texture (brightness) information is used as the two-dimensional information. In the following description, the points different from the operation of the face verification system SYS 2 will be described mainly. The same reference numerals are designated to common parts and their description will not be repeated.
  • the controller 10 a in the face registration system SYS 1 on the basis of the stereo images (measurement information) EA 1 of the person HMa to be registered which are repeatedly captured at different timings, the feature information of the person HMa to be registered and the model perfection level of the individual model GM 1 is obtained and registered as the registrant information EC 1 of the person HMa to be registered in the person information storage 34 .
  • the registrant information EC 1 registered in the person information storage 34 is used at the time of authentication in the face verification system SYS 2 and the like.
  • FIG. 17 is a flowchart showing operations of the controller 10 a in the face registration system SYS 1 .
  • three-dimensional coordinates at feature points in the face of the person HMa to be registered are calculated on the basis of stereo images (measurement information) obtained by sequentially (repeatedly) photographing the face of the person HMa to be registered at different time points (step SP 6 ).
  • the model perfection level Hp of the individual model is calculated in step SP 7 .
  • step SP 8 When it is determined in step SP 8 that the face information (corrected face information) EB 1 required for generation of the registrant information EC 1 is obtained, the face information generating process (steps SP 1 to SP 8 ) is finished, and the process proceeds into step SP 9 .
  • step SP 9 an individual model is generated from a standard model of a face on the basis of three-dimensional coordinates at the feature points in the face.
  • step SP 10 a fluctuation correction in the individual model is executed.
  • step SP 11 three-dimensional shape information (three-dimensional information) and texture information (two-dimensional information) is extracted from the corrected individual model.
  • the resultant information is further subjected to step SP 12 , thereby obtaining a face feature amount (feature information) d of the person HMa to be registered.
  • step SP 21 the generated feature information (face feature amount d) of the person HMa to be registered and the model perfection level Hp of the individual model from which the face feature amount d is extracted is registered as information used for authentication (registrant information EC 1 ) into the person information storage 34 .
  • the registrant information EC 1 of the person HMa to be registered which is registered in the person information storage 34 in the face registration system SYS 1 is used for the authenticating operation in the face verification system SYS 2 .
  • the face verification system SYS 2 using information for face authentication the feature information extracted from the individual model and the model perfection level Hp based on the reliability of the face information is registered as information for face authentication on the person HMa to be registered.
  • the configuration of the face authentication system 1 B in the second embodiment is similar to that in the first embodiment.
  • the same reference numerals are designated to elements having functions similar to those in the first embodiment and their description will not be repeated.
  • the face information EB 1 (or EB 2 ) according to required authentication accuracy is generated on the basis of stereo images repeatedly obtained at different timings, and the model fitting process is performed by using the face information EB 1 (or EB 2 ).
  • the model fitting process is performed by using the face information EB 1 (or EB 2 ).
  • the case of generating (updating) face information each time a stereo image is input and, each time the face information is updated, executing the model fitting process by using the face information to update the model will be described.
  • FIGS. 18 and 19 are flowcharts showing operations of the controller 10 b in the face verification system SYS 2 in the second embodiment.
  • steps SP 51 to SP 57 processes similar to the steps SP 1 to SP 7 are performed.
  • face information is generated for each of stereo images of the person HMb to be authenticated which are input repeatedly at different timings (step SP 54 ), and update of face information in which face information generated in the past is reflected is sequentially executed (step SP 56 ).
  • step SP 57 on the basis of the reliability of the three-dimensional coordinates at each of the feature points sequentially obtained in step SP 55 , the model perfection level Hp of an individual model is calculated.
  • step SP 58 on the basis of the face information (corrected face information) generated sequentially (successively) in step SP 56 , the model is updated by the model fitting each time (successively).
  • the model fitting a process of updating three-dimensional information and a process of updating two-dimensional information are performed as described above.
  • the model fitting method a method similar to that in the first embodiment may be used.
  • step SP 58 in the second embodiment model fitting in which movement of a semi-control point CSv (which will be described later) newly set for the standard model is also considered is performed.
  • the process of changing two-dimensional information is performed in a manner similar to that in the first embodiment.
  • step SP 59 the continuous model updating process in steps SP 51 to SP 58 is finished and whether the process proceeds into the next process (step SP 60 ) or not is determined.
  • the determining method a method similar to that of the step SP 8 , that is, a method of determining whether the model perfection level Hp exceeds the preset threshold TH 1 or not can be employed. By the method, until an individual model necessary for required authentication accuracy is generated, measurement information can be obtained repeatedly.
  • step SP 59 When it is determined in step SP 59 that the processes in steps SP 51 to SP 58 are finished, the process proceeds into step SP 60 .
  • steps SP 60 to SP 62 processes similar to those in the steps SP 10 to SP 12 ( FIG. 8 ) are executed.
  • a fluctuation correction is executed on the individual model obtained in the processes in the steps SP 51 to SP 59 .
  • step SP 61 three-dimensional shape information (three-dimensional information) and texture information (two-dimensional information) is extracted from the corrected individual model.
  • step SP 62 a predetermined information compressing process is executed, thereby generating the face feature amount (feature information) d of the person HMa to be authenticated.
  • steps SP 63 and SP 64 processes similar to shoe in the steps SP 13 and SP 14 ( FIG. 8 ) are executed, and face authentication of the person HMb to be authenticated is performed by using the information EC 2 (such as the face feature amount (feature information) d) of the person HMa to be authenticated.
  • the information EC 2 such as the face feature amount (feature information) d
  • overall similarity Re as similarity between the person HMb to be authenticated and the person HMa to be compared (person HMa to be registered) is calculated (step SP 13 ).
  • a comparing operation authentication determination between the person HMb to be authenticated and the person to be compared is performed on the basis of the overall similarity Re (step SP 14 ).
  • steps SP 65 to SP 68 shown in FIG. 19 processes similar to those of the steps SP 15 to SP 18 are executed. Briefly, the model perfection level Hpa included in the registrant information EC 1 of the person to be compared determined as the same person as the person HMb to be authenticated is compared with the model perfection level Hpb included in the information EC 2 of the person HMb to be authenticated, which is generated by the controller 10 b , and the registrant information EC 1 registered in the person information storage 34 is changed (updated).
  • FIG. 20 is a flowchart showing the model fitting in which the semi-control points are considered.
  • FIG. 21 is a diagram showing semi-control points CSv on a standard model.
  • FIG. 22 is a diagram showing correspondence between a three-dimensional position in an individual model of semi-control points and a two-dimensional position on two images.
  • the semi-control point CSv is a point which is useful as individual identification information like a wrinkle appearing below an eye or between the nose and the mouth but is provided in a portion whose position is difficult to be specified more than the standard control point COj.
  • the model fitting in which the semi-control points CSv are considered is realized by performing processes in steps SP 71 to SP 78 shown in FIG. 20 .
  • step SP 71 a control point is moved by a method similar to that in the first embodiment.
  • corrected three-dimensional coordinates Mm (j) of each of the feature points Qj updated in step SP 56 are substituted as new three-dimensional coordinates for a corresponding individual control point Cj of an individual model generated by the model fitting of last time.
  • each of the semi-control points CSv on the individual model also moves.
  • step SP 72 the three-dimensional coordinates of the semi-control point CSv moved in association with movement of the individual control point Cj are calculated by a proper interpolating method using the three-dimensional coordinates of each of the individual control points Cj.
  • step SP 73 two-dimensional coordinates of each of the semi-control points in the two images (in this case, the images G 3 and G 4 ) input in the step SP 51 are calculated on the basis of the three-dimensional coordinates of each of the semi-control points CSv. Specifically, as shown in FIG. 22 , by performing reverse operation using the equation (1) from the three-dimensional coordinates of the semi-control points CSv, the two-dimensional coordinates of the semi-control point CSv in each of the two images are calculated.
  • step SP 74 correlation value computation on a predetermined area using the semi-control point CSv as a center is performed between two images every semi-control point CSv.
  • a predetermined area RS (for example, an area having 5 ⁇ 5 pixels) having the semi-control point CSv as a center on the image G 3 is cut out and, while shifting the predetermined area RS by narrow width (for example, one pixel by one pixel) around the semi-control point CSv on the image G 4 , the correlation value computation is performed. Calculation of a correlation value is performed by using the equation (4), and an area including the position in which the correlation value is the largest, that is, the same part the most is specified.
  • step SP 75 the position in which the correlation value is the largest coincides with the position of the semi-control point CSv on the image G 4 calculated in the step SP 73 or not is determined. If YES, the model fitting is finished. On the other hand, if NO, the process proceeds into step SP 76 .
  • the coordinates of the semi-control point CSv on the image G 4 are corrected. Concretely, a correction of setting the center position of the predetermined area RS in the position (coordinates) at which the correlation value is the largest on the image G 4 as a two-dimensional coordinate position of the semi-control point CSv on the image G 4 is made.
  • step SP 77 three-dimensional coordinates of the semi-control point CSv are newly calculated by using the equation (1) from the two-dimensional coordinates of the semi-control point CSv on the image G 3 and the corrected two-dimensional coordinates of the semi-control point CSv on the image G 4 .
  • step SP 78 the three-dimensional coordinates of the semi-control point CSv newly calculated are substituted for the semi-control point CSv on the individual model, thereby deforming the model.
  • the model fitting in which movement of the semi-control point CSv is considered is performed.
  • an individual model in which the face shape of the person HMb to be authenticated is reflected more can be generated.
  • FIG. 23 is a flowchart showing operations of the controller 10 a in the face registration system SYS 1 of the second embodiment.
  • processes similar to the steps SP 1 to SP 12 are executed on the basis of stereo images (registration images) of the person HMa to be registered which are repeatedly captured at different timings.
  • the face feature amount “d” of the person HMa to be registered and the model perfection level Hp (obtained in the step SP 57 ) of the individual model from which the face feature amount “d” is extracted are generated as the registrant information EC 1 .
  • step SP 81 a process similar to that in the step SP 21 ( FIG. 17 ) is performed.
  • the generated feature information (face feature amount “d”) of the person HMa to be registered and the generated model perfection level Hp is registered as information (registrant information EC 1 ) used for the authenticating operation into the person information storage 34 .
  • the process of determining shift to the next process (steps SP 9 and SP 60 ) is executed on the basis of the model perfection level Hp in steps SP 8 and SP 59 , and the authentication determination is conducted on the basis of the overall similarity Re in steps SP 14 and SP 64 .
  • the invention is not limited to the embodiment.
  • the person HMb to be authenticated and the person HMa to be compared may be determined as the same person.
  • the authentication determination in which both of the model perfection level Hp and the overall similarity Re are considered may be executed in step SP 14 .
  • various methods can be employed as the method of determining the shift to the next process (steps SP 9 and SP 60 ) executed in the steps SP 8 and SP 59 .
  • a method of determining whether shift to the next process (steps SP 9 and SP 60 ) is made or not depending on whether time lapsed from image capture start exceeds predetermined time or not may be used.
  • the shift determination (steps SP 8 and SP 59 ) is not performed, but shift may be simply made to the next process (steps SP 9 and SP 60 ) each time a stereo image is input.
  • the authenticating operation of executing an authentication determination in which both the model perfection level Hp and the overall similarity Re is considered is also expressed as an operation that when the model perfection level Hp is lower than the threshold TH 1 , the person HMb to be authenticated and the person HMa to be compared are not determined as the same person. Consequently, the accurate authenticating operation in which the reliability of face information used for authentication is sufficiently reflected can be realized.
  • the case of determining whether the person HMb to be authenticated and the person HMa to be compared are the same or not by comparing the model perfection level Hp and the overall similarity Re with the corresponding reference values (thresholds) TH 1 and TH 2 , respectively, has been described.
  • the invention is not limited to the case.
  • a function VH using both of the overall similarity Re and the model perfection level Hp as variables as shown in Equation (17) is set.
  • step SP 14 when the value of the function VH exceeds the predetermined threshold TH 3 , it may be determined that the person HMb to be authenticated and the person HMa to be registered are the same person.
  • VH r ⁇ Hp Re ( 17 ) where ⁇ expresses a constant.
  • step SP 8 when the model perfection level Hp exceeds the threshold TH 1 , the process automatically proceeds into the next process (steps SP 9 and SP 60 ).
  • the invention is not limited to the embodiment.
  • a mode of proceeding into the next process (steps SP 9 and SP 60 ) on receipt of a direct authentication instruction operation from the user may be used. More specifically, when the authentication instruction operation is not input, the process does not proceed into the next process (steps SP 9 and SP 60 ) but repeatedly executes the face information generating process (steps SP 1 to SP 8 ).
  • the authentication instruction is input, if the model perfection level at that time (just before or after the input) is lower than the predetermined threshold TH 1 , a signal (response) of authentication disable is sent back, and the face information generating process is repeated. If the model perfection level exceeds the predetermined value, the process may proceed into the next process (steps SP 9 and SP 60 ). As a result, a system having high response to whether authentication can be performed or not can be provided.
  • the invention is not limited to the case.
  • the inverse number in the right side of the equation (16) may be set as the overall similarity Re.
  • the function VH shown in the equation (17) is expressed by the product between the model perfection level Hp and the overall similarity Re.
  • FIG. 24 is a flowchart showing the replacing operation in the face part detecting steps SP 3 and SP 53
  • FIG. 25 is a diagram showing the corresponding feature points Qj in the obtained two images.
  • step SP 91 ( FIG. 24 ) a process similar to that in the step SP 3 is executed to calculate the two-dimensional coordinates at each of the feature points Qj in the two images.
  • step SP 92 a process similar to that in the step SP 74 ( FIG. 20 ) is executed, and a correlation value in a predetermined area using the feature point Qj as a center is computed between the two images.
  • a predetermined area RW for example, an area having 5 ⁇ 5 pixels
  • the correlation value is computed.
  • the correlation value is calculated by using the equation (4) and the position in which the correlation value is the largest, that is, the area including the same part the most is specified.
  • step SP 93 a process similar to that in the step SP 76 is executed, and the coordinates of the feature point Qj on the image G 2 are corrected. Specifically, the correction is made to set the center position in the predetermined area RW in the position (coordinates) where the correlation value is the largest on the image G 2 to the position of the two-dimensional coordinates of the feature point Qj on the image G 2 .
  • the reliability evaluation value H (j) of the three-dimensional coordinates of each of the feature points is calculated by using the correlation value F (j) between corresponding areas in two images, the invention is not limited to the case.
  • the reliability evaluation value H (j) of the three-dimensional coordinates of each of the feature points can be calculated on the basis of the following elements (FA 1 and FA 2 ).
  • a contrast value FC (j) in each of corresponding areas in two images is calculated.
  • the reliability evaluation value H (j) of the three-dimensional coordinates at each feature point may be calculated.
  • the contrast value FC (j) a value obtained by accumulating differential absolute values of brightness values in adjacent pixels can be used.
  • the distances Df 1 (j) and Df 2 (j) from the cameras to each feature point can be calculated by using the three-dimensional position of the camera based on the camera parameters and three-dimensional coordinates at each feature points.
  • the face information may include texture information of each area using the feature point Qj as a vertex in the face captured on the basis of stereo images (measurement information).
  • the model perfection level Hp may be calculated on the basis of reliability of the two-dimensional information (texture information).
  • the ratio HT of giving texture information mapped to each area (patch) on an individual model can be used as the model perfection level Hp of the individual model.
  • the texture giving ratio HT becomes higher when images captured at various angles are obtained. It is considered that an individual model having the high texture giving ratio HT is generated on the basis of a number of stereo images. Consequently, it can be said that the reliability of data of the individual model having the high texture giving ratio is high.
  • the model perfection level Hp calculated on the basis of the texture giving ratio of the individual model can be also expressed as an evaluation value indicative of reliability of texture information (two-dimensional information) extracted from the individual model in steps SP 11 and SP 61 .
  • the authentication systems 1 A and 1 B may use both model perfection level in which reliability of texture information is reflected (also called “two-dimensional model perfection level”) and model perfection level in which reliability of the three-dimensional shape information is reflected (also called “three-dimensional model perfection level”). Consequently, in the process of updating the registrant information EC 1 (steps SP 15 to SP 18 and steps SP 65 to SP 68 ) performed after end of the authentication determination, the process of updating the feature information based on the two-dimensional information (local two-dimensional face feature amount d (k) ) and the process of updating the feature information based on the three-dimensional shape information (three-dimensional shape feature amount d S ) can be performed separately.
  • the feature information based on the two-dimensional information and the feature information based on the three-dimensional information can be updated separately. Each time the authentication determination is executed, the authentication information of higher accuracy can be assured efficiently.
  • the model perfection level Hp of the individual model is calculated by accumulating, in time series, the reliability evaluation values H (j) of the three-dimensional coordinates at the feature points of stereo images.
  • the invention is not limited to the embodiment. Concretely, it is also possible to calculate an average value of the reliability evaluation values H (j) at the feature points in stereo images and calculate the model perfection level Hp of an individual model by using the average value.
  • the model perfection level Hp of an individual model in which dispersion of the reliability evaluation values H (j) at the feature points are also considered may be calculated.
  • the model perfection level Hp is calculated by using a method such that a standard deviation SD (j) of the reliability evaluation values H (j) at the feature points is calculated and, when the standard deviation SD (j) increases, the model perfection level Hp of the individual model deteriorates. Consequently, when the reliability evaluation values H (j) at feature points have dispersion, in other words, when a predetermined number of face images of low reliability are included in measurement information which is repeatedly input, the model perfection level Hp can be calculated so as to be low.
  • the brightness value of each of pixels in a patch is used as two-dimensional information in the foregoing embodiments
  • the color tone of each patch may be used as the two-dimensional information.
  • the MDA method is used as a method of determining the transformation matrix A in step SP 6 in the foregoing embodiment, the invention is not limited to the method.
  • the Eigenspace method (EM) for obtaining the projective space to increase the difference between the within-class variance and the between-class variance from a predetermined feature space may be used.
  • coincidence between the person to be authenticated and the person to be registered is determined by using not only three-dimensional shape information but also texture information as shown in the equation (16) in the embodiment, the invention is not limited to the determination but coincidence between the person to be authenticated and the person to be registered may be determined by using only the three-dimensional shape information. To improve the authentication accuracy, it is preferable to use also the texture information.
  • three-dimensional shape information of a face is obtained by using a plurality of images input from a plurality of cameras in the embodiment
  • the invention is not limited to the embodiment.
  • three-dimensional shape information of the face of a person to be authenticated may be obtained by using a three-dimensional shape measuring device constructed by a laser beam emitter L 1 and a camera LCA as shown in FIG. 26 and measuring reflection light of a laser beam emitted from the laser beam emitter L 1 by the camera LCA.
  • a method of obtaining three-dimensional shape information with an input device including two cameras as in the foregoing embodiment as compared with an input device using a laser beam, three-dimensional shape information can be obtained with a relatively simpler configuration.
  • the person information storage 34 of the controller 10 a and the person information storage 34 of the controller 10 b are constructed as different members in the embodiment, the invention is not limited to the configuration. For example, a single storage may be shared.
  • the standard model of a face is obtained from the three-dimensional model database 32 in the controller 10 a ( 10 b ) in the embodiment, the invention is not limited to the configuration.
  • the standard model of a face may be obtained from a model storage provided on the outside of the controller 10 a ( 10 b ) via a network such as LAN and the Internet.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Collating Specific Patterns (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
US11/588,025 2005-10-28 2006-10-26 Authentication system and registration system related to facial feature information Expired - Fee Related US7925060B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2005314426A JP4645411B2 (ja) 2005-10-28 2005-10-28 認証システム、登録システム及びプログラム
JPJP2005-314426 2005-10-28
JP2005-314426 2005-10-28

Publications (2)

Publication Number Publication Date
US20070098230A1 US20070098230A1 (en) 2007-05-03
US7925060B2 true US7925060B2 (en) 2011-04-12

Family

ID=37996337

Family Applications (1)

Application Number Title Priority Date Filing Date
US11/588,025 Expired - Fee Related US7925060B2 (en) 2005-10-28 2006-10-26 Authentication system and registration system related to facial feature information

Country Status (2)

Country Link
US (1) US7925060B2 (ja)
JP (1) JP4645411B2 (ja)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090304238A1 (en) * 2007-12-07 2009-12-10 Canon Kabushiki Kaisha Imaging apparatus, control method, and recording medium thereof
US20100209000A1 (en) * 2009-02-17 2010-08-19 Seiko Epson Corporation Image processing apparatus for detecting coordinate position of characteristic portion of face
US20100295959A1 (en) * 1997-10-09 2010-11-25 Fotonation Vision Limited Red-eye filter method and apparatus
US20110043610A1 (en) * 2009-08-24 2011-02-24 Samsung Electronics Co., Ltd. Three-dimensional face capturing apparatus and method and computer-readable medium thereof
US20110058071A1 (en) * 1997-10-09 2011-03-10 Tessera Technologies Ireland Limited Detecting Red Eye Filter and Apparatus Using Meta-Data
US20110164832A1 (en) * 2010-01-04 2011-07-07 Samsung Electronics Co., Ltd. Image-based localization feature point registration apparatus, method and computer-readable medium
US8233674B2 (en) 2007-03-05 2012-07-31 DigitalOptics Corporation Europe Limited Red eye false positive filtering using face location and orientation
US8264575B1 (en) 1997-10-09 2012-09-11 DigitalOptics Corporation Europe Limited Red eye filter method and apparatus
US20120314904A1 (en) * 2010-01-12 2012-12-13 Yasushi Hamada Image collation system, image collation method and computer program
US8422780B2 (en) 2005-11-18 2013-04-16 DigitalOptics Corporation Europe Limited Method and apparatus of correcting hybrid flash artifacts in digital images
US20190005305A1 (en) * 2017-06-30 2019-01-03 Beijing Kingsoft Internet Security Software Co., Ltd. Method for processing video, electronic device and storage medium
US10230860B2 (en) * 2016-08-08 2019-03-12 Kabushiki Kaisha Toshiba Authentication apparatus for carrying out authentication based on captured image, authentication method and server
CN109460690A (zh) * 2017-09-01 2019-03-12 虹软(杭州)多媒体信息技术有限公司 一种用于模式识别的方法和装置

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4910507B2 (ja) * 2006-06-29 2012-04-04 コニカミノルタホールディングス株式会社 顔認証システム及び顔認証方法
US8170297B2 (en) * 2007-01-19 2012-05-01 Konica Minolta Holdings, Inc. Face authentication system and face authentication method
JP4952267B2 (ja) * 2007-01-19 2012-06-13 コニカミノルタホールディングス株式会社 ３次元形状処理装置、３次元形状処理装置の制御方法、および３次元形状処理装置の制御プログラム
WO2009124139A1 (en) * 2008-04-01 2009-10-08 The Government Of The United States Of America, As Represented By The Secretaty Of The Navy Methods and systems of comparing face models for recognition
JP5153434B2 (ja) * 2008-04-22 2013-02-27 キヤノン株式会社 情報処理装置及び情報処理方法
JP5195741B2 (ja) * 2009-12-25 2013-05-15 株式会社デンソー 生体活動計測装置
JP5836724B2 (ja) * 2011-09-21 2015-12-24 キヤノン株式会社 画像認識方法、画像認識装置及びプログラム
KR101841750B1 (ko) * 2011-10-11 2018-03-26 한국전자통신연구원 매칭 정보에 의한 입체 영상 보정 장치 및 그 방법
US10929849B2 (en) * 2013-10-25 2021-02-23 Onevisage Sa Method and a system for performing 3D-based identity verification of individuals with mobile devices
WO2015128961A1 (ja) 2014-02-26 2015-09-03 株式会社日立製作所 顔認証システム
US10614204B2 (en) 2014-08-28 2020-04-07 Facetec, Inc. Facial recognition authentication system including path parameters
US10698995B2 (en) 2014-08-28 2020-06-30 Facetec, Inc. Method to verify identity using a previously collected biometric image/data
US10803160B2 (en) 2014-08-28 2020-10-13 Facetec, Inc. Method to verify and identify blockchain with user question data
US9953247B2 (en) * 2015-01-29 2018-04-24 Samsung Electronics Co., Ltd. Method and apparatus for determining eye position information
US9734682B2 (en) 2015-03-02 2017-08-15 Enovate Medical, Llc Asset management using an asset tag device
US20160301690A1 (en) * 2015-04-10 2016-10-13 Enovate Medical, Llc Access control for a hard asset
JP6526527B2 (ja) * 2015-09-09 2019-06-05 株式会社東芝 識別装置及び認証システム
CN105469072B (zh) * 2015-12-14 2019-05-31 依视路国际公司 一种评估眼镜佩戴者和其所佩戴眼镜搭配度的方法和系统
USD987653S1 (en) 2016-04-26 2023-05-30 Facetec, Inc. Display screen or portion thereof with graphical user interface
JP6878842B2 (ja) * 2016-11-09 2021-06-02 コニカミノルタ株式会社 画像処理装置、認証方法および認証プログラム
CN109003298A (zh) * 2017-06-02 2018-12-14 视镜科技有限公司 虚拟眼镜选配方法与系统
EP3462284A4 (en) * 2017-06-12 2019-07-17 Midea Group Co., Ltd. CONTROL METHOD, CONTROL DEVICE, INTELLIGENT MIRROR AND COMPUTER READABLE STORAGE MEDIUM
JP6981265B2 (ja) * 2018-01-11 2021-12-15 富士通株式会社 生体認証装置、生体認証方法、および生体認証プログラム
JP7157303B2 (ja) 2018-02-01 2022-10-20 ミツミ電機株式会社 認証装置
US10963675B2 (en) * 2019-01-30 2021-03-30 Realnetworks, Inc. Method for selecting images in video of faces in the wild
WO2020178893A1 (ja) * 2019-03-01 2020-09-10 日本電気株式会社 辞書生成装置、生体認証装置、監視システム、辞書生成方法、および記録媒体
WO2022254501A1 (ja) * 2021-05-31 2022-12-08 三菱電機株式会社 個人認証装置及び個人認証システム

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040017930A1 (en) * 2002-07-19 2004-01-29 Samsung Electronics Co., Ltd. System and method for detecting and tracking a plurality of faces in real time by integrating visual ques
US6721444B1 (en) * 1999-03-19 2004-04-13 Matsushita Electric Works, Ltd. 3-dimensional object recognition method and bin-picking system using the method
US20040086157A1 (en) * 2002-11-01 2004-05-06 Kabushiki Kaisha Toshiba Person recognizing apparatus, person recognizing method and passage controller
US6775397B1 (en) * 2000-02-24 2004-08-10 Nokia Corporation Method and apparatus for user recognition using CCD cameras
JP2004288222A (ja) 2004-07-13 2004-10-14 Nec Corp 画像照合装置及びその画像照合方法並びにその制御プログラムを記録した記録媒体
US6961446B2 (en) * 2000-09-12 2005-11-01 Matsushita Electric Industrial Co., Ltd. Method and device for media editing
US7050608B2 (en) * 2001-03-09 2006-05-23 Kabushiki Kaisha Toshiba Face image recognition apparatus
US7120278B2 (en) * 2001-08-24 2006-10-10 Kabushiki Kaisha Toshiba Person recognition apparatus
US7421097B2 (en) * 2003-05-27 2008-09-02 Honeywell International Inc. Face identification verification using 3 dimensional modeling

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4426029B2 (ja) * 1999-09-27 2010-03-03 富士通株式会社 画像照合方法および装置
JP4611575B2 (ja) * 2001-06-20 2011-01-12 セコム株式会社 本人認証用の顔画像照合装置

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6721444B1 (en) * 1999-03-19 2004-04-13 Matsushita Electric Works, Ltd. 3-dimensional object recognition method and bin-picking system using the method
US6775397B1 (en) * 2000-02-24 2004-08-10 Nokia Corporation Method and apparatus for user recognition using CCD cameras
US6961446B2 (en) * 2000-09-12 2005-11-01 Matsushita Electric Industrial Co., Ltd. Method and device for media editing
US7050608B2 (en) * 2001-03-09 2006-05-23 Kabushiki Kaisha Toshiba Face image recognition apparatus
US7120278B2 (en) * 2001-08-24 2006-10-10 Kabushiki Kaisha Toshiba Person recognition apparatus
US20040017930A1 (en) * 2002-07-19 2004-01-29 Samsung Electronics Co., Ltd. System and method for detecting and tracking a plurality of faces in real time by integrating visual ques
US20040086157A1 (en) * 2002-11-01 2004-05-06 Kabushiki Kaisha Toshiba Person recognizing apparatus, person recognizing method and passage controller
US7421097B2 (en) * 2003-05-27 2008-09-02 Honeywell International Inc. Face identification verification using 3 dimensional modeling
JP2004288222A (ja) 2004-07-13 2004-10-14 Nec Corp 画像照合装置及びその画像照合方法並びにその制御プログラムを記録した記録媒体

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8264575B1 (en) 1997-10-09 2012-09-11 DigitalOptics Corporation Europe Limited Red eye filter method and apparatus
US7973828B2 (en) * 1997-10-09 2011-07-05 Tessera Technologies Ireland Limited Red-eye filter method and apparatus
US20100295959A1 (en) * 1997-10-09 2010-11-25 Fotonation Vision Limited Red-eye filter method and apparatus
US20110058071A1 (en) * 1997-10-09 2011-03-10 Tessera Technologies Ireland Limited Detecting Red Eye Filter and Apparatus Using Meta-Data
US8422780B2 (en) 2005-11-18 2013-04-16 DigitalOptics Corporation Europe Limited Method and apparatus of correcting hybrid flash artifacts in digital images
US8233674B2 (en) 2007-03-05 2012-07-31 DigitalOptics Corporation Europe Limited Red eye false positive filtering using face location and orientation
US20090304238A1 (en) * 2007-12-07 2009-12-10 Canon Kabushiki Kaisha Imaging apparatus, control method, and recording medium thereof
US8411908B2 (en) * 2007-12-07 2013-04-02 Canon Kabushiki Kaisha Imaging apparatus, control method, and recording medium thereof
US8170292B2 (en) * 2007-12-07 2012-05-01 Canon Kabushiki Kaisha Imaging apparatus, control method, and recording medium thereof
US20120194700A1 (en) * 2007-12-07 2012-08-02 Canon Kabushiki Kaisha Imaging apparatus, control method, and recording medium thereof
US20100209000A1 (en) * 2009-02-17 2010-08-19 Seiko Epson Corporation Image processing apparatus for detecting coordinate position of characteristic portion of face
US8803950B2 (en) * 2009-08-24 2014-08-12 Samsung Electronics Co., Ltd. Three-dimensional face capturing apparatus and method and computer-readable medium thereof
US20110043610A1 (en) * 2009-08-24 2011-02-24 Samsung Electronics Co., Ltd. Three-dimensional face capturing apparatus and method and computer-readable medium thereof
US20110164832A1 (en) * 2010-01-04 2011-07-07 Samsung Electronics Co., Ltd. Image-based localization feature point registration apparatus, method and computer-readable medium
US8498472B2 (en) * 2010-01-04 2013-07-30 Samsung Electronics Co., Ltd. Image-based localization feature point registration apparatus, method and computer-readable medium
US20120314904A1 (en) * 2010-01-12 2012-12-13 Yasushi Hamada Image collation system, image collation method and computer program
US8938132B2 (en) * 2010-01-12 2015-01-20 Nec Corporation Image collation system, image collation method and computer program
US10230860B2 (en) * 2016-08-08 2019-03-12 Kabushiki Kaisha Toshiba Authentication apparatus for carrying out authentication based on captured image, authentication method and server
US20190005305A1 (en) * 2017-06-30 2019-01-03 Beijing Kingsoft Internet Security Software Co., Ltd. Method for processing video, electronic device and storage medium
US10733421B2 (en) * 2017-06-30 2020-08-04 Beijing Kingsoft Internet Security Software Co., Ltd. Method for processing video, electronic device and storage medium
CN109460690A (zh) * 2017-09-01 2019-03-12 虹软(杭州)多媒体信息技术有限公司 一种用于模式识别的方法和装置

Also Published As

Publication number Publication date
JP4645411B2 (ja) 2011-03-09
US20070098230A1 (en) 2007-05-03
JP2007122454A (ja) 2007-05-17

Similar Documents

Publication Publication Date Title
US7925060B2 (en) Authentication system and registration system related to facial feature information
US20070050639A1 (en) Authentication apparatus and authentication method
US7876931B2 (en) Face recognition system and method
US7421097B2 (en) Face identification verification using 3 dimensional modeling
JP4093273B2 (ja) 特徴点検出装置、特徴点検出方法および特徴点検出プログラム
EP2580711B1 (en) Distinguishing live faces from flat surfaces
JP4950787B2 (ja) 画像処理装置及びその方法
JP4653606B2 (ja) 画像認識装置、方法およびプログラム
US7773798B2 (en) Modeling system, and modeling method and program
JP4321350B2 (ja) パラメタ推定方法、パラメタ推定装置および照合方法
US20070046662A1 (en) Authentication apparatus and authentication method
KR20160033553A (ko) 3차원 얼굴모델 투영을 통한 얼굴 인식 방법 및 시스템
JP4952267B2 (ja) ３次元形状処理装置、３次元形状処理装置の制御方法、および３次元形状処理装置の制御プログラム
JP2005309765A (ja) 画像認識装置、画像抽出装置、画像抽出方法及びプログラム
JP4426029B2 (ja) 画像照合方法および装置
CN111652018B (zh) 一种人脸注册方法和认证方法
KR20020022295A (ko) 3차원 정보를 이용한 얼굴 인식 장치 및 방법
KR100930722B1 (ko) 계층적인 구조의 영상 모델을 이용한 3차원 영상 생성방법과 장치, 이를 이용한 영상 인식 방법과 특징점 추출방법 그리고 상기 방법들을 수행하는 프로그램이 기록된기록매체
JP4814666B2 (ja) 顔解析システム
JP4816874B2 (ja) パラメータ学習装置、パラメータ学習方法、およびプログラム
JP5244345B2 (ja) 顔認証装置
JP4525523B2 (ja) 認証システム、認証方法およびプログラム
JP2008015871A (ja) 認証装置、及び認証方法
JP4956967B2 (ja) 認証システム、認証方法およびプログラム
KR20210112037A (ko) 아바타를 생성하는 장치, 방법 및 컴퓨터 프로그램

Legal Events

Date Code Title Description
AS Assignment

Owner name: KONICA MINOLTA HOLDINGS, INC., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NORITA, TOSHIO;KAWAKAMI, YUICHI;REEL/FRAME:018474/0372

Effective date: 20061017

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

REMI Maintenance fee reminder mailed
LAPS Lapse for failure to pay maintenance fees
STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20150412