JP2014115821A - Face feature extraction device and face authentication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a face feature extraction device facilitating work for having an administrator confirm and correct feature points extracted from a face image and a face authentication system using the feature points information extracted by the face feature extraction device.SOLUTION: A feature extraction means 12 of an administrator terminal 1 extracts a plurality of feature points corresponding to respective parts of a face from a face image in which the face of a user is photographed. A generation means 13 generates teaching information indicating the positions of representative feature points corresponding to representative parts which are the feature points extracted from the face image and predetermined among the respective parts of the face. A display control means 14 displays the teaching information superposed on the face image on a display unit 40. In addition, a re-extraction means 16 re-extracts the feature points corresponding to at least that part from the face image based on the part of the face or a correction position in the face image corresponding to the part of the face indicated through an operation unit 50.

Description

  The present invention relates to a face feature extraction device that extracts feature points corresponding to each part of a face from a face image, and a face authentication system that uses information on feature points extracted by the face feature extraction device.

  A face authentication system is used in which a face image obtained by photographing a face of a person to be collated is collated with a face image of a user registered in advance to determine whether or not to authenticate the person to be collated. Yes. The face authentication system compares the face feature information extracted from the face image obtained at the time of matching with the registered face feature information (hereinafter referred to as registered face information), and uses the registered person to be matched. It is determined whether it is a person.

  If the registered face information is inadequate, there is a risk that authentication will fail even though the person to be verified is the registered user (so-called “personal rejection”). Therefore, automatic correction using various correction parameters is performed on the feature information extracted from the face image. However, since appropriate feature information may not be obtained even if automatic correction is performed, it is necessary for the administrator of the face authentication system to manually correct the feature information.

  In so-called biometric authentication, in addition to the face, physical features such as fingerprints and eyeball irises are used to determine whether or not the user is the user. Regardless of which physical feature is used, the feature information needs to be properly obtained in order to obtain sufficient authentication accuracy, and the administrator must manually correct the feature information automatically extracted from the image. There are things that must be done. As an example of a case where a fingerprint is used, Patent Document 1 discloses that a feature point automatically extracted from a fingerprint image is superimposed on the fingerprint image and displayed on a display unit, and a position to be corrected by a user with a mouse or the like. A fingerprint correction system capable of instructing and performing a feature point correction operation on a fingerprint image is disclosed.

JP-A-4-306778

  However, if all of the automatically extracted feature points are displayed as they are, it is difficult for the administrator of the face authentication system to recognize what each feature point represents, and the administrator cannot determine the position of the feature point. The work of checking and correcting inappropriate feature points becomes very complicated.

  Therefore, the present invention provides a face feature extraction device that facilitates an administrator to check and correct feature points extracted from a face image, and face authentication using feature point information extracted by the face feature extraction device The purpose is to provide a system.

  In order to solve such a problem, the present invention provides feature extraction means for extracting a plurality of feature points corresponding to each part of a face from a face image in which a user's face is copied, and feature points extracted from the face image. Teaching information indicating a position of a representative feature point corresponding to a predetermined representative part of each part of the face on the face image is superimposed on the face image and displayed on the display unit; There is provided a face feature extraction apparatus characterized by comprising:

  The teaching processing means preferably displays the teaching information with some feature points among the plurality of feature points corresponding to the representative site as representative feature points.

  It further has re-extracting means for acquiring a correction position on the face image of the representative part instructed via the operation unit, and re-extracting at least a feature point corresponding to the representative part from the face image based on the correction position. preferable.

  The teaching processing means displays teaching information and a face image on the display unit so that an item indicating the representative part can be selected via the operation unit, and the re-extraction means corresponds to the item selected via the operation unit. It is preferable to re-extract the feature points from the face image based on the correction position instructed for the representative part to be performed.

  It is preferable to further include a re-extracting unit that acquires a correction position on the face image of the representative feature point, which is instructed via the operation unit, and determines the correction position as the position of the representative feature point on the face image.

  The present invention also includes a storage means, a feature extraction means for extracting a plurality of feature points corresponding to each part of the face from a face image in which a user's face is copied, and a feature point extracted from the face image. A teaching processing means for superimposing the teaching information indicating the position on the face image of the representative feature point corresponding to a predetermined representative part of each part of the face on the face image, and the operation unit; In accordance with an instruction input via the registration means for storing information of a plurality of feature points in the storage means as registered face information of the user, and the face information of the person to be collated extracted from the photographed image And an authentication unit that determines whether the person to be verified is a user.

  According to the face feature extraction apparatus and the face authentication system of the present invention, it is easy for an administrator to check and correct feature points extracted from a face image.

It is a schematic block diagram of the management terminal in the face authentication system of one Embodiment of this invention. It is a figure for demonstrating the example of teaching information. It is the figure which showed the example of the face data confirmation screen and the face data correction screen. It is the flowchart which showed the operation example of the registration process by a management terminal. It is a schematic block diagram of the authentication terminal in the face authentication system of one Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The face authentication system of this embodiment has a management terminal and a plurality of authentication terminals. These terminals are communicably connected to each other via a communication line such as a local area network (LAN). Alternatively, the management terminal and the authentication terminal may be connected to each other via a USB cable. The management terminal and the authentication terminal may be integrated.

  First, the management terminal will be described. The management terminal is a terminal used for managing the face authentication system by the administrator of the face authentication system, for example, a personal computer on which management software is executed. The management terminal registers feature information of a user (for example, position information of feature points extracted from a user's face image) in the face authentication system, or registered feature information (registered face information) Used to correct Further, the management terminal acquires authentication record data from the authentication terminal, displays the data on the display unit, or records the data on the recording unit.

  The management terminal of this embodiment is an example of a face feature extraction device. The management terminal supports representative parts of the face such as eyes, nose and mouth (hereinafter referred to as representative parts) that can be easily identified by visual observation among a plurality of feature points extracted from the user's face image. Teaching information for teaching the administrator the position of the feature point to be generated is generated. The management terminal does not display all the extracted feature points on the display unit, but selects a representative feature point that is a feature point of the representative part from the extracted feature points, and teaching information about the representative feature point Is superimposed on the face image. In addition, the management terminal re-extracts the feature points when the manager instructs to correct the position of the specific part indicated by the teaching information on the face image. As a result, the burden on the administrator for checking and correcting the feature point is reduced, and a feature point with an appropriate position on the face image is obtained.

  FIG. 1 is a schematic configuration diagram of a management terminal 1 in a face authentication system according to an embodiment of the present invention. The management terminal 1 includes a registration processing unit 10, a storage unit 20, an input / output unit 30, a display unit 40, and an operation unit 50.

  The registration processing unit 10 includes one or a plurality of processors and their peripheral circuits. The registration processing unit 10 extracts feature points of the user's face from an image obtained by photographing the user's face registered in the face authentication system. The registration processing unit 10 causes the display unit 40 to display a screen for the administrator to check and correct the extracted feature points, and receives a correction instruction input by the administrator via the operation unit 50. In addition, the registration processing unit 10 transmits registration data such as identification information, face images, and feature information of the user to be registered, which is determined by the operation of the administrator, to the authentication terminal.

  An image to be processed by the registration processing unit 10 is taken by an external photographing device such as a digital camera and is taken in from these devices. Note that the image to be processed by the registration processing unit 10 may be an image taken by an imaging unit of an authentication terminal described later.

  The storage unit 20 includes at least one of a semiconductor memory, a magnetic recording medium and its access device, and an optical recording medium and its access device. The storage unit 20 stores a computer program, various parameters, data, and the like for controlling the management terminal 1. In addition, the storage unit 20 stores registration data of each user as master data of the face authentication system.

  The input / output unit 30 includes a communication interface connected to an external device such as an authentication terminal, and a control circuit thereof. The display unit 40 is composed of, for example, a liquid crystal display, and a face data confirmation screen for the administrator to confirm the feature points extracted by the registration processing unit 10 and face data for the administrator to instruct correction of the feature points Display the correction screen. The operation unit 50 is composed of, for example, a mouse, and is used by an administrator to input an instruction to determine a feature point or an instruction to correct a feature point of a certain part to the registration processing unit 10. The display unit 40 and the operation unit 50 may be integrally configured as a touch panel display.

  Next, functions of each unit included in the registration processing unit 10 will be described. The registration processing unit 10 includes a face detection unit 11, a feature extraction unit 12, a generation unit 13, a display control unit 14, a registration unit 15, and a re-extraction unit 16. These units of the registration processing unit 10 are implemented as a functional module of a program that operates on the microprocessor unit, for example. These means may be constituted by independent integrated circuits, firmware, a microprocessor, and the like.

  The face detection unit 11 detects a face area, which is an area in which a user's face is shown, from a processing target image acquired via the input / output unit 30, and includes only pixels included in the face area from the image. Is cut out as a face image. The face detection unit 11 enlarges or reduces the face area as necessary so that the size of the face area is equal to the face area of a template face image described later. The face detection unit 11 passes the face image to the feature extraction unit 12.

  In order to detect a face area, the face detection unit 11 calculates, for example, one or a plurality of feature amounts from a partial region in the acquired image, and a predetermined condition that the feature amount is considered to correspond to a human face. If the condition is satisfied, the partial area is detected as a face area. Specifically, the face detection unit 11 performs edge pixel extraction using, for example, a Sobel filter or the like, and calculates a feature amount such as a direction distribution of the edge pixel in the partial region or a luminance distribution near the edge. . Alternatively, the predetermined conditions described above may be determined in advance by obtaining the values of these feature values in advance from a plurality of images obtained by photographing a person's face and determining the range thereof.

  The feature extraction unit 12 extracts a plurality of feature points corresponding to each part of the face from the face image acquired from the face detection unit 11. The feature extracting unit 12 may use any of various conventionally proposed methods for extracting feature points. In this embodiment, the feature extraction unit 12 evaluates the similarity of the image features of each part constituting the face between the face image acquired from the face detection unit 11 and a template face image prepared in advance. To search for feature points.

  Specifically, the feature extraction unit 12 uses, as a reference image, the peripheral regions of the detailed portions of the plurality of template face images for the detailed portions constituting each portion such as the eyes, nose, mouth, eyebrows, and facial contours. Use. As an example of this detailed part, for example, the part “right eye” includes the top of the eye, the corner of the eye, the upper end of the eye, and the lower end of the eye. The same applies to the “left eye”. The site “nose” includes the tip of the nose (nose apex), the back of the nose, the right end of the nose, the left end of the nose, and the like. The part “mouth” includes the right end of the mouth, the left end of the mouth, the upper end of the mouth, the lower end of the mouth, and the like. The part “right eyebrow” includes an eyebrow head, an eyebrow butt, and the like. The same applies to “left eyebrow”. The site “contour” includes the center of the jaw, the right side of the jaw, the left side of the jaw, and the like. In addition to this, the feature extraction unit 12 uses the peripheral area of the template face image as a reference image for a part such as a cheek located between the parts.

  Then, the feature extraction unit 12 obtains, for each template, an evaluation value that is a difference between the image feature of the face image acquired from the face detection unit 11 and the image feature of the reference image at one position on the face image. The sum S of values is calculated. Examples of image features here include the luminance value and hue value of each pixel. The feature extraction unit 12 calculates the sum S of evaluation values at each position on the face image, and searches for a position where the value of the sum S of evaluation values is minimum. Then, the feature extraction unit 12 extracts the minimum position as a feature point indicating the target detailed part. When there is only one template, the feature extraction unit 12 extracts a position where the above-described evaluation value regarding the template is minimum as a feature point.

  The feature extraction unit 12 represents, for each extracted feature point, a code for identifying the feature point, position information (coordinate information) of the feature point on the face image, and a part or a detailed part corresponding to the feature point. The part information and the image feature of the detailed part (peripheral image feature) are associated with each other. Then, the feature extraction unit 12 creates a feature point list combining these data for each feature point. This feature point list is feature information. The feature extraction unit 12 stores the feature point list in the storage unit 20 and passes it to the generation unit 13.

  The feature extraction unit 12 may calculate the evaluation value in consideration of the relative positional relationship between the detailed parts. For example, when the feature extraction means 12 extracts the feature point of the corner of the eye in a state where the feature point of the eye is extracted, the corner of the eye is within a certain range separated from the feature point of the eye by the average eye size. Therefore, the feature extraction unit 12 corrects the evaluation value to a larger value so that the position farther from the range is less likely to be extracted as the feature point of the corner of the eye. Good. Moreover, the feature extraction means 12 is good also as a structure which extracts only the feature point regarding the representative site | part mentioned later.

  The generation means 13 is a feature point (representative feature) corresponding to a predetermined representative portion that can be easily identified visually by an administrator, such as eyes, nose and mouth, among a plurality of feature points extracted from the face image. Teaching information for teaching the position of the point) on the face image to the administrator is generated. The generation unit 13 passes the generated teaching information to the display control unit 14.

  The eyes, nose, and mouth illustrated here are suitable as representative parts because the administrator can easily recognize their positions on the image. On the other hand, for example, the position of an outline tends to be ambiguous even on an image, and is not suitable as a representative part. Note that the positions of the eyes, nose and mouth are often considered to be relatively important even in the verification process by the authentication terminal, and can be said to be an important part having a high influence on verification.

  Here, two examples of teaching information will be described. In the first example, the generation unit 13 generates teaching information using all feature points extracted by the feature extraction unit 12 for each representative part as representative feature points. For example, for the “eyes”, the generating means 13 may include the top of the eyes, the corner of the eyes, the upper and lower ends of the eyes, the nose for the “nose”, the right nose and the left end of the nose, the right end of the nose and the left end of the mouth for the “mouth”. Teach information is generated for the feature points at the upper and lower ends of the mouth. Then, the generation unit 13 uses the position coordinates of the feature points of each representative part on the face image and information for specifying marks such as x marks arranged in accordance with the positions of the feature points as teaching information.

  In the second example, the generation unit 13 selects a predetermined feature point that allows the administrator to recognize the position relatively easily by visual observation among the feature points constituting each representative part. The teaching information is generated as follows. For example, the generating means 13 uses the eyes and corners of the eyes for the “eyes”, the nose tip for the “nose”, and the right and left ends of the mouth for the “mouth”. Generate. Then, the generation unit 13 uses, as teaching information, the position coordinates of the representative feature points on the face image and information for specifying marks such as x marks arranged in accordance with the positions of the representative feature points.

  The mark may be any mark as long as it indicates the position of each feature point, and its shape is not limited to the x mark. Further, the generation unit 13 may apply the first example to a certain part and apply the second example to another part.

  2A to 2C are diagrams for explaining examples of teaching information. FIG. 2A is a schematic diagram showing the feature points extracted by the feature extraction unit 12 with x marks 211. Examples of teaching information generated by the generating unit 13 for such feature points are shown in FIGS. 2 (B) and 2 (C).

  FIG. 2B is a schematic diagram showing a first example of teaching information. In FIG. 2B, a cross mark 221 indicating the feature points of the “right eye”, the top of the eyes, the top of the eyes, and the bottom of the eyes, and the cross mark 221 indicating the feature points of the “left eye”, the top of the eyes, the top of the eyes, and the top of the eyes. 222 indicates the characteristic points of the nose apex, right nose and left nose of “nose”, and “x” 224 indicates the characteristic points of the right end of the mouth, the left end of the mouth, the upper end of the mouth, and the lower end of the mouth. It is displayed as teaching information. FIG. 2C is a schematic diagram illustrating a second example of teaching information. In FIG. 2C, a cross mark 231 indicating the eye and the corner of the eye as the representative feature point of the “right eye”, and a cross mark 232 indicating the head and the corner of the eye as the representative feature point of the “left eye” are the representative feature points of the “nose”. Are indicated as teaching information, and an X mark 233 indicating the nose tip is displayed as teaching information, and an X mark 234 indicating the right end of the mouth and the left end of the mouth is displayed as representative feature points of the “mouth”. In the example of FIGS. 2B and 2C, the positions of the feature points of the right eye, the left eye, and the mouth match the positions of the actual parts, that is, the X mark 221, 222, 224 or the X mark 231, 232, 234. On the other hand, it can be seen from the x mark 223 or the x mark 233 that the position of the feature point of the nose is shifted.

  The display control unit 14 superimposes the teaching information on the face image and displays it on the display unit 40. The display control unit 14 causes the display unit 40 to display a face data confirmation screen including the generated teaching information and the face image so that the administrator can confirm the extracted feature points. The administrator can instruct whether or not to correct the position of the feature point via the operation unit 50 on the screen. Upon receiving the instruction, the display control unit 14 causes the display unit 40 to display a face data correction screen for the administrator to correct the position of the feature point for each representative part. The administrator can manually correct the position of the feature point for each representative site via the operation unit 50 on the screen. The display control means 14 may display the teaching information superimposed on the face image even on the face data correction screen. When the manager corrects the position of the representative part, the re-extraction means 16 re-extracts the feature points as will be described later. At that time, after the generation unit 13 generates teaching information based on the re-extracted feature points, the display control unit 14 displays the face data confirmation screen on the display unit 40 again. The generation unit 13 and the display control unit 14 are an example of a teaching processing unit that generates teaching information and superimposes the teaching information on a face image and displays the teaching information on a display unit.

  FIGS. 3A to 3C are diagrams illustrating examples of a face data confirmation screen and a face data correction screen. FIG. 3A shows an example of a face data confirmation screen. In the region 311 on the left side of the screen, the teaching information 312 to 315 indicated by “x” indicating the representative parts of the right eye, the left eye, the nose and the mouth are superimposed on the face image. The administrator can determine whether or not the position needs to be corrected for each representative part by looking at this. Then, the administrator clicks the “correct” button 316 or the “confirm” button 317 displayed on the right side of the screen with the mouse, for example, to correct the position of the feature point via the operation unit 50. Alternatively, a confirmation instruction can be input.

  FIG. 3B is an example of a face data correction screen. When the administrator selects the “correct” button 316 on the face data confirmation screen of FIG. 3A, the display control means 14 receives the instruction and causes the display unit 40 to display this face data correction screen. On the right side of the screen, a “right eye” button 325, a “left eye” button 326, a “nose” button 327, and a “mouth” button 328 indicating representative parts are displayed. The manager sequentially selects the representative parts to be corrected using these buttons. Then, for each representative part, the administrator moves the cursor to the correct position of the representative part (position around the center of the representative part) on the face image displayed in the area 321 on the left side of the screen, for example, a mouse Click to enter the correction position. Black circles 322 and 323 displayed on the right and left eyes of the face image indicate that the correction position has been input by the administrator. The “nose” button 327 is displayed in black and an arrow 324 is displayed on the face image. In the screen of FIG. 3B, the administrator is currently performing an operation of inputting the correction position of the nose. Is shown.

  The administrator can select the “Execute correction” button 330 after inputting the correction positions for all the representative parts. Then, the input correction position is transferred to the re-extraction means 16 via the display control means 14, and the re-extraction means 16 re-extracts the feature points based on the correction position. The administrator can also select the “Cancel” button 329 to cancel the correction work. At this time, the registration processing unit 10 stops the process. Although not shown in FIG. 3 (B), for example, the center of each representative part on the face image is displayed as a sample on the edge of the screen, and the administrator can operate with reference to the sample. You may do it.

  FIG. 3C is another example of the face data correction screen. In FIG. 3C, as in FIG. 3A, teaching information 332 to 335 indicated by “X” are superimposed on the face image on the area 331 on the left side of the screen. In this way, the teaching information may be displayed on the face data correction screen. Further, although FIG. 3B has been described on the assumption that the correction positions of all the representative parts are input, the administrator may correct only a part of the representative parts determined to be necessary. For example, in FIG. 3C, the right eye, left eye, and mouth feature point positions are in agreement with the actual part positions based on the right eye teaching information 332, the left eye teaching information 333, and the mouth teaching information 335. Recognize. On the other hand, it can be seen from the nose teaching information 334 that the position of the feature point of the nose is shifted. For this reason, the administrator may select the “nose” button 327 on the right side of the screen and correct only the position of the nose. A black circle 336 displayed on the nose of the face image indicates that the correction position has been input by the administrator.

  The registration unit 15 determines the current feature information as a registration target when an instruction to determine the position of the feature point is input by the administrator. At that time, the registration means 15 determines whether the user information to be registered is input separately or automatically assigned identification information (ID number), user information such as name and affiliation, and face image. The feature information is registered in the storage unit 20 as registration data. Further, the registration unit 15 transmits registration data to the authentication terminal via the input / output unit 30.

  The re-extracting means 16 re-extracts at least feature points corresponding to the part from the face image based on the correction position of the part taught on the face image and instructed by the administrator via the operation unit 50. The re-extracting unit 16 passes the re-extracted feature point information to the generating unit 13. Then, the generation unit 13 generates teaching information based on the re-extracted feature points, and the display control unit 14 displays the face data confirmation screen on the display unit 40 again. However, since it is considered that the feature point at the correct position can be obtained if re-extraction is performed once, the re-extraction means 16 passes the information of the re-extracted feature point to the registration means 15 and re-extracts the feature point. The registration means 15 may register as it is.

  The feature point extraction method by the re-extraction means 16 is the same as the feature point extraction method by the feature extraction means 12. However, the re-extracting means 16 extracts the feature points from the face image so that the feature points corresponding to the part for which the correction position is instructed are easier to extract as the position from the correction position instructed by the administrator is closer. Re-extract. Note that the re-extraction unit 16 may extract feature points by a method different from that of the feature extraction unit 12.

  Specifically, the re-extraction unit 16 uses the same template as the feature extraction unit 12 and uses the same image feature of the face image to be processed and the image feature of the reference image at the candidate position of the feature point on the face image. Is calculated for each template. The re-extraction means 16 obtains a correction value E according to the distance d between the correction position designated by the administrator and the candidate position, and obtains an evaluation value that is a value obtained by adding the correction value E to the above-described difference. The correction value E is a value that increases with the distance d. Then, the re-extraction means 16 calculates the sum S ′ of evaluation values for each template. The re-extraction means 16 calculates the sum S ′ of evaluation values at each position on the face image, and searches for a position where the value of the sum S ′ of evaluation values is minimum. Then, the re-extraction means 16 extracts the minimum position as a feature point indicating the target detailed part.

  The re-extracting means 16 uses the evaluation value obtained by adding the correction value E to the degree of difference by increasing the evaluation value at a point far from the correction position instructed by the administrator, and extracting a point closer to the correction position as a feature point This is because weighting is performed so that it is easily performed. Note that the re-extraction means 16 does not obtain the evaluation value by adding the correction value E to the degree of difference, but obtains a correction coefficient α corresponding to the distance d, and obtains a value obtained by multiplying the degree of difference by the correction coefficient α. It may be an evaluation value. The correction coefficient α is also a value that increases with the distance d.

  Further, it is more preferable that the re-extraction means 16 obtains the correction value E or the correction coefficient α in consideration of not only the distance d from the correction position but also the direction. For example, when the position of the right eye is corrected and the feature point of the right eye is re-extracted, the probability that the “right eye corner” is within a certain range in the left direction (that is, outside the face) with respect to the correction position of the “right eye”. Therefore, when the position of the “right eye corner” is searched, the re-extraction means 16 takes the correction value E or the correction coefficient α so as to take a small value for the candidate position of the feature point within a certain range in the left direction. It is good to decide. At this time, the re-extraction means 16 may increase the correction value E or the correction coefficient α for the reverse rightward position.

  Further, the re-extracting means 16 has a feature point corresponding to a part other than the representative part on the facial image when extracted by the feature extracting means 12 when the correction position is designated for the characteristic point of the representative part. Even if the feature point of a part other than the representative part is re-extracted so that the original position, which is the position, is closer to the position before the representative feature point is corrected, it is easier to extract the position near the correction position of the representative feature point. Good.

  Note that the re-extraction means 16 may re-extract only the feature points corresponding to the part instructed to be corrected by the administrator without re-extracting all the feature points of the face.

  FIG. 4 is a flowchart showing an operation example of registration processing by the management terminal 1. When the start of the registration process is instructed by the administrator's operation, first, the face detection unit 11 acquires an image via the input / output unit 30 (step S1). The face detection unit 11 detects a face area in which the user's face is reflected from the acquired image, and cuts out only pixels included in the face area from the image (step S2). The face detection unit 11 passes the cut image to the feature extraction unit 12.

  The feature extraction unit 12 extracts a plurality of feature points corresponding to each part of the subject's face from the face image (step S3). The feature extraction unit 12 stores the obtained feature point information in the storage unit 20 and passes it to the generation unit 13. Next, the generation unit 13 generates teaching information indicating the position of the feature point of the representative part among the extracted feature points (step S4). At this time, the generation means 13 generates teaching information indicating the positions of all the feature points of each representative part or teaching information indicating the positions of only the representative feature points of each representative part. Then, the display controller 14 superimposes the generated teaching information and the face image on the display unit 40 (step S5).

  Here, when the administrator inputs an instruction to correct the position of the representative part via the operation unit 50 (Yes in step S6), the re-extraction means 16 re-extracts the feature points based on the instructed correction position ( Step S7). Thereafter, the generation unit 13 returns to the process of generating teaching information (step S4). On the other hand, when the administrator inputs an instruction to confirm a feature point via the operation unit 50 (No in step S6), the registration unit 15 registers information on the confirmed feature point in the storage unit 20 as registration data ( Step S8). This completes the operation of the registration processing unit 10.

  After the re-extraction process by the re-extraction means 16 (step S7), the registration means 15 does not return to the process of generating the teaching information (step S4), and the registration means 15 registers the feature point information as it is. You may progress to (step S8).

  As described above, the management terminal 1 does not display all the feature points extracted by the feature extraction unit 12 from the user's face image on the display unit 40, but the eyes, Teaching information indicating the position of the feature point corresponding to the representative part suitable for confirmation of the nose and mouth is superimposed and displayed on the face image. As a result, the burden on the administrator for checking and correcting feature points is reduced. In the management terminal 1, the re-extracting unit 16 re-extracts the feature points when the administrator instructs the position correction on the face image of a part indicated by the teaching information. Thereby, feature points with appropriate positions on the face image are extracted.

  In the face data correction screen of FIG. 3B, the example has been described in which the administrator corrects the position of the feature point for each part of the face. However, instead of the part unit, the unit of individual feature points on which the teaching information is displayed. Thus, the administrator may be able to correct the position of the feature point. For example, on the face image displayed in the area 321 on the left side of the face data correction screen in FIG. 3B, the administrator moves the cursor to a feature point whose position is to be corrected and designates the feature point. May be configured such that the correction position of each feature point can be indicated by dragging and dropping with a mouse, for example. Alternatively, a selection button similar to the buttons 325 to 328 indicating the representative part is displayed for each detailed part such as “right eye corner” or “right eye corner”, as in the face data correction screen of FIG. The administrator may be able to indicate the correction position for each detailed part.

  When the position is corrected in units of individual representative feature points in this way, the re-extraction means 16 determines the instructed correction position as the position of the representative feature point, and uses the determined feature point information. The registration unit 15 may register in the storage unit 20. In this case, the re-extraction means 16 may re-extract feature points other than the representative feature points based on the correction positions of the representative feature points.

  Further, the face authentication system in which the management terminal 1 is used may be a face authentication system that employs an authentication method using three-dimensional face data. In this case, the management terminal 1 may use the feature information obtained by the above processing so that 3D processing means (not shown) three-dimensionalizes the face data.

  This 3D processing means estimates a three-dimensional face shape using a three-dimensional model based on the user's face image and feature information extracted from the image by the feature extraction means 12 and modified as necessary. 3D face shape data is created. The three-dimensional model is a model for reproducing a registered person's head and generating face images in various directions. The 3D processing means is, for example, any one of various known three-dimensional model generation methods described in JP2011-210114A, JP2009-2111148A, JP2011-215843A, and the like. Can be used.

  Then, the 3D processing means centered on the face image facing directly in front, for example, the face image viewed from 25 × 5 × 5 directions whose directions are changed to ± 30 degrees in 15 degree increments in the vertical and horizontal directions, respectively. The feature information is registered in the storage unit 20. The feature information at this time is extracted from each face image by the feature extraction means 12.

  The 3D processing means converts the feature information extracted by the feature extraction means 12 based on the three-dimensional face shape data based on the feature information in the reference direction (usually a face image facing directly in front). Direction feature information may be used. The 3D processing means may convert the feature information extracted from the two-dimensional face image into the three-dimensional coordinate system using the three-dimensional face shape data as the registered feature information. In addition, only the feature information of the reference direction is registered, and at the time of verification by the authentication terminal, the processing unit of the authentication terminal performs conversion to rotate the direction with respect to the feature information to generate the feature information for verification Also good.

  Next, the authentication terminal will be described. The authentication terminal collates the facial feature information of the person to be collated obtained from the image captured during the collation with the facial feature information (registered facial information) of the user registered in advance using the management terminal 1. Then, it is determined whether or not the person to be verified is authenticated. The authentication terminal is provided, for example, for each gate or automatic door where traffic is restricted, and controls the gate, automatic door, and the like according to the authentication result.

  FIG. 5 is a schematic configuration diagram of the authentication terminal 100 in the face authentication system according to the embodiment of the present invention. The authentication terminal 100 includes an authentication processing unit 110, a storage unit 120, an input / output unit 130, and a registration processing unit 140. In the authentication terminal 100, an image captured by the imaging unit 200 is input to the authentication processing unit 110.

  The imaging unit 200 is installed so that the face of the person to be collated can be photographed according to the environment in which the authentication terminal 100 is operated. The imaging unit 200 generates an image showing the face of the person to be verified as an input face image. For this purpose, the imaging unit 200 includes a camera including an optical system that forms an image of the face of the person to be collated on a two-dimensional array of solid-state imaging elements such as a CCD or a CMOS. The imaging unit 200 outputs the generated input face image to the authentication processing unit 110.

  Note that the imaging unit 200 may create a color multi-tone image as an input face image, or may be a camera that has a sensitivity in the near infrared region and creates a gray image. Moreover, it is preferable that the image pick-up element array which the image pick-up part 200 has has the pixel number of the grade which can distinguish the features, such as the eyes of the face reflected in the input face image, a nose, and a mouth.

  The authentication processing unit 110 has one or a plurality of processors and their peripheral circuits. Then, the authentication processing unit 110 extracts facial feature information (hereinafter referred to as input face information) of the person to be collated from the face image input by the imaging unit 200, and uses the input face information registered in the storage unit 120 in advance. It is determined whether or not the person to be verified is authenticated as a candidate by comparing with the registered face information of the person. The authentication processing unit 110 is an example of an authentication unit that collates face information of a person to be collated extracted from a captured image with registered face information and determines whether the person to be collated is a user.

  The storage unit 120 includes at least one of a semiconductor memory, a magnetic recording medium and its access device, and an optical recording medium and its access device. The storage unit 20 stores a computer program for controlling the authentication terminal 100, various parameters, data, and the like. In addition, the storage unit 120 stores registration data of each user. The storage unit 120 stores the authentication result as authentication record data.

  The input / output unit 130 includes, for example, a communication interface connected to an automatic door, a passage gate, or an external device that controls an electric lock, and a control circuit thereof. When the input / output unit 130 receives from the authentication processing unit 110 a signal indicating successful authentication of the verification target person, the input / output unit 130 outputs a signal requesting, for example, unlocking of the electric lock to the connected device. Alternatively, the input / output unit 130 may output a signal indicating the authentication result by the authentication processing unit 110 to a display or the like that is installed in the vicinity of the imaging unit 200 and notifies the person to be verified of the authentication result by display or voice. Good.

  The authentication terminal 100 is connected to the management terminal 1 and receives user registration data from the management terminal 1 via the input / output unit 130. At that time, the registration unit 15 of the management terminal 1 transmits the registration data of the user to the authentication terminal according to the instruction input by the administrator, and the storage unit 120 stores the registration data as the registered face information of the user. Remember me. Thereby, the feature information of the user's face is registered in the authentication terminal 100 as registered face information. In addition, when it is necessary to correct the registered face information stored in the storage unit 120 for a certain user, the registration unit 15 of the management terminal 1 registers the user according to the instruction input by the administrator. Data is transmitted to the authentication terminal, and the registered face information of the correction target user registered in the storage unit 120 is corrected with the registered data.

  Next, functions of each unit included in the authentication processing unit 110 will be described. The authentication processing unit 110 includes a face detection unit 111, a feature extraction unit 112, a collation unit 113, and an authorization unit 114. These means of the authentication processing unit 110 are mounted as a functional module of a program that operates on the microprocessor unit, for example. These means may be constituted by independent integrated circuits, firmware, a microprocessor, and the like.

  Similar to the face detection unit 11 of the management terminal 1, the face detection unit 111 detects a face area in which the face of the person to be collated is shown from the input face image. For example, the face detection unit 111 passes an image obtained by extracting only pixels included in the face area from the input face image to the feature extraction unit 112.

  The feature extraction unit 112 extracts input face information from the face area of the input face image, like the feature extraction unit 12 of the management terminal 1. The feature extraction unit 112 passes the obtained input face information to the collation unit 113. In addition, the feature extraction unit 112 stores the input face information in the storage unit 120.

  The collating unit 113 collates the input face information of the person to be collated with each registered face information stored in the storage unit 120, and determines the user corresponding to the registered face information similar to the input face information as the person to be collated. Identify candidates that may be the same person. For this purpose, the collating unit 113 first calculates the similarity representing the degree of similarity between the user's face and the face of the person to be collated for each user by comparing the input face information and each registered face information. To do.

  For example, the matching unit 113 obtains an individual similarity between the input face information and the registered face information by using surrounding image features for each feature point extracted by the feature extraction unit 112, and calculates the sum of the total similarities. Calculate as degrees. In this case, the smaller the difference between the input face information and the registered face information, that is, the greater the similarity between the face of the person to be collated and the face of the user, the greater the similarity value. The collation unit 113 may calculate an amount representing the degree of similarity between the registered user's face and the collation target person's face by another known method.

  Then, the collating unit 113 obtains the maximum value of the calculated similarity, and determines that the registered face information having the maximum value is similar to the input face information when the maximum value is equal to or greater than a threshold value, for example. The collation means 113 identifies the user corresponding to the similar registered face information as a candidate. The verification unit 113 passes the candidate identification information to the authorization unit 114. On the other hand, when the maximum value is less than the threshold value, the verification unit 113 notifies the authorization unit 114 that there is no candidate.

  The authorization unit 114 authenticates the person to be verified as a user identified as a candidate. At this time, the authorization unit 114 outputs a signal for unlocking the electric lock, for example, as a signal indicating a successful authentication via the input / output unit 130, and succeeds in the authentication on a display or the like installed in the vicinity of the imaging unit 200. Notify that. On the other hand, when notified from the verification unit 113 that there is no candidate, the authorization unit 114 does not authenticate the verification target person. At this time, the authorization unit 114 outputs a signal indicating an authentication failure via the input / output unit 130.

  Further, the face authentication system is not limited to authenticating the person to be collated using one input face image, but may be one that finally authenticates the person to be collated using a plurality of input face images having different shooting times. For example, the face authentication system extracts images of a person's face area from a plurality of input face images taken at a plurality of points in time, and applies a known tracking technique to the same person For the face areas that are determined to include the face of the face, the similarity between the face area image and each of a plurality of registered face images registered in advance is calculated in chronological order, and the similarity is used. It may be a face authentication device (so-called walk-through face authentication device) that authenticates a person walking in the monitoring area.

  In this case, the face authentication system counts, for each user, the number of first hits, which is the number of times that the similarity between the face information of a certain user and the input face information is maximized among the users. The face authentication system calculates, for each user, a first-order hit rate that is a ratio of the number of first-order hits to the number of input face images in a period in which the person to be verified is captured. Then, when the first-rank hit rate of a certain user becomes equal to or higher than a threshold (for example, 70%), the face authentication system may determine that the person to be verified is the user. Alternatively, when the number of first hits of a certain user exceeds the threshold value (for example, 7 times) or more, the face authentication system may determine that the person to be verified is the user.

DESCRIPTION OF SYMBOLS 1 Management terminal 10 Registration processing part 11 Face detection means 12 Feature extraction means 13 Generation means 14 Display control means 15 Registration means 16 Re-extraction means 20 Storage part 30 Input / output part 40 Display part 50 Operation part 100 Authentication terminal 110 Authentication processing part 111 Face detection unit 112 Feature extraction unit 113 Collation unit 114 Authorization unit 120 Storage unit 130 Input / output unit 200 Imaging unit

Claims (6)

Feature extraction means for extracting a plurality of feature points corresponding to each part of the face from a face image in which the face of the user is copied;
Feature information extracted from the face image and superimposing on the face image teaching information indicating a position on the face image of a representative feature point corresponding to a predetermined representative part of each part of the face Teaching processing means for displaying on the display unit,
A facial feature extraction apparatus characterized by comprising:   The facial feature extraction apparatus according to claim 1, wherein the teaching processing unit displays the teaching information by using a part of feature points corresponding to the representative part as the representative feature points.   Re-extracting means for acquiring a correction position on the face image of the representative part instructed via the operation unit, and re-extracting at least a feature point corresponding to the representative part from the face image based on the correction position The facial feature extraction device according to claim 1, further comprising: The teaching processing means displays the teaching information and the face image on the display unit so that an item indicating the representative part can be selected via the operation unit,
The facial feature according to claim 3, wherein the re-extracting unit re-extracts a feature point from the facial image based on a correction position instructed for a representative part corresponding to the item selected via the operation unit. Extraction device.   Re-extracting means for obtaining a correction position on the face image of the representative feature point, which is instructed via the operation unit, and determining the correction position as the position of the representative feature point on the face image; The facial feature extraction apparatus according to claim 1 or 2. Storage means;
Feature extraction means for extracting a plurality of feature points corresponding to each part of the face from a face image in which the face of the user is copied;
Feature information extracted from the face image and superimposing on the face image teaching information indicating a position on the face image of a representative feature point corresponding to a predetermined representative part of each part of the face Teaching processing means for displaying on the display unit,
In accordance with an instruction input via the operation unit, a registration unit that stores information on the plurality of feature points in the storage unit as registered face information of the user;
Authentication means for collating face information of a person to be collated extracted from a captured image with the registered face information and determining whether the person to be collated is the user;
A face authentication system comprising:

Images