JP5618594B2 - Face image matching device - Google Patents

Description

  The present invention relates to a face image processing device, and more particularly to a face image matching device that matches a face image of a person in a plurality of input images.

  2. Description of the Related Art Conventionally, an entrance / exit management system in which a user stands in front of a door and is authenticated by an authentication device at an entrance of an office room and the like is permitted to enter the room when authenticated. In this entrance / exit management system, the face of the user who is going to enter the room in front of the door is photographed by an authentication camera installed near the door and the input face image extracted from the photographed image is registered in the system in advance. The face image of the person who is allowed to enter the room is collated. Then, if the person of the input face image matches one of the registered face images of the person permitted to enter the room registered in the system, the photographed person is authenticated as a person who should be permitted to enter the room, and the electric lock on the door is You can enter the room because it is unlocked.

  Here, in order to collate face images with high accuracy, it is preferable that the faces of the persons shown in the face images are in the same direction. When collation processing is performed using face images showing faces facing different directions, the shape and position on the face image of characteristic parts of the face such as eyes, nose and mouth are different. There is a risk that collation accuracy may be lowered. In addition, depending on the face orientation, there may be a characteristic part that cannot be seen in the face image, which may further reduce the matching accuracy.

  Therefore, a plurality of registered face images having different face orientations are prepared for each of a plurality of registered persons, and the most similar registered face image is determined based on the similarity between the registered face image and the input face image of each face orientation. There is a face image authentication device that determines that a person is a person of an input face image (see Patent Document 1). According to the face image authentication device disclosed in Patent Literature 1, a person of a registered face image having the highest similarity to the input face image is specified from registered face images of various face orientations, and has the same face orientation as the input face image. It is expected that the registered face image is determined to be the most similar.

  Further, there is disclosed a face identifying device that uses a plurality of unique data related to a person such as a fingerprint image as well as a face image to collate a photographed person with a registered person (see Patent Document 2). . In this face identification device, a plurality of similarities between a plurality of unique data related to a specific target person such as a face image and a fingerprint image acquired by a camera or the like and similar specific data related to a person registered in the device in advance are calculated. Then, the calculated plurality of similarities are integrated to collate the person to be identified with the registered person. As a method for integrating a plurality of similarities, a method for calculating the sum of similarities is described.

JP 2009-245338 A JP 2005-242890 A

  However, in the face image authentication device described in Patent Document 1, an input face image, a registered face image having a face orientation different from the face orientation shown in the input face image by a person different from the person appearing in the input face image, and The similarity may be higher than expected. For this reason, this face image authentication device may mistakenly determine that the person in the input face image is another person (accepting another person), or conversely, determine that the person is not the other person (reject himself / herself) and fail the verification. is there. In order to determine that this verification failure matches the input face image and the registered face image while allowing some differences so that it can be determined that the person is the person even if there is a slight change in facial expression or lighting. It is thought to happen.

  On the other hand, the face identification device described in Patent Document 2 does not consider the face orientation, and simply calculates the average, sum, or product of a plurality of similarities for each registered person, and integrates the similarities. . Therefore, in this face identification device, if the similarity between the input face image and the registered face image with a different person and different face orientation becomes very high, the very high similarity affects the similarity after integration. May cause effects. As a result, the degree of similarity after integration of a person different from the person of the input face image becomes very high, and there is a problem that accepting another person who erroneously determines that the person of the input face image is another person or rejecting the person is generated. is there.

  An object of the present invention is to provide a face image matching device that improves the matching accuracy of an input face image.

  According to one embodiment of the present invention for solving such a problem, a face image collation device that collates a plurality of input face images acquired sequentially with a registered face image registered in advance is provided. Such a face image matching device includes, for each of a plurality of registered persons, a storage unit that stores a plurality of face images having different face orientations as registered face images, and a registered face image for each input face image. Similarity degree calculation means for calculating the similarity degree, and voting for registered face images having a similarity degree equal to or higher than a predetermined value, and for each predetermined face image, a total of the voting values for a predetermined number of temporally continuous input face images Voting means for obtaining a vote total value, and person determining means for determining one of a plurality of registered persons as a person of the input face image based on the vote total value.

  In such a face image matching device, it is preferable that the person determining means determines a registered person having the largest sum of vote total values for each registered person as a person of the input face image.

  In the face image collating apparatus, it is preferable that the person determining means determines a registered person having the largest vote total value as a person of the input face image.

  Further, in such a face image collation apparatus, the person determination means inputs a registered person whose vote count value exceeds a predetermined collation threshold based on a predetermined number of temporally continuous input face images counted by the voting means. It is preferable to determine the person of the face image.

  ADVANTAGE OF THE INVENTION According to this invention, the face image collation apparatus which improved the collation precision of the input face image can be provided.

It is a block diagram which shows the structure of the walk-through type entrance / exit management system which concerns on one Example of this invention. It is a figure which shows an example of the registration face data based on a present Example. It is a figure which shows the several input image which imaged the person in which an imaging part is walking in time series. It is a figure which shows the voting process of the voting means which concerns on a present Example. It is a figure which shows the similarity table by which the similarity degree of the input face image of the person A and each registration face image of the registration face data of the person A was memorize | stored with a solid bar graph. It is a figure which shows the similarity table by which the similarity degree of the input face image of the person A and each registration face image of the registration face data of the person B was memorize | stored with a solid bar graph. It is a figure which shows the similarity table by which the similarity degree of the input face image of the person A and each registration face image of the registration face data of the person C was memorize | stored with a solid bar graph. It is a figure which shows the relationship between the registration face data of person A-C, the input face image of person A in the time t, t + 1, and t + 2, the time-sequential voting table, and a vote totaling table. It is a flowchart which shows an example of the registration process of the registration face data by the face data registration apparatus which concerns on a present Example. It is a flowchart which shows an example of the face collation process by the face collation apparatus which concerns on a present Example.

  The inventor of the present application has the highest similarity between the input face image and the registered face image of the person himself / her in various face orientations when the face orientation is the same, but the angle of the face orientation of the input face image is the same. The knowledge that it is high to some extent even in the case of near was obtained. Furthermore, it has been found that even if it is a face image of another person, the degree of similarity with the registered face image having the same face direction and a face direction close to that is increased to some extent. In addition, as described above, the similarity between the input face image and the registered face image with a face direction different from the face direction reflected in the input face image by a person different from the person shown in the input face image becomes very high. In this case, it was found that the degree of similarity between the registered face image showing a high similarity and the registered face image having a different face orientation does not increase.

  Based on these findings, the face image matching device according to the present invention stores registered face data having a plurality of registered face images for each registered person. For each acquired input image, the face image matching device obtains a similarity between the input face image and the registered face images of various registered faces of each registered person, and the input face image is based on the similarity. A registered face image similar to is selected, and a vote is performed for the person and face direction corresponding to the selected registered face image. Then, the face image collation device determines that the person who has voted the most as a result of voting is the person of the input face image. As a result, the face image collation device according to the present invention can detect the other person even if the input face image has a very high degree of similarity to the registered face data of another person whose face direction is completely different from the person of the input face image. It is possible to prevent the input face image from being erroneously determined as a person, and to improve face matching accuracy.

  In particular, the face image matching device according to the present invention is automatically operated by an authentication camera installed near the ceiling of the entrance, even if the user does not stop in front of an entrance such as a gate for authentication. The present invention is applicable to a face collation device of a “walk-through” type entrance / exit management system that performs continuous face-to-face shooting and performs face authentication using a plurality of photographed user's face images. In a walk-through type entrance / exit management system, a user can be authenticated by a so-called walk-through and permitted to enter a room, for example, only by passing through the imaging area of an authentication camera installed near the ceiling of a corridor. It becomes possible. For this reason, the user does not have to stop in front of the authentication camera near the entrance and face the camera in a predetermined direction such as the front direction. Bring very high convenience. In such a walk-through type entrance / exit management system, since a plurality of images whose face orientation changes every moment with the time of the user passing through the imaging area of the camera, the face image matching device according to the present invention is Face matching can be performed as described above based on the similarity between a plurality of input face images with different face orientations and a plurality of registered face images with different face orientations registered in the system in advance.

  Hereinafter, an embodiment in which a face image matching device according to the present invention is applied to a face matching device of a walk-through type entrance / exit management system will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a walk-through type entrance / exit management system 1 according to an embodiment of the present invention. As shown in FIG. 1, the walk-through type entrance / exit management system 1 includes imaging units 10 and 30, a face data registration device 20, a face matching device 40, and a face matching result processing unit 50. The image capturing unit 10 and the face data registration device 20, the image capturing unit 30 and the face collation device 40, the face data registration device 20 and the face collation device 40, and the face collation device 40 and the face collation result processing unit 50 are in various formats. Can be connected by wired communication such as a communication cable or internal wiring, or wireless communication such as radio waves or infrared rays.

  As the imaging unit 10, for example, a known camera including an imaging element such as a CCD element or a C-MOS element, an optical system component, an A / D converter, and the like can be used. This camera may be a black and white type having sensitivity in the near-infrared wavelength band, but is preferably a color type having sensitivity in the visible light wavelength band so that the skin color portion can be easily extracted. Further, the accuracy such as the resolution of the camera can be used in accordance with the storage capacity and processing capability of the face data registration device 20 and the face matching device 40, the accuracy of face matching desired by the face matching device 40, and the like. . Furthermore, the registration image may be a color image or a gray scale image represented by multi-tone pixel values, but a color image having a luminance resolution of about 8 bits is preferable.

  The imaging unit 10 photographs a person who is permitted to enter a room, such as an employee permitted to enter the room, and uses the obtained two-dimensional image data as a registered person registration image to be registered by the face data registration device 20 via a communication or the like. The data is sent to the data registration device 20.

  The face data registration device 20 can be configured by an information processing device such as a so-called computer, for example. The face data registration device 20 uses the registration image of the registered person received from the imaging unit 10 to generate registered face data having registered face images of various faces of the registered person and angle information regarding the face direction. Then, the face data registration device 20 sends the generated registered face data and angle information to the face collation device 40 via a computer-readable storage medium such as a CD-ROM, DVD-R / W and / or flash memory or communication. You can register.

  As illustrated in FIG. 1, the face data registration device 20 includes an image input interface unit 210, a control unit 220, a storage unit 230, and a registered face data output interface unit 240.

  The image input interface unit 210 is an interface unit through which the control unit 220 receives a registered person registration image from the imaging unit 10. Therefore, the image input interface unit 210 is suitable for, for example, LAN, universal serial bus (USB), SCSI, wireless LAN, or infrared communication according to the communication method for connecting the imaging unit 10 and the face data registration device 20. Including a simple communication interface adapter.

  The control unit 220 includes a processor and its peripheral circuits, and controls the registered face data registration process of the face data registration device 20 by the processor executing various programs stored in the storage unit 230. The processor and storage unit 230 may include a microprocessor or an ASIC. The program executed by the processor may be provided by a computer-readable storage medium such as a CD-ROM and / or DVD-R / W or communication, and stored in the storage unit 230 or may be provided as firmware. .

  The control unit 220 receives a registration image of a registered person from the imaging unit 10 via the image input interface unit 210, generates registration face images of various directions of the registered person using the received registration image, and registers them. The data is stored in the storage unit 230 as face data. Further, the control unit 220 can send the registered face data stored in the storage unit 230 to the face matching device 40 via the registered face data output interface unit 240.

  FIG. 2 is a diagram illustrating an example of registered face data 422 according to the present embodiment. Each registered face data 422 is stored in association with personal information related to each registered person such as identification information of the registered person. As shown in FIG. 2, the registered face data 422 includes registered face images with various face orientations. In this example, the face orientation is changed to ± 30 degrees in increments of 15 degrees in the vertical direction (pitch angle) and the horizontal direction (yaw angle), respectively, with the registered face image of “angle A13” facing directly in front. The registered face data 422 includes registered face images of 5 = 25 types of face orientations. Therefore, for example, when the registered face data 422 shown in FIG. 2 is registered for 100 people, 25 × 100 = 2500 registered face images are stored in the storage unit 230 as the registered face data 422.

  In the following, the types of face orientations will be described as 25 types shown in FIG. 2, but the present invention is not limited to this. For example, the pitch angle and yaw angle increments and ranges according to the shooting range of the imaging unit 10, the storage capacity of the face data registration device 20 and the face matching device 40, the processing capability, the desired matching accuracy of the face matching device 40, and the like. Can be arbitrarily changed to increase or decrease the number of registered face images. For example, if the imaging range of the imaging unit 10 is the pitch angle ± 30 degrees and the yaw angle ± 45 degrees, the control unit 220 sets the pitch angle and yaw angle increments to 15 degrees, and 5 × 7 = 35 types. A registered face image for the face may be generated. Further, the control unit 220 may generate 7 × 7 = 49 types or 9 × 9 = 81 types of registered face images for face orientation. However, from the viewpoint of collation accuracy, the pitch angle and yaw angle increment are preferably 15 degrees or less.

  Each registered face image of the registered face data 422 is stored in the storage unit 230 in association with the face orientation. For example, as shown in FIG. 2, the control unit 220 assigns identification information or identification numbers such as “angle A1” to “angle A25” to 25 types of face orientations, and each face orientation identification information is assigned to each face orientation identification information. The registered face image can be associated and stored in the storage unit 230.

  Further, the control unit 220 can cause the storage unit 230 to store each face direction identification information in association with a combination of the pitch angle and yaw angle of the face direction as angle information. For example, as shown in FIG. 2, the 25 types of face orientations “angle A1” to “angle A25” are centered on the face orientation “angle A13” facing the front of the pitch angle = 0 degrees and the yaw angle = 0 degrees. , It is associated with 25 combinations of 5 types of pitch angles and 5 types of yaw angles up to ± 30 degrees in increments of 15 degrees.

  In the example of FIG. 2, “angle A11” is a face facing 30 degrees to the left and pitch angle = 0 degrees and yaw angle = −30 degrees, and “angle A15” is 30 degrees to the right. Face angle with pitch angle = 0 degrees and yaw angle = + 30 degrees, “angle A3” is 30 degrees face up, pitch angle = −30 degrees and yaw angle = 0 degrees, “angle A23” is below It is associated with angle information of a combination of a pitch angle = + 30 degrees and a yaw angle = 0 degrees with a face orientation of 30 degrees.

  For example, the control unit 220 assigns numbers 0 to 4 to the above five types of pitch angles and yaw angles, and uses a combination of two numbers 0 to 4 as face direction identification information. May be stored as angle information.

  As illustrated in FIG. 1, the control unit 220 includes a registration face image extraction unit 2210, a 3D face model generation unit 2220, and a registered face data generation unit 2230. The control unit 220 controls the registration process of the registered face data 422 in the face data registration device 20 by causing the processor to execute a program for executing these means stored in the storage unit 230.

  The registration face image extraction unit 2210 extracts a face area in which a person's face is reflected as a registration face image from the registration image received from the imaging unit 10 via the image input interface unit 210.

  In order to extract a face area, the registration face image extraction means 2210 can use various known methods. For example, the registration face image extraction unit 2210 calculates one or a plurality of feature amounts from a partial region in the registration image, and when the predetermined amount condition that the feature amount is considered to correspond to a person's face is satisfied, The partial area may be extracted as a face area. Specifically, the registration face image extraction unit 2210 performs edge pixel extraction using, for example, a Sobel filter and the like, and features such as the direction distribution of the edge pixel in the partial region or the luminance distribution near the edge Can be calculated. In addition, the predetermined conditions described above can be determined in advance by obtaining the values of these feature values from a plurality of images obtained by photographing a person's face in advance and determining the range thereof.

  As another method for extracting a face area, the registration face image extraction unit 2210 performs template matching with a registration image using a face area template prepared in advance, or performs face matching from an edge image of the registration image. Alternatively, the face area may be extracted by detecting the edge distribution of the contour shape of the head area. Further, the registration face image extraction unit 2210 may extract a face region by combining these known methods.

  The three-dimensional face model generation unit 2220 generates a three-dimensional face model from the registration face image received from the registration face image extraction unit 2210. The three-dimensional face model is for the registered face data generation means 2230 to generate registered face images for various faces by reproducing the head of a registered person.

  In order to generate a three-dimensional face model, the three-dimensional face model generation means 2220 can use any of various known three-dimensional face model generation methods. For example, as described in JP-A-2009-2111151, the three-dimensional face model generation unit 2220 uses registration information for three-dimensional face shape data including facial unevenness information and three-dimensional arrangement information of feature points. The skin color and texture information of the face image can be mapped to generate a three-dimensional face model.

  The registered face data generation unit 2230 generates registered face data 422 from the 3D face model received from the 3D face model generation unit 2220.

  For this purpose, the registered face data generation unit 2230 can use any of various known methods for acquiring two-dimensional face images of various face orientations from the three-dimensional face model. For example, the registered face data generation means 2230 reproduces the head of the three-dimensional face model by changing it in various directions as described in the above-mentioned Japanese Patent Application Laid-Open No. 2009-211111, and this is reproduced in the two-dimensional space. By projecting, registered face images of various face orientations can be generated. Here, the registered face image may be image data itself such as a pixel value, but arbitrary information that can be acquired from image data indicating the feature of a registered person's face such as arrangement information of feature points and / or texture information. May be included.

  The storage unit 230 stores programs and data for the face data registration device 20 to execute various operations, and includes a semiconductor memory such as a RAM, a ROM, and / or an EPROM. The storage unit 230 may include a magnetic storage medium such as a hard disk and / or an optical storage medium such as a CD-ROM and / or DVD-R / W and an access device thereof.

  The registered face data output interface unit 240 is an interface unit that sends the registered face data 422 generated by the control unit 220 to the face matching device 40. Therefore, similar to the image input interface unit 210, the registered face data output interface unit 240 includes an appropriate communication interface adapter or the like according to a communication method for connecting the face data registration device 20 and the face matching device 40. . When the registered face data 422 is transferred from the face data registration device 20 to the face matching device 40 by a computer-readable storage medium, the registered face data output interface unit 240 may not be provided.

  The imaging unit 30 captures a walking person to be collated by the face collation device 40 a plurality of times, and sequentially transmits a plurality of two-dimensional image data obtained by photographing to the face collation device 40 through communication or the like as an input image.

  In the present embodiment, the imaging unit 30 is, for example, one or a plurality of surveillance cameras installed near the ceiling of a corridor in front of the entrance of a room that manages the entrance of an office building. For example, a CCD element or a C-MOS A known camera including an image sensor such as an element, an optical system component, an A / D converter, and the like can be used. The camera of the imaging unit 30 may be a black and white type having sensitivity in the near-infrared wavelength band, similar to the camera of the imaging unit 10, but is a color type having sensitivity in the visible light wavelength band so that the skin color portion can be easily extracted. Is preferred. Further, the accuracy of the resolution of the camera of the imaging unit 30 can be appropriate depending on the situation of the monitoring area and the desired face matching accuracy. Furthermore, the input image may be a color image or a grayscale image represented by multi-gradation pixel values, like the registration image, but a color image having a luminance resolution of about 8 bits is preferable. The imaging unit 30 can take an image of the monitoring area at predetermined time intervals (for example, 100 msec) and sequentially send the acquired monitoring image as an input image to the face matching device 40.

  FIG. 3 is a diagram showing a plurality of input images in which a surveillance camera installed in the vicinity of the ceiling of a corridor in front of the entrance of a room that manages entry in an office building or the like takes a time series of users in a room that is walking. is there. FIG. 3 shows input face images 421t0, 421t1, and 421t2 obtained by photographing the face of the user whose surveillance camera moves in the photographing area at each time t, t + 1, and t + 2. Since the face direction of the user passing in front of the surveillance camera changes little by little with time, the face of the user having a slightly different face direction is reflected in the input face images 421t0, 421t1, and 421t2.

  The face matching device 40 can be configured by an information processing device such as a so-called computer, for example. Then, the face collation device 40 collates whether or not the person in the input image received from the imaging unit 30 matches a registered person such as a person who is permitted to enter the room registered in the face collation device 40, and the collation result is obtained. The result is sent to the face matching result processing unit 50 by communication or the like.

  As illustrated in FIG. 1, the face matching device 40 includes an image input interface unit 410, a storage unit 420, a registered face data input interface unit 430, a control unit 440, and a face matching result output interface unit 450. .

  The image input interface unit 410 is an interface unit that the control unit 440 receives an input image from the imaging unit 30. Therefore, the image input interface unit 410 is configured to include an appropriate communication interface adapter or the like according to a communication method for connecting the image capturing unit 30 and the face collation device 40, similarly to the image input interface unit 210 described above.

  The storage unit 420 stores a program and data for the face matching device 40 to execute various operations, and includes a semiconductor memory such as a RAM, a ROM, and / or an EPROM, for example, like the storage unit 230 described above. The The storage unit 420 may include a magnetic storage medium such as a hard disk and / or an optical storage medium such as a CD-ROM and / or DVD-R / W and an access device thereof.

  As described above, the registered face data 422 generated by the face data registration device 20 is sent to and stored in the storage unit 420 via a computer-readable storage medium or communication.

  The registered face data input interface unit 430 is an interface unit that the control unit 440 receives the registered face data 422 from the face data registration device 20 through communication or the like. Therefore, the registered face data input interface unit 430 includes an appropriate communication interface adapter or the like according to the communication method for connecting the face data registration device 20 and the face collation device 40, similarly to the image input interface unit 210 described above. Is done. When the registered face data 422 is transferred from the face data registration device 20 by a computer-readable storage medium, the registered face data input interface unit 430 may not be provided.

  Similar to the control unit 220 described above, the control unit 440 includes a processor and its peripheral circuits, and executes various programs stored in the storage unit 420 by the processor, whereby the face verification process of the face verification device 40 is performed. To control. A program executed by the processor is provided in the same manner as the control unit 220 and stored in the storage unit 420.

  As shown in FIG. 1, the control unit 440 includes an input face image extraction unit 4411, a similarity calculation unit 4412, a voting table update unit 4413, a voting unit 4414, and a person determination unit 4415, and stores them. The face matching process in the face matching device 40 is controlled by the processor executing a program for executing these means stored in the unit 420. The control unit 440 sequentially receives input images from the imaging unit 30 via the image input interface unit 410, collates the received input images with the registered face data 422 stored in the storage unit 420, and compares the collation result with the face collation result. The result is sent to the face matching result processing unit 50 via the output interface unit 450.

  The input face image extraction unit 4411 extracts a face region in which a person's face is reflected from the input image received from the imaging unit 30 via the image input interface unit 410, as in the case of the registration face image extraction unit 2210 described above. Input face image. This input face image may include arbitrary information indicating the characteristics of a person's face in the input image obtained from the input image, as in the above-described registered face image.

  The similarity calculation unit 4412 calculates the similarity between the input face image extracted by the input face image extraction unit 4411 and all the registered face images of all the registered face data 422 stored in the storage unit 420. . Then, the similarity calculation unit 4412 stores the calculated similarity in the storage unit 420 as a similarity table in association with the person and face orientation of the registered face image. Here, the similarity table is prepared for each registered face data 422 for one person, and has an area in which similarities are stored for the number of face orientations of all registered face images included in each registered face data 422. It is a table. The storage unit 420 stores a similarity table for the number of registered persons corresponding to all the registered face data 422 stored in the storage unit 420.

  The similarity may be a value that increases or decreases linearly or nonlinearly as the input face image and the registered face image are similar. Hereinafter, the similarity will be described as a value that increases as the similarity increases. In this embodiment, since the input face image and each registered face image are two-dimensional face images, the similarity calculation unit 4412 can calculate the similarity using a general method.

  For example, the similarity calculation unit 4412 uses the image features such as feature point arrangement information and texture information, and the similarity between the input face image and the registered face image (that is, the face reflected in the input face image and the registered face image). (Similarity with the face reflected in the screen). The similarity calculation unit 4412 may obtain the similarity by using each feature alone, but since the accuracy of the similarity becomes higher when a plurality of features are used together, a plurality of features are used together in this embodiment. To obtain the similarity.

There are various methods for calculating the degree of similarity between images depending on the purpose. For example, when two image features of feature point arrangement information and texture information are used, the similarity can be obtained as in the following equation (1).
Similarity (p, q) = αF (p, q) + βG (p, q) (1)

  Here, p and q are images to be compared, α and β are predetermined coefficients, F () is a function that evaluates the degree of matching of feature point positions, and G () is a function that evaluates the degree of matching of textures. It is.

  The function F () may be, for example, a function for obtaining the reciprocal of the sum of Euclidean distances between corresponding feature points with respect to the coordinates of each feature point of the input face image and the registered face image.

  Further, the function G () is obtained by comparing the textures in the vicinity of the corresponding feature points of the input face image and the registered face image as disclosed in, for example, Japanese Patent No. 2690132, and the difference in the histogram matching. A function for obtaining the reciprocal of may be used. Alternatively, the function G () obtains the reciprocal of the difference in histogram matching obtained by normalizing the size, brightness, and color for an inverted trapezoidal region connecting both eyes and both corners of the input face image and registered face image. It can be a function.

  As described above, when calculating the similarity using the feature of another face image, the similarity calculating unit 4412 obtains the degree of matching of the feature instead of the above F () and G (). The similarity can be calculated using a function suitable for the above. Alternatively, the similarity calculation unit 4412 may calculate the similarity between the input face image and the registered face image by pattern matching between the input face image and the registered face image.

  The voting table update unit 4413 removes old voting values related to input images acquired in the past from the voting total table stored in the storage unit 420. Here, the vote count table is an area prepared for each registered face data 422 for one person and storing the vote count values for the number of face orientations of all the registered face images included in each registered face data 422. It is a table that has. Therefore, a voting total table for the number of registered persons corresponding to all the registered face data 422 stored in the storage unit 420 is stored in the storage unit 420. In the vote counting table, for a plurality of input images photographed sequentially up to that point, the vote values totaled corresponding to the persons and face orientations of all registered face images similar to the input face image are displayed as vote counting values. Is stored by voting means 4414 described later. A method for counting vote values by the voting means 4414 and a method for storing them in the vote counting table will be described in detail in the explanation of the voting means 4414 later.

  For example, the voting table updating unit 4413 can multiply each voting total value stored in the voting total table by a predetermined ratio smaller than 1 every predetermined time. Thereby, the voting table update unit 4413 can gradually reduce the influence of each vote value accumulated regarding the input image acquired before that on each vote value accumulated regarding the input image acquired thereafter. The old vote value can be removed from the vote totaling table as time passes.

  When the time-series voting table is stored in the storage unit 420 for the latest input images within the predetermined number, the voting table update unit 4413 is the oldest among the time-series voting tables stored every predetermined time. By discarding the time-series voting table related to the input image, the oldest voting value can be removed from the vote counting table. Here, the time-series voting table is an area prepared for each registered face data 422 for one person and storing voting values for the number of face orientations of all registered face images included in each registered face data 422. It is a table that has. Therefore, a time-series voting table for the number of registered persons corresponding to all the registered face data 422 stored in the storage unit 420 is stored in the storage unit 420 for the number of input images for each shooting time of the input image. ing. In the time-series voting table, the voting value corresponding to each person and face direction of all registered face images similar to the input face image is input by the voting means 4414 described later with respect to the input images taken at the respective shooting times. Remembered. The vote total value stored in the vote total table is a value obtained by totaling the vote values stored in the time series vote table for all the input images within the latest predetermined number for each person and face orientation. . Therefore, the voting table updating unit 4413 can remove the oldest voting value from the voting total table by discarding the time-series voting table related to the oldest input image. The predetermined number is preferably about 10. Further, when the voting table update unit 4413 does not use the time series voting table for updating the vote totaling table, the time series voting table may not be provided. The method of storing the voting value in the time series voting table by the voting unit 4414 will be described in detail in the description of the voting unit 4414 below.

  FIG. 4 is a diagram showing the voting process of the voting means 4414 according to the present embodiment. 4A shows an example of registered face data 422, and FIGS. 4B and 4C show examples of time-series voting tables 4241 and 4242 corresponding to the persons of the registered face data 422, respectively. Indicates. As described above, the time series voting table is not always necessary, but hereinafter, the voting means 4414 will be used to explain the voting value that is added to the voting total value of the voting total table.

  The voting unit 4414 reads the similarity in the similarity table stored in the storage unit 420 and determines that the registered face image whose similarity is equal to or greater than a predetermined voting threshold is similar to the input face image 421. For example, the predetermined voting threshold value may be a value obtained by calculating an average value and a variance of the similarity between the test input image and the registered face image and adding twice the variance to the average value.

  The voting unit 4414 may determine that the top N registered face images among all the similarities stored in the similarity table are similar to the input face image 421. In this case, N is a positive integer and is preferably, for example, 1 to 3% of the total number of registered face images. For example, as described above, when 2500 registered face images are stored in the storage unit 420 as the registered face data 422, N may be 25 to 75.

  Alternatively, the voting unit 4414 may determine that a registered face image whose similarity is within the upper predetermined number for each person is similar to the input face image 421. The predetermined number M in this case is a positive integer, and may be 1 to 3 or the like.

  For example, it is assumed that the voting unit 4414 determines that only one registered face image indicated by reference numeral 4221 is similar to the input face image 421 among the registered face images illustrated in FIG. This is likely to occur when the person of the registered face data 422 is different from the person of the input face image 421. In this case, the voting means 4414 adds, for example, a vote value “1” to the vote total value corresponding to the face orientation “angle A12” of the registered face image 4221 of the vote total table. In the time-series voting table 4241 of FIG. 4B, the vote value “1” to be added is indicated at the position corresponding to the face direction “angle A12” of the registered face image 4221, and the others are “0”. .

  Similarly, it is assumed that the voting unit 4414 determines that nine registered face images surrounded by a frame denoted by reference numeral 4222 are similar to the input face image 421 among the registered face images illustrated in FIG. This is likely to occur when the person of the registered face data 422 is the same as the person of the input face image 421. That is, since the person of the input face image 421 is the person of the registered face data 422, it is determined that the input face image 421 is similar to the input face image 421 even if the face orientation of the registered face image is slightly different.

  In this case, the voting means 4414 corresponds to the face orientations “angle A6” to “angle A8”, “angle A11” to “angle A13”, and “angle A16” to “angle A18” of the registered face image 4222 of the vote count table. For example, a vote value “1” can be added to the vote total value. In the time series voting table 4242 of FIG. 4C, the face orientations “angle A6” to “angle A8”, “angle A11” to “angle A13”, and “angle A16” to “angle A18” of the registered face image 4222 are displayed. The voting value “1” to be added is shown in nine locations corresponding to “0”, and the others are “0”.

  Next, since the similarity between the input face image and the registered face image with different face orientations in a different person is prominently high, an erroneous determination that erroneously determines the person of the input face image as another person is voting means. How it is avoided by 4414 votes will be described.

  FIGS. 5 to 7 illustrate similarity tables 423a to 423a in which the similarity between the input face image 421at0 of the person A at time t and the registered face images of the registered face data 422a to 422c of the persons A to C is stored. It is the figure which showed c by the solid bar graph. Here, it is assumed that the face direction of the input face image 421at0 is “angle A12”.

  FIG. 5 shows an example of the similarity table 423a in which the similarity between the input face image 421at0 of the person A and each registered face image of the registered face data 422a of the person A who is the person is stored. In the similarity table 423a of FIG. 5, since the registered face data 422a belongs to the person A himself / herself of the input face image 421at0, the similarity 423a1 to the registered face image 422a1 having the same face orientation “angle A12” as the input face image 421at0. Indicates the highest value. Also, the similarity with the registered face image 422a2 facing the surroundings of the similarity 423a1 is a high value above a certain level. This is because the registered face data 422a is that of the person A of the input face image 421at0, so the facial feature point arrangement information, color information (texture), etc. are common, and the degree of similarity even if the face orientation is slightly different Because it becomes higher.

  FIG. 6 shows an example of a similarity table 423b in which the similarity between the input face image 421at0 of the person A at time t and each registered face image of the registered face data 422b of the person B who is another person is stored. . In the similarity table 423b of FIG. 6, the person B of the registered face data 422b is different from the person A of the input face image 421at0, so the similarity is not as high as the similarity table 423a of FIG. Nevertheless, the similarities 423b1 and 423b2 between the registered face images 422b1 and 422b2 and the input face image 421at0 are equal to or greater than a predetermined voting threshold.

  Of the similarities in the similarity table 423b, the similarity 423b1 shows a slightly higher value than the others. This is probably because the face orientation “angle A12” of the registered face image 422b1 is the same as that of the input face image 421at0, and the arrangement of the feature points of the face image is similar, so the degree of similarity is high. The similarity 423b2 of the registered face image 422b2 will be described later.

  FIG. 7 shows an example of the similarity table 423c in which the similarity between the input face image 421at0 of the person A at time t and each registered face image of the registered face data 422c of the person C who is another person is stored. . In the similarity table 423c of FIG. 7, the person C of the registered face data 422c is different from the person A of the input face image 421at0, similar to the similarity table 423b of FIG. not high. Nevertheless, the similarity 423c1 between the registered face image 422c1 and the input face image 421at0 is greater than or equal to a predetermined voting threshold.

  Of the similarities in the similarity table 423c, the similarity 423c1 shows a slightly higher value than the others. Similar to the registered face image 422b1 in FIG. 6, the face orientation “angle A12” of the registered face image 422c1 is the same as that of the input face image 421at0. Probably higher.

  On the other hand, in the similarity table 423b of FIG. 6, the similarity 423b2 protrudes compared to others and the similarity is high. This is likely to occur when the extraction of facial feature points, such as the nose tip, which immediately affects the position information of other feature points when extraction fails, is difficult. In such a case, the arrangement of the feature points extracted from the input face image 421at0 of the person A and the arrangement of the feature points extracted from the registered face image 422b2 of the person B with different face orientations are very close to each other. In addition, as a result, the textures near the feature points are similar to each other, so that the degree of similarity may be higher than expected.

  In the example of FIGS. 5 and 6, the input face image 421at0 of the face orientation “angle A12” facing left slightly toward the person A and the face orientation facing right toward the person B who is a round face than the person A The similarity 423b2 with the registered face image 422b2 of “angle A15” is higher than the similarity 423a1 with the registered face image 422a1 with the same face orientation “angle A12” of the person. In this case, when the one showing the highest value is simply selected from the similarities between the input face image 421at0 and each registered face image of each registered face data 422a to 422c, the person B is the person of the input face image 421at0. Will be erroneously determined.

  In order to prevent such an erroneous determination, the voting unit 4414 selects a registered face image determined to be similar with reference to the similarity stored in the similarity table, and a person associated with the selected registered face image. And vote for face orientation. The voting process of the voting means 4414 is performed for a plurality of input face images.

  FIG. 8 shows the registered face data 422a to c of the persons A to C, the input face images 421t0 to 421t0 to 2 at the times t, t + 1 and t + 2, and the time series voting tables 424at0 to 2, 424bt0 to 2 and 424ct0 to 2 It is a figure which shows the relationship with total table 425a-c. The registered face data 422a-c of the persons A to C in FIG. 8 and the input face image 421t0 at the time t correspond to the registered face data 422a-c and the input face image 421t0 of the persons A to C in FIGS. Referring to FIG. 8, the manner in which voting means 4414 collects votes on the registered face images of person A himself in input face images 421 at 0-2 will be described.

  As shown in FIG. 8, it is assumed that an input face image 421at0 of the person A with the face orientation “angle A12” is obtained at time t. For each of the input face image 421at0 and the registered face data 422a-c of the persons A to C, as shown in FIGS. 5 to 7, the registered face images 422a2, 422b1, stored in the similarity tables 423a-c, Assume that 422b2 and 422c2 are selected as registered face images similar to the input face image 421at0. In this case, as shown in the time-series voting tables 424at0 to ct0, the voting unit 4414 has votes corresponding to the face orientations of the registered face images 422a2, 422b1, 422b2, and 422c1 in the vote counting table for each of the persons A to C. For example, the vote value “1” is added to the total value.

  For example, in the time series voting table 424at0 of FIG. 8, the vote value “1” surrounded by “O” corresponds to the face direction “angle A12” of the person A, and the input of the person A himself / herself in the input face image 421at0. It is determined that the registered face image having the same face orientation “angle A12” as the face image 421 at 0 and the face orientation in the vicinity thereof is determined to be similar to the input face image 421 at 0 and the vote value “1” is added. In the time-series voting tables 424bt0 and 424ct0, the vote value “1” surrounded by “Δ” corresponds to the face orientation “angle A12” of the persons B and C, and the person A in the input face image 421at0 It is determined that the registered face image with the same face direction “angle A12” as the input face image 421at0 for other people B and C is determined to be similar to the input face image 421at0 and the vote value “1” is added. . Furthermore, the vote value “1” surrounded by “□” in the time-series vote table 424bt0 corresponds to the face direction “angle A15” of the person B, and is the person A of the input face image 421at0 and the person B of the other person. This indicates that a registered face image having a different face orientation “angle A15” accidentally becomes similar to the input face image 421at0 and the vote value “1” is added.

  It is assumed that the face orientation of the person A slightly changes at time t + 1, and an input face image 421at1 having the face orientation “angle A11” is obtained. For this reason, the input face image 421at1 has the highest similarity to the registered face image having the same face orientation “angle A11” as the input face image 421at1 in the registered face data 422a of the person A of the input face image 421at1, and the face orientation The degree of similarity with a registered face image whose face is close to “angle A11” also has a high value above a certain level.

  In this case, the vote value “1” surrounded by “o” in the time series vote table 424at1 corresponds to the same face A “angle A11” as the input face image 421at1, and the input face image 421at1 It is determined that a registered face image having the same face direction “angle A11” as the input face image 421 at 1 of the person A himself and a face direction in the vicinity thereof is determined to be similar to the input face image 421 at 1 and the vote value “1” is added. Yes. In the time series voting tables 424bt1 and 424ct1, the vote value “1” surrounded by “Δ” corresponds to the face orientation “angle A11” of the persons B and C, and the person A in the input face image 421at1 It is determined that the registered face image of the other person B and C but having the same face orientation “angle A11” as the input face image 421at1 is similar to the input face image 421at1, and the vote value “1” is added. .

  It is assumed that the face orientation of the person A further changes at time t + 2, and an input face image 421at2 having the face orientation “angle A21” is obtained. For this reason, the input face image 421at2 has the highest similarity to the registered face image of the face orientation “angle A21” in the registered face data 422a of the person A of the input face image 421at2, and the angle to the face orientation “angle A21”. The degree of similarity with a registered face image facing a face close to is also a high value above a certain level.

  In this case, the vote value “1” surrounded by “◯” in the time series vote table 424at2 corresponds to the face direction “angle A21” of the person A, and the input face image of the person A himself / herself in the input face image 421at2 It is determined that the registered face image having the same face orientation “angle A21” as 421 at 2 and the face orientation in the vicinity thereof is determined to be similar to the input face image 421 at 2 and the vote value “1” is added. In the time series voting tables 424bt2 and 424ct2, the vote value “1” surrounded by “Δ” corresponds to the face angle “angle A21” of the persons B and C, and the person A in the input face image 421at2 It is determined that the registered face image of the other person B and C but having the same face orientation “angle A21” as the input face image 421at2 is similar to the input face image 421at2, and the vote value “1” is added. . Further, in the time series voting table 424ct2, the vote value “1” surrounded by “□” corresponds to the face direction “angle A25” of the person C and is different for the person C different from the person A of the input face image 421at0. The registered face image with the face orientation “angle A25” is accidentally similar to the input face image 421at2 and the vote value “1” is added, similarly to the registered face image 422b2 of FIG. The similarity of the registered face image of the person C with the face orientation “angle A25” with the input face image 421 at 2 is higher than the similarity of the registered face image with the same face orientation “angle A21” with the same person A as the input face image 421 at 2. To do.

  In the example shown in FIG. 8, if it is determined that the person of the registered face image having the highest similarity with the input face image is the person of the input face image, the face orientation of the input face image 421at0 and the person B “angle A15 ”Is higher than the similarity between the input face image 421 at 0 and the registered face image of the face direction“ angle A12 ”of the person A, so that the person in the input face image 421 at 0 is mistakenly the person B. It will be judged. Further, the similarity between the input face image 421at2 and the registered face image of the face orientation “angle A25” of the person C is higher than the similarity between the input face image 421at2 and the registered face image of the face orientation “angle A21” of the person A. The person in the input face image 421at2 is erroneously determined to be the person C.

  Therefore, the voting means 4414 according to the present invention, as shown in the vote totaling tables 425a to 425c, for each of the persons A to C, the times indicated by the time series voting tables 424at0 to 424bt0 to 2 and 424ct0 to 424ct0. The sum of the vote values at t, t + 1, and t + 2 is calculated, and the calculated sum of the vote values is stored in the vote count tables 425a to 425c as the vote count values. Then, among the vote total values stored in the vote total tables 425a to 425c, the vote total values of the face orientation “angle A16” and “angle A17” of the person A are “3”, which is the maximum. This is because when the user's face passing through the imaging area is photographed in time series as in walk-through authentication, the face orientation of the photographed user changes, but the change becomes moderate. Accordingly, regarding the vote value for the person A of the input face image, the vote count value is increased as a result of voting because the vote value is added in the same face direction as the input face image and a face direction with an angle close to that face direction. . On the other hand, the person B and C do not have a high vote value like the person A as shown in the vote totaling tables 425b and 425c. This is because the similarity between the input face image, the person of the input face image, and the registered face image of others is obtained even when a plurality of input face images with slightly different face orientations are obtained as in the walk-through face authentication. Since the similarity between the input face image and the registered face image of the person is suddenly caused by the failure of feature point extraction as described above, the similarity with the registered face image of another person is input. This is because if the face orientation is slightly different from the face image, it tends to decrease immediately.

  The person determining unit 4415 selects the person with the largest vote total value as the most voted person from the vote totaling table in which the voting result by the voting unit 4414 is stored, and determines that the person is the person of the input face image. For example, in the vote count tables 425a to 425c of FIG. 8, the person determination unit 4415 selects the person A corresponding to the vote count table 425a having the maximum vote count value “3”. Then, the person determining unit 4415 compares the maximum vote total value “3” with a predetermined collation threshold, and if the maximum vote total value “3” is equal to or greater than the collation threshold, the person A is input to the input face image. It is determined that the person 421t0-2 is a person shown in the input image. The matching threshold value is determined according to the number of time-series voting tables used in the voting unit 4414, and can be set to 30% of the number, for example.

  The person determination unit 4415 may further select the person who has obtained the maximum total by summing the vote count values for each face in the vote count table for each person as the person of the input face image. For example, in the example shown in FIG. 8, the total of the ballot count values for each face in the person A vote count table 425 a is 21, and the total of the ball count values for each face in the person B vote count table 425 b is 4. In the vote count table 425c of the person C, the sum of the vote count values for each face is 4. In this case, the person A whose total sum of the vote total values for each person is 21 is selected as the person of the input face images 421at0 to 421at2. In addition, an area for storing vote count values for the number of registered persons is prepared in the vote count table, and in this vote count table, the voting means 4414 vote for a person corresponding to a registered face image similar to the input face image. However, similar results can be obtained. Furthermore, the person determination means 4415 has the largest vote total value among the vote total values for each face direction stored in all the ballot total tables, and also calculates the vote total value for each face in the vote total table. The registered person who has obtained the maximum sum total for each person may be the person of the input face image.

  In the above-described example, the voting unit 4414 calculates the sum by accumulating the results of voting for each person and each face direction at each time over a plurality of input images. However, the voting means 4414 limits the face direction to be voted to the face direction of the input face image and a face direction whose angle is close to the face direction, and calculates the sum of the vote values of the limited face direction for each person. Good. That is, the voting means 4414 votes for the face orientation of the registered face image determined to be similar to the input face image based on the similarity stored in the similarity table at each time, and the sum of the vote values is the maximum. It is estimated that the face orientation is the face orientation of the input face image at that time. The voting means 4414 limits the voting value for each person at each time to a face direction (for example, “4 neighborhoods”, “8 neighborhoods” or “24 neighborhoods”) within an angle within a certain range from the estimated face orientation. Sum total v_sum (i, t) is obtained. Here, the subscript i is an identifier relating to the registered person, and t is time. The voting means 4414 then performs v_sum (i, t), v_sum (i, t + 1) and v_sum (i, t + 2) for each person at all shooting times, for example, three times t, t + 1, and t + 2 in the example of FIG. ) For the total sum v_sum_all (i). At this time, for example, the voting unit 4414 adds the weight to the vote value as the face direction of the input face image estimated as the face direction of the registered face image is estimated in the vote count table, and totals the total sum of vote values for each person. May be.

  In this case, the person determination unit 4415 determines that the person i corresponding to the maximum value among the total sum v_sum_all (i) for each person is the person of the input face image. As a result, even if the face orientation of the registered face image is slightly different, the similarity of the input face image is higher for the person who is the input face image, while the similarity is lower for the other person. Can be improved. In addition, the face orientation is close to the estimated face orientation not only when the face orientation changes slowly but also when the face orientation changes suddenly, as in the case of input face images taken in time series by walkthrough. Since the sum of the vote values is obtained only for, the face collation accuracy can be increased.

  As described above, the voting unit 4414 counts vote values for each person and each face direction of the registered face image determined to be similar to the input face images 421 at 0 to 2. As a result, the face collation apparatus 40 according to the present invention enables the registered face of the person A and the person B having a different face shape “angle A15” as in the registered face image 422b2 of the registered face data 422b of FIG. Even if the similarity between the image and the input face image 421at0 of the person A is unexpectedly high, it is possible to avoid the person B being erroneously determined as the person of the input face images 421at0 to 421at2.

  The face matching result output interface unit 450 is an interface unit for the control unit 440 to send a face matching result to the face matching result processing unit 50. Therefore, the face matching result output interface unit 450 includes an appropriate communication interface adapter or the like according to the communication method for connecting the face matching result processing unit 50 and the face matching device 40, similarly to the image input interface unit 210 described above. Composed.

  The face matching result processing unit 50 receives the matching result from the face matching device 40 via communication, and performs appropriate processing based on the matching result. The face matching result processing unit 50 may be one or a plurality of devices separated from the face matching device 40 or may be integrated with the face matching device 40.

  For example, the face matching result processing unit 50 may be an electric lock that locks / unlocks a door by an electric signal. Then, when the face collation device 40 sends a collation result indicating that the person in the input image is one of the entry-permitted persons registered to the electric lock by an electric signal, the electric lock unlocks the door and enters the room. Otherwise, the electric lock can be prohibited from entering the room with the door locked.

  Further, for example, the face matching result processing unit 50 may be an alarm device having a user output device such as a display, a lamp, an alarm, or a speaker. Then, when the face matching result processing unit 50 receives a matching result indicating that the person in the input image is not one of the persons permitted to enter the room from the face matching device 40, the face matching result processing unit 50 outputs the matching result to the user output device in an appropriate format. Thus, the staff or security staff may be notified that a person who is not a person who is permitted to enter the room has attempted to enter the room.

  Next, the flow of processing of the walk-through type entrance / exit management system 1 of the present embodiment will be described with reference to FIGS. 9 and 10.

  FIG. 9 is a flowchart illustrating an example of registration processing of the registered face data 422 by the face data registration apparatus 20 according to the present embodiment. The registration process of the registered face data 422 of the face data registration device 20 in the walk-through type entrance / exit management system 1 of the present embodiment will be described with reference to FIG. When receiving the registration image from the imaging unit 10 via the image input interface unit 210, the control unit 220 of the face data registration device 20 starts the following processing.

  The control unit 220 passes the registration image received from the imaging unit 10 to the registration face image extraction unit 2210. Then, the registration face image extraction unit 2210 extracts a face area from the received registration image and generates a registration face image (step S110).

  The 3D face model generation means 2220 maps the registration face image data extracted by the registration face image extraction means 2210 to 3D face shape data, and generates a 3D face model of the registered person (step S120). .

  The registered face data generation means 2230 changes the face of the 3D face model generated by the 3D face model generation means 2220 in various directions and reproduces the registered face images generated in various directions. Is stored in the storage unit 230 (step S130).

  The control unit 220 repeats the processing of steps S110 to S140 for the registration images of all registered persons, and then transmits all the registered face data 422 stored in the storage unit 230 via the registered face data output interface unit 240. It can be sent to the face matching device 40.

  As described above, when the registration image of the registered person photographed from the imaging unit 10 is sent, the control unit 220 may pass the sent registration image to the registration face image extraction unit 2210 at any time. However, these registration images may be temporarily stored as recording data in a storage medium such as a video tape or a hard disk. The control unit 220 reads the recorded data stored in the storage unit medium by the access device of the storage unit 230, cuts out the read recorded data frame by frame, and passes it to the registration face image extraction unit 2210 as a registration image. Also good.

  FIG. 10 is a flowchart illustrating an example of face matching processing by the face matching device 40 according to the present embodiment. The face matching process of the face matching device 40 in the walk-through type entrance / exit management system 1 of the present embodiment will be described with reference to FIG. Assume that the registered face data 422 of the registered person is generated by the face data registration device 20 as described with reference to FIG. 9 and stored in the storage unit 420 of the face matching device 40. When the control unit 440 of the face matching device 40 sequentially receives input images from the imaging unit 30 via the image input interface unit 410, the control unit 440 executes the following processing.

  The control unit 440 passes the input image received from the imaging unit 30 to the input face image extraction unit 4411. Then, the input face image extraction unit 4411 extracts a human face area from the received input image in the same manner as the registration face image extraction unit 2210 described above, and stores it in the storage unit 420 as an input face image (step S210).

  In step S210, when a plurality of persons are photographed in the input image, the input face image extraction unit 4411 extracts a plurality of face regions from the input image. In this case, the control unit 440 repeats the processes of steps S210 to S270 for each of a plurality of input face images included in the input image.

  The similarity calculation unit 4412 calculates the similarity between the input face image extracted by the input face image extraction unit 4411 and registered face images of various face orientations in the registered face data 422 of each registered person stored in the storage unit 420. Calculate each. Then, the similarity calculation unit 4412 stores the calculated similarity in the similarity table in association with the face orientation identification information of each registered face image for each registered person (step S220). For example, when there are 100 registered persons and 25 registered face images are registered per person as shown in FIG. 2, the similarity calculating unit 4412 calculates 25 × 100 = 2500 sheets. The similarity between the registered face image and the input face image may be obtained and stored in the similarity table.

  The voting table update unit 4413 removes the old voting value added to the voting total value of the voting total table for the input images acquired in the past from the voting total table (step S230).

  The voting unit 4414 selects a registered face image similar to the input face image with reference to the similarity stored in the similarity table by the similarity calculating unit 4412, and votes for the person and face orientation of the selected registered face image. Vote in the tabulation table (step S240). When a plurality of input face images are extracted from the input image in step S210 and image tracking is performed for each of the extracted input face images, the voting unit 4414 performs the above voting for each input face image to be tracked. You may vote in the total table.

  The person determining unit 4415 refers to the vote totaling table voted by the voting unit 4414 and selects the person having the maximum vote total value as the person of the input face image (step S250).

  The person determination unit 4415 determines whether or not the maximum vote total value selected in step S250 is equal to or greater than a verification threshold value (step S260).

  When the maximum vote total value is less than the verification threshold value (No in Step S260), the control unit 440 returns to Step S210 and receives an input image from the imaging unit 30. Then, the control unit 440 passes the received input image to the input face image extraction unit 4411 and repeats the processes of steps S210 to S270.

  If the maximum vote count value is equal to or greater than the verification threshold (Yes in step S260), the person determination unit 4415 determines that the person corresponding to the selected maximum vote count value is the person of the input face image as the input image. And output as a collation result (step S270). The control unit 440 collates with the person determination unit 4415 that the person of the input face image has been determined to be one of the registered persons and / or personal information regarding the registered person who has been determined to be the person of the input face image. As a result, the collation result is sent to the face collation result processing unit 50 by the face collation result output interface unit 450, and the process ends.

  According to the face image matching device according to the present invention, since the similarity between the input face image and the registered face image having a different face direction in the person different from the input face image is very high, the person in the input face image is erroneously determined as another person. Since it is possible to prevent others from being accepted or rejected, it is possible to improve the accuracy of face matching.

  The face image collation apparatus according to the present invention is not only a walk-through type entrance / exit management system, but also an entrance / exit management for performing face authentication for a user to temporarily stop in front of the entrance of the room and obtain permission to enter the room. The present invention can be applied to a face matching device of the system. In this face collation device, for example, the above-described walk-through type image is obtained by using, as a plurality of input images, images of users facing various faces taken from various angles by a plurality of cameras installed in front of the entrance of the room. Face matching processing similar to that performed by the face matching device of the entrance / exit management system can be performed.

  In addition, the present invention can be applied to a face matching device that performs face matching for detecting a suspicious person from a monitoring image of a monitoring camera in an in-store monitoring system. In the face matching device of this in-store monitoring system, a face image of a person who needs to be monitored who has had a previous history of cheating is registered in advance as registered face data, and an input face image extracted from the monitoring image and the face image are stored in advance. Check the registered face image registered. In this case, the face matching device can perform face matching in the same manner as the above-described face matching device, using a plurality of monitoring images taken in time series from the monitoring camera as input images. Moreover, this face collation apparatus can use a plurality of monitoring images taken by a plurality of monitoring cameras installed at a plurality of locations in the store as input images. When the face collation device determines that the person in the input image is one of the monitoring target persons in which the registered face data is registered, the face collation result processing device such as an alarm device informs that fact. Notify the monitor.

  In addition, for example, in the above-mentioned in-store monitoring system, the input image sent from the imaging unit may be temporarily stored as recording data in a storage medium such as a video tape or a hard disk. In this case, the control unit of the face collation device can read the recorded data stored in the storage medium by the access device of the storage unit and cut out the read recorded data frame by frame to make an input image. Then, as described above, the face collation device collates whether or not the person in the input image is a registered person in the registered face data, so that a suspicious person such as a monitoring target person is reflected in the stored recording data. Can analyze.

  Furthermore, in the embodiment described above, the face data registration device 20 and the face collation device 40 are separate devices, but the face data registration device 20 and the face collation device 40 according to the present invention may be the same device. In this case, the imaging unit 10, the image input interface unit 210, the control unit 220, and / or the storage unit 230 for the face data registration device are respectively the imaging unit 30, the image input interface unit 410, and the control unit for the face matching device 40. 440 and / or storage unit 420 may be shared. Further, the registered face data output interface unit 240 and the registered face data input interface unit 430 are unnecessary. For example, in one embodiment of the in-store monitoring system, when the control unit of the face data registration device determines that a suspicious person is included in the monitoring image received from an imaging unit such as a monitoring camera in the store, the monitoring image The registered face data may be registered using the image for registration.

  The present invention can be applied to various face matching devices in an authentication system, a monitoring system, and the like.

DESCRIPTION OF SYMBOLS 1 Walk through type entrance / exit management system 10 Image pick-up part 20 Face data registration apparatus 30 Image pick-up part 40 Face collation apparatus 50 Face collation result process part 210 Image input interface part 220 Control part 230 Storage part 240 Registered face data output interface part 410 Image input Interface unit 420 Storage unit 430 Registered face data input interface unit 440 Control unit 450 Face matching result output interface unit 2210 Registration face image extraction unit 2220 Three-dimensional face model generation unit 2230 Registration face data generation unit 4411 Input face image extraction unit 4412 Similar Degree calculation means 4413 Voting table update means 4414 Voting means 4415 Person determining means

Claims (4)

A face image collation device for collating a plurality of input face images acquired sequentially with a registered face image registered in advance,
For each of a plurality of registered persons, a storage unit that stores a plurality of face images having different face orientations of the registered persons as the registered face images;
Similarity calculation means for calculating the similarity with the registered face image for each input face image;
Vote for each of the registered face images having a similarity equal to or higher than a predetermined value, and add up vote values for each of the registered face images for a predetermined number of temporally continuous input face images. Voting means to seek,
Person determining means for determining any of the plurality of registered persons as a person of the input face image based on the vote count value;
A face image collating apparatus characterized by comprising:   The face image matching device according to claim 1, wherein the person determining unit determines a registered person having a maximum sum of the vote total values for each registered person as a person of the input face image.   3. The face image matching device according to claim 1, wherein the person determining unit determines a registered person having the largest vote total value as a person of the input face image. 4.   The person determination means is a person of the input face image for a registered person whose voting count value exceeds a predetermined matching threshold based on a predetermined number of temporally continuous input face images counted by the voting means. The face image matching device according to claim 2, wherein the face image matching device is determined as follows.

Images