JP2010086403A - Facial recognition device, facial recognition method, and passage controller - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a facial recognition device, a facial recognition method and a passage controller, capable of recognizing a person with high accuracy, in a short time. <P>SOLUTION: This facial recognition device 100 is provided with a registration information storage means 108 for preliminarily storing registration information, and an image input means 102A for consecutively enterinh images, including at least the face of the person. The facial recognition device detects a facial area of the person from among the consecutive images entered from the image input means, in each image. The facial recognition device determines the presence of a recognition intention of the person, based on the position and the size of the face area detected in each image. The facial recognition device selects either of the first collation system for conducting collation, based on the plurality of detected facial area images and the registration information stored by the registration information storage means, or the second collation system for conducting collation based on at least one of the detected facial area images and the registration information stored by the registration information storage means, and determines whether the person is one who has registered in advance. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a face recognition device, a face recognition method, and a traffic control device for recognizing a face of a person to be photographed based on a face image of the person to be photographed acquired by a camera.

  In general, the face recognition apparatus has a camera such as a video camera. When a person stops in front of the camera and faces his face to the lens of the camera, the face recognition device captures a face image of the person (target person) with the camera. The camera inputs the captured image to the face recognition device.

  The face recognition device detects a face area from the input image. The face recognition device extracts feature information of a face area image (face image). The face recognition device stores in advance a plurality of feature information of a person's face as registration information. The face recognition device collates the feature information extracted from the face image with the registered information, and performs authentication (face authentication) as to whether there is matching feature information. Thereby, the face recognition device recognizes a person. For example, if the face recognition device used in the traffic control device determines that there is matching feature information, it determines that the target person is a person permitted to pass.

Usually, the face recognition device extracts feature information after photographing a plurality of human face images. For this reason, when taking a picture after a person stops in front of the camera, there is a problem that it takes time to complete the authentication. Therefore, a personal authentication system is provided in which a person (pedestrian) who is walking is imaged to acquire an image and perform authentication (see, for example, Patent Document 1).
JP 2005-275869 A

  The personal authentication system as described above needs to secure a space for photographing a figure of a person walking. For this reason, there is a problem that the place where the apparatus can be installed is limited.

  Further, in the personal authentication system as described above, it is difficult to stably obtain a pedestrian's face image due to variations in how the face looks (angle etc.), how to walk, or lighting conditions. For this reason, there is a problem that the certification system is lowered.

  An object of one embodiment of the present invention is to provide a face recognition device, a face recognition method, and a traffic control device that can recognize a person with higher accuracy in a short processing time.

  A face recognition apparatus according to an embodiment of the present invention includes a registration information storage unit that stores registration information in advance, an image input unit that continuously inputs an image including at least a face of a person, and an input from the image input unit. Face detection means for detecting the face area of the person for each image from the consecutive images obtained, and determining the presence or absence of the person's authentication intention based on the position and size of the face area detected for each image by the face detection means An action determination unit that performs the matching by a first matching method based on the image of the face area detected by the face detection unit and the registration information stored by the registration information storage unit, A second collation unit that performs collation by a second collation method based on the image of the face area detected by the face detection unit and the registration information stored by the registration information storage unit; A selecting unit that selects one of a matching unit and the second collating unit, and a recognizing unit that determines whether the person is a pre-registered person by the collating unit selected by the selecting unit; It comprises.

  In addition, a face recognition method according to an embodiment of the present invention includes a registration information storage unit that stores registration information in advance and an image input unit that continuously inputs an image including at least a face of a person. A face recognition method used in a recognition device, wherein the face area of the person is detected for each image from successive images input from the image input means, and based on the position and size of the face area detected for each image A first verification method for determining whether the person has an authentication intention, and performing verification based on the detected images of the plurality of face regions and the registration information stored in the registration information storage unit, Based on the detected at least one face area image and the registered information stored in the registered information storage means, one of the second matching methods is selected, and the selected matching method is selected. Yo , It is determined whether the person the person is registered in advance.

  A traffic control device according to an embodiment of the present invention is a traffic control device that controls opening and closing of a door provided in an area where traffic is restricted, and stores in advance registration information of a person who is permitted to pass. Registration information storage means; image input means for continuously inputting an image including at least a face of a person; and face detection for detecting the face area of the person for each image from the continuous images input from the image input means. Means for determining the presence or absence of the person's authentication intention based on the position and size of the face area detected for each image by the face detection means, the image of the face area detected by the face detection means, A first collation unit that performs collation by a first collation method based on registration information stored in the registration information storage unit; an image of a face area detected by the face detection unit; Selection means for selecting one of the second collating means for performing collation by the second collation method based on the registration information stored by the means, and the first collating means and the second collating means. And a recognizing unit that determines whether or not the person is a pre-registered person by the matching unit selected by the selecting unit, and a case where the recognizing unit determines that the person is a pre-registered person Gate control means for controlling to open a door provided in an area where traffic is restricted.

  According to one embodiment of the present invention, it is possible to provide a face recognition device, a face recognition method, and a traffic control device that can recognize a person with higher accuracy in a short processing time.

Hereinafter, a face recognition device, a face recognition method, and a traffic control device according to a first embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram for schematically explaining a configuration example of a face recognition apparatus 100 according to the first embodiment of the present invention. It is assumed that the face recognition device 100 is incorporated in, for example, a traffic control device that restricts persons permitted to pass. The face recognition device 100 is assumed to be installed in a place where only a specific person is allowed to pass, for example, an entrance such as a building and a corporate building, or a gate such as an amusement facility and a transportation facility.

  Note that the face recognition device 100 collates feature information obtained from the acquired face image with feature information registered in advance as registration information, and determines whether or not there is at least one person with matching feature information. Assuming that the configuration is

  As shown in FIG. 1, the face recognition device 100 includes an input unit 101 and a control unit 102. The input unit 101 is a unit for acquiring information from the outside. The input unit 101 includes a camera 103 and an operation unit 104.

  The camera 103 is an imaging unit. The camera 103 is composed of, for example, an industrial television (ITV) camera. The camera 103 captures a predetermined range of moving images (a plurality of continuous images). Thereby, the camera 103 captures an image including the face of the pedestrian. The camera 103 digitally converts the captured image using an A / D converter (not shown) and sequentially transmits it to the control unit 102. Note that a configuration may be employed in which a photograph is scanned by a scanner instead of a camera, and registration information is added, collated, and searched.

  The operation unit 104 is an operation means for causing an operator (including a pedestrian) to input an operation. The operation unit 104 includes, for example, an input key or a card reader. The operation unit 104 transmits, to the control unit 102, input information input using an input key or information read from a card (for example, an IC card) held by a pedestrian using a card reader.

  As a result, the face recognition apparatus 100 performs either 1: N verification that performs person authentication only by face authentication, or 1: 1 and 1: n verification that performs person authentication by face authentication and identification information verification. The person can be authenticated. 1: N verification is a method of performing authentication to determine whether or not a face that matches registered information exists among a plurality of people (N people).

  The 1: 1 and 1: n verification is a method for authenticating one person or a small number of persons (n persons) smaller than N persons. When 1: 1 and 1: n verification is used, the face recognition apparatus 100 performs face authentication and authentication using identification information input by an input key or a card reader.

  Furthermore, the input unit 101 may include a display unit that sequentially displays the status of face authentication and the like for pedestrians. Further, the operation unit 104 and the display unit may be configured integrally. In this case, the operation unit and the display unit function as a touch panel.

  The control unit 102 is a unit that performs various processes based on information input by the input unit 101. The control unit 102 includes a face detection unit 105, a motion determination unit 106, a recognition unit 107, a registration information storage unit 108, an output unit 109, and a gate control unit 110. Further, the control unit 102 includes an interface 102A and an interface 102B.

  The interface 102A and the interface 102B receive signals from the outside of the control unit 102. The interface 102A is connected to the camera 103. Images including at least the face of a person are continuously input from the camera 103 to the interface 102A. That is, the interface 102A functions as an image input unit.

  The interface 102B is connected to the operation unit 104. An operation signal from the operator is input from the operation unit 104 to the interface 102B. That is, the interface 102B functions as an operation input unit.

  The face detection unit 105 functions as a face detection unit. The face detection unit 105 detects an area (face area) in which a human face is captured in an input image captured by the camera 103. That is, the face detection unit 105 detects the face image (face image) and position of a person (pedestrian) moving within the shooting range of the camera 103 based on the input image.

  For example, the face detection unit 105 detects a face area by obtaining a correlation value while moving a template prepared in advance in an input image. Here, the face detection unit 105 detects the position where the highest correlation value is calculated as a face area.

  There are various methods for detecting the face area. The face recognition apparatus 100 according to the present embodiment can also be realized by using another method for detecting a face area such as an eigenspace method or a subspace method.

  The face recognition apparatus 100 can also detect the position of a facial part such as eyes, nose, and mouth from the detected face area. Specifically, for example, document [1] (Kazuhiro Fukui, Osamu Yamaguchi: “Face feature point extraction by combination of shape extraction and pattern matching”, IEICE Transactions (D), vol. J80-D-II. , No. 8, pp2170-2177 (1997), literature [2] (Mayumi Yuasa, Akiko Nakajima: “Digital Make System Based on High-Precision Facial Feature Point Detection” Proceedings of the 10th Image Sensing Symposium, pp219-224 ( 2004) and the like.

  In this embodiment, a configuration in which authentication is performed using a face image will be described as an example. However, the configuration for realizing the present invention is not limited to this. For example, the configuration may be such that authentication is performed using images of the iris, retina, and eyes. In this case, the face recognition apparatus 100 can detect an eye region in the image, zoom the camera, and enlarge and acquire the eye image.

  In any case, the face recognition apparatus 100 acquires information that can be handled as an image in which a plurality of pixels are two-dimensionally arranged.

  When one face is extracted from one input image, the face recognition apparatus 100 obtains a correlation value with the template for the entire image, and detects the maximum position and size as a face region.

  When a plurality of faces are extracted from one input image, the face recognition apparatus 100 obtains a local maximum value of correlation values for the entire image and narrows down candidate face positions in consideration of overlap in one image. . Furthermore, the face recognition apparatus 100 simultaneously detects a plurality of face regions in consideration of the relationship (temporal transition) with past images that have been continuously input.

  In the present embodiment, the face recognition apparatus 100 is described as detecting a human face area as an example, but the present invention is not limited to this. For example, the face recognition apparatus 100 can also detect a person area. For example, using the technology disclosed in Document [3] (Nobuhito Matsugura, Hideki Ogawa, Taku Yoshimi: “Life Support Robot Coexisting with People” Toshiba Review Vol. 60 No. 7, pp 112-115 (2005). By doing so, the face recognition apparatus 100 can detect a person area.

  Note that the camera 103 continuously acquires images and transmits the images to the control unit 102 frame by frame. For this reason, the face detection unit 105 sequentially detects the face area each time an image is input.

  The operation determination unit 106 functions as an operation determination unit. The motion determination unit 106 determines the motion of the person based on the face area image (face image) detected by the face detection unit 105. The motion determination unit 106 determines whether or not the person is trying to authenticate from the position and size of the face area, that is, whether or not there is an authentication intention (verification intention).

  The operation determination unit 106 manages the coordinates of the face area detected by the face detection unit 105. When the face detection unit 105 detects a face area, the motion determination unit 106 compares the coordinates and size at which the face area was detected with the previous frame. Based on the comparison result, the motion determination unit 106 determines whether the face areas detected in the two frames are of the same person.

  If it is determined that they are the same person, the action determination unit 106 associates the two face images as being the same person. That is, the operation determination unit 106 tracks the same person that is reflected in images that are continuously input.

  In this case, the motion determination unit 106 matches the pattern of the face area image with the previous frame in order not to associate different persons as the same person. If there is a difference equal to or greater than a predetermined threshold as a result of matching, the motion determination unit 106 may be configured to determine that the person in the face area detected in the previous frame and the current frame is a different person. Good.

  As described above, by tracking the face area of the same person, the movement of the pedestrian can be determined. For example, when the size of the face area increases, the face recognition apparatus 100 can determine that a pedestrian is approaching the present apparatus. In addition, the amount of information used for face authentication can be increased by associating the pedestrian's face areas reflected in a plurality of frames with the same person. Thereby, face authentication can be performed more accurately.

  For example, as shown in FIG. 2, when the size of the face area in the captured input image 10 and the size of the predetermined frame 11 become approximately the same, the motion determination unit 106 tries to authenticate the pedestrian. It can be determined that there is an intention to collate.

  The size of the frame 11 is set assuming the size of the face area when a person stands in front of the camera 103 of the present apparatus. That is, when the size of the face area in the captured input image 10 and the size of the predetermined frame 11 become approximately the same, the motion determination unit 106 approaches the device and shows the camera 103 a face. It is determined that it is in a state. If the change in the coordinates of the face area from the previous frame to the current frame is less than a predetermined threshold, the action determination unit 106 determines that the person has stopped. That is, when it is determined that the person has stopped, the operation determination unit 106 determines that there is a verification intention.

  In addition, when an operation is input from the operation unit 104, the operation determination unit 106 determines that the operator (pedestrian) has a intention to collate. The operation determination unit 106 controls the timing of outputting the collation result so that the collation result is output when it is determined that there is an intention to collate.

  The recognition unit 107 extracts feature information (face feature) based on the face image detected by the face detection unit 105. The registration information storage unit 108 stores human face feature information as registration information. The registration information storage unit 108 functions as registration information storage means. That is, the registration information storage unit 108 stores in advance feature information extracted from a face image of a person to be recognized in this apparatus.

  The recognition unit 107 compares the extracted feature information with the feature information stored in the registered information storage unit 108. That is, the recognition unit 107 determines whether or not feature information that matches or approximates feature information extracted from the face image is stored in the registered information storage unit 108. In this case, the recognition unit 107 functions as a recognition unit.

  The recognition unit 107 includes a plurality of verification methods. The recognizing unit 107 appropriately selects a collation method based on the number of face images that can be used for authentication. That is, the recognizing unit 107 appropriately selects a collation method based on the number of face images acquired until the operation determination unit 106 confirms the collation intention.

  For example, the recognition unit 107 includes a first matching method and a second matching method. When the first matching method is selected, the recognition unit 107 integrates a plurality of face images, extracts one feature information, and performs matching based on the extracted feature information. That is, in this case, the recognition unit 107 functions as a first matching unit.

  When the second matching method is selected, the recognition unit 107 extracts feature information for each of a plurality of face images, and performs matching for each extracted feature information. That is, in this case, the recognition unit 107 functions as a second matching unit.

  The first matching method is more accurate than the second matching method because the feature information is extracted by integrating a plurality of images. However, in order to perform the first collation method, a predetermined number of face images are required.

  The predetermined number of face images necessary for performing the first collation method can be arbitrarily set. However, the accuracy of collation by the recognition unit 107 increases in proportion to the number of face images. For example, when there are few face images used for extracting feature information, there is a possibility that it is not possible to cope with fluctuations in the movement of a person's face. For this reason, the required number of face images is limited according to the required accuracy of collation.

  Therefore, the recognition unit 107 of the face recognition apparatus 100 according to the present embodiment uses the first verification method and gives priority to the accuracy of verification when there is a margin in processing time. Further, when the processing time has no allowance, the recognizing unit 107 uses the second verification method and gives priority to the verification processing time.

  The recognition unit 107 functions as a selection unit that selects a collation method. That is, the recognizing unit 107 selects the first verification method when a predetermined number or more of face images have been acquired before confirming the authentication intention. The recognizing unit 107 selects the second matching method when a predetermined number or more of face images cannot be acquired before confirming the authentication intention.

  That is, the face recognition apparatus 100 can select a matching method as appropriate according to the movement of the pedestrian. For this reason, the face recognition apparatus 100 can perform collation with high accuracy in a short processing time.

  The first collation method and the second collation method will be described in detail with reference to FIGS.

  FIG. 3 is an explanatory diagram for explaining the first collation method.

  The above-mentioned first collation method is described in, for example, Document [4] (Kazuhiro Fukui, Osamu Yamaguchi, Kenichi Maeda: “Face Recognition System Using Moving Images” IEICE Research Report PRMU, vol 97, No. 113, pp17. -24 (1997), literature [5] (Kenichi Maeda, Sadaichi Watanabe: “Pattern matching method introducing local structure”, IEICE Transactions (D), vol. J68-D, No. 3, pp345. It can be realized by using the mutual subspace method disclosed in 352 (1985).

  The face detection unit 105 detects the position of the face component, and cuts out an image (face image) of the face area based on the detected position of the face component. That is, the face detection unit 105 cuts out a face area (an image of an area of m pixels × n pixels) from the input image. The recognizing unit 107 extracts shading information of the image of the face area cut out by the face detecting unit 105 as feature information.

  Here, the shade value of the image of the area of m pixels × n pixels is used as the shade information as it is. That is, dimensional information of m × n pixels is used as a feature vector. The recognition unit 107 calculates the similarity of a plurality of images by the simple similarity method. That is, the recognition unit 107 performs normalization by the simple similarity method so that the vector and the length of the vector are set to “1”, respectively. The recognizing unit 107 calculates a similarity indicating a similarity between a plurality of feature vectors by calculating an inner product. When there is one image acquired by the camera 103, the feature of the image can be extracted by the above processing.

  Further, by using a moving image composed of a plurality of continuous images in order to output a recognition result, the face recognition device 100 can perform a recognition process with higher accuracy. For this reason, in the present embodiment, description will be given by taking recognition processing using a moving image as an example.

  When performing recognition processing using a moving image, the camera 103 continuously captures an imaging region. The face detection unit 105 cuts out an image of a face area (an image of m × n pixels) from a plurality of continuous images captured by the camera 103. The recognizing unit 107 acquires the feature vector of the extracted plurality of face area images for each image. The recognition unit 107 obtains a correlation matrix from the acquired feature vector for each image.

  The recognition unit 107 obtains an orthonormal vector from the correlation matrix of the feature vector, for example, by Karhunen-Loeve expansion (KL expansion). As a result, the recognition unit 107 can calculate and specify a partial space indicating facial features in a continuous image.

  When calculating the subspace, the recognition unit 107 obtains a correlation matrix (or covariance matrix) of feature vectors. The recognition unit 107 obtains an orthonormal vector (eigenvector) by performing KL expansion on the correlation matrix of the feature vector. Thereby, the recognition unit 107 calculates the partial space.

  The recognition unit 107 selects k eigenvectors corresponding to eigenvalues in descending order of eigenvalues. The recognizing unit 107 represents a subspace using a set of k selected eigenvectors.

  In the present embodiment, the recognition unit 107 obtains a correlation matrix “Cd = ΦdΔdΦdT” based on the feature vector. The recognition unit 107 diagonalizes the correlation matrix “Cd = ΦdΔdΦdT” to obtain a matrix Φd of eigenvectors. This information, that is, the matrix Φd is a partial space indicating the characteristics of the face of the person to be recognized.

  The registration information storage unit 108 stores the partial space calculated by the above method as registration information. Note that the feature information stored in the registered information storage unit 108 is, for example, a feature vector of m × n pixels. However, the feature information stored in the registration information storage unit 108 may be a face image in a state before the feature is extracted. The feature information stored in the registration information storage unit 108 may be information indicating a partial space or a correlation matrix in a state before KL expansion is performed.

  The recognizing unit 107 calculates the degree of similarity between the calculated partial space and one or more pieces of feature information (information indicating the partial space here) stored in the registered information storage unit 108. Thereby, the recognition unit 107 determines whether or not a person registered in advance is included in the image captured by the camera 103.

  There are various methods for calculating the similarity between a plurality of subspaces. In the above-described method, the recognition data in the registration information stored in advance and the input data are expressed as a partial space. That is, the mutual subspace method specifies facial feature information stored in advance in the registered information storage unit 108 and feature information created based on an image photographed by the camera 103 as a partial space. In this method, an “angle” formed by the two partial spaces is calculated as the similarity.

  Here, the partial space calculated based on the input image will be described as an input partial space. The recognizing unit 107 obtains a correlation matrix “Cin = ΦinΔinΦinT” based on the input data string (image captured by the camera 103).

  The recognition unit 107 performs diagonalization with the correlation matrix “Cin = ΦinΔinΦinT” to obtain an eigenvector Φin. The recognition unit 107 calculates a similarity between the partial space specified by Φin and the partial space specified by Φd. That is, the recognition unit 107 obtains a similarity (0.0 to 1.0) between two partial spaces.

  When there are a plurality of face areas in the input image, the recognition unit 107 performs recognition processing on each face area in order. That is, the recognizing unit 107 calculates the degree of similarity based on the feature information (registered information) stored in the registered information storage unit 108 and the image of the face area. Thereby, the recognition unit 107 can obtain the result of the recognition process for all persons in the input image. For example, when X persons walk toward the apparatus storing Y dictionary, the recognition unit 107 performs recognition processing, that is, similarity calculation X × Y times. Thereby, the recognition part 107 can output the result of the process of recognition with respect to all of X persons.

  When an image matching the registration information stored in the registration information storage unit 108 is not found in the plurality of input images, that is, when the recognition result by the recognition unit 107 is not output, the recognition unit 107 Performs the recognition process again based on the next image (the image of the next frame) taken by the camera 103.

  In this case, the recognition unit 107 adds the correlation matrix input to the partial space, that is, the correlation matrix related to one frame, to the sum of the correlation matrices related to a plurality of past frames. The recognition unit 107 calculates the eigenvector again. The recognition unit 107 creates a partial space again. As a result, the recognition unit 107 updates the partial space related to the input image.

  When the face images of the walking person are continuously photographed and collated, the recognition unit 107 sequentially updates the partial space. That is, the recognition unit 107 performs recognition processing every time an image is input. As a result, the accuracy of verification gradually increases in accordance with the number of captured images.

  4 and 5 are explanatory diagrams for explaining the second collation method.

  The second collation method described above is described in, for example, Document [6] (Erki Oya, Hidemitsu Ogawa, Makoto Sato, “Pattern Recognition and Subspace Method”, Sangyo Tosho, 1986), and Document [7] ( Toshiba (Tatsuo Kosakaya): It can be realized by using a method disclosed in “Image recognition apparatus, method and program” (Japanese Patent Laid-Open No. 2007-4767).

  Document 6 describes a method of recognizing a person by projecting onto a partial space as registration information created in advance from a plurality of face images by the partial space method. As shown in FIG. 4, when using the method described in Document 6, the recognition unit 107 can perform collation using one image.

  Document 7 describes a method of creating an image (perturbation image) in which, for example, the orientation and state of a face are intentionally changed using a model for one image. In this case, as shown in FIG. 5, collation can be performed by the first collation method using a plurality of perturbation images having different face orientations and states.

  When performing the perturbation, the recognition unit 107 generates a perturbation image by perturbing the coordinates of each part of the face detected by the face detection unit 105 in an arbitrary direction. The amount to be perturbed may be in any range. In addition, the type of the face part (facial feature point) to be perturbed may be any part. In addition, the types of face portions (facial feature points) to be perturbed may be a plurality of types.

  The registration information storage unit 108 stores personal face characteristic information as registration information using personal identification information as a key. That is, the registration information storage unit 108 stores identification information and face feature information in association with each other. Note that the registration information storage unit 108 may store one piece of identification information and a plurality of face feature information in association with each other. When recognizing a person based on a photographed image, the face recognition apparatus 100 may use a plurality of pieces of face feature information for recognition. Further, the registered information storage unit 108 may be provided outside the face recognition device 100.

  The output unit 109 outputs, for example, an input image from the input unit 101, an image of the face area detected by the face detection unit 105, or feature information. Further, the output unit 109 outputs the recognition result obtained by the recognition unit 107 to an external device. For example, the output unit 109 outputs the recognition result to the gate control unit 110.

  The gate control unit 110 functions as a gate control unit. The gate control unit 110 controls the door 111 installed in the vicinity of the face recognition apparatus 100 shown in FIG. For example, in the case where the recognition result is OK, when the passable signal output from the output unit 109 is received, the gate control unit 110 opens the door 111. Thereby, the pedestrian can pass through the door 111. In addition, the gate control part 110 closes the door 111, when a passable signal is no longer detected.

  When the input unit 101 or the control unit 102 includes a display unit, the output unit 109 outputs a recognition result obtained by the recognition unit 107 to the display unit. When the display unit receives the recognition result, the display unit displays the received recognition result. Thereby, the face recognition apparatus 100 can alert | report a recognition result with respect to a pedestrian. In response to this, a control signal (such as a gate control signal to a gate device) is output to an external device. Further, the display unit may be provided outside, for example, as a monitoring monitor.

  For example, the output unit 109 creates and outputs a display screen so as to display the feature information stored in the registration information storage unit 108 that is close to the input face image. Further, the output unit 109 outputs a signal to an external device so as to sound an alarm when the similarity is equal to or higher than a predetermined threshold as a result of the collation. Furthermore, the output unit 109 outputs a passable signal to the gate control unit 110 when the similarity is greater than or equal to a predetermined threshold value as a result of the collation.

  The control unit 102 includes, for example, a configuration such as a CPU, RAM, ROM, and nonvolatile memory. The CPU controls the control performed by the control unit 102. The CPU functions as various processing means by operating based on the control program and control data stored in the ROM or nonvolatile memory. That is, the processing performed by the face detection unit 105, the operation determination unit 106, and the recognition unit 107 is realized by the CPU executing a program.

  The RAM is a volatile memory that functions as a working memory for the CPU. That is, the RAM functions as a storage unit that temporarily stores data being processed by the CPU. The RAM temporarily stores data received from the input unit. The ROM is a non-volatile memory in which a control program, control data, and the like are stored in advance.

  The nonvolatile memory is configured by a storage medium that can write and rewrite data, such as an EEPROM or an HDD. In the non-volatile memory, a control program and various data are written according to the operation application of the face recognition apparatus 100.

  FIG. 6 is a flowchart for explaining processing in the face recognition apparatus 100 shown in FIG.

The camera 103 captures and acquires an image of the shooting area (step S11). The camera 103 inputs the acquired image to the face detection unit 105 of the control unit 102. The face detection unit 105 detects an area (face area) where a person's face is captured in an image (input image) input from the camera 103 (step S12).
When the face detection unit 105 detects a face area, the motion determination unit 106 compares the coordinates and size of the face area detected with the previous frame (step S13).

  Based on the comparison result, the motion determination unit 106 determines whether or not the face areas detected in the two frames are of the same person. For this purpose, the motion determination unit 106 determines whether or not the difference between the coordinates and size at which the face area is detected in the previous frame is within a predetermined threshold (step S14).

  When it is determined that the difference is within the threshold (YES in step S14), the motion determination unit 106 determines that the persons in the face area detected in the previous frame and the current frame are the same person (step S15). If it is determined that the difference exceeds the threshold (NO in step S14), the motion determination unit 106 determines that the person in the face area detected in the previous frame and the current frame is different (step S14). S16).

  The motion determination unit 106 determines whether the face area of the person is at a predetermined position (step S17). That is, as shown in FIG. 2, the motion determination unit 106 determines whether the size of the face area in the input image 10 captured by the camera 103 and the size of the predetermined frame 11 have become comparable.

  When it is determined that the face area of the person is not at the predetermined position (step S17, NO), the motion determination unit 106 determines whether identification information is input by the operation unit 104 (step S18). When the input of identification information is not confirmed (step S18, NO), the operation | movement determination part 106 determines with a collation intention not being confirmed (collation intention unconfirmed) (step S19). If the face recognition apparatus 100 determines that the verification intention has not been confirmed, the face recognition apparatus 100 proceeds to step S11 and acquires an image of the next frame.

  When it is determined in step S17 that the face area of the person is present at the predetermined position (step S17, YES), or when input of identification information is confirmed in step S18 (step S18, YES), the operation determination unit 106 It is determined that there is an intention to collate (step S20).

  The recognizing unit 107 determines whether or not a predetermined number of face images of the person to be authenticated have been photographed before confirming the intention of collation by the operation determining unit 106 (step S21). If the recognition unit 107 determines that a predetermined number or more of face images have been acquired before confirming the verification intention (step S21, YES), the recognition unit 107 selects the first verification method (step S22). If the recognition unit 107 determines that a predetermined number or more of face images have not been acquired before confirming the verification intention (step S21, NO), the recognition unit 107 selects the second verification method (step S23).

  The recognition unit 107 authenticates the person using the selected collation method (step S24). That is, the recognition unit 107 extracts feature information based on the face image detected by the face detection unit 105. The recognition unit 107 compares the extracted feature information with the feature information stored in the registered information storage unit 108. That is, the recognition unit 107 determines whether or not feature information that matches or approximates feature information extracted from the face image is stored in the registered information storage unit 108.

  As a result of authentication by the recognizing unit 107, when it is determined as OK (step S24, YES), the output unit 109 outputs a passable signal to the external device or the gate control unit 110 (step S25).

  Further, when it is determined as NG as a result of authentication by the recognition unit 107 (NO in step S24), the output unit 109 outputs information indicating that the verification has failed to an external device or a display unit (step S26). Thereby, the face recognition apparatus 100 notifies a pedestrian that the collation is NG.

  As described above, the face recognition device according to this embodiment includes a plurality of collation methods. The face recognition device continuously captures images and sequentially detects a human face area. The face recognition device determines the action of a person based on the position and coordinates of a face area that is continuously detected. The face recognition device selects one of the above-described stitching methods based on the determined action of the person and performs authentication. Thereby, the face recognition apparatus can select the optimal collation method according to the movement of the person.

  As a result, it is possible to provide a face recognition device, a face recognition method, and a traffic control device that can recognize a person with higher accuracy in a short processing time.

  In the present embodiment, the configuration has been described in which one of the first collation method and the second collation method is selected, but the present invention is not limited to this. For example, a configuration in which switching is performed step by step based on the number of shots, processing time, and the like from among three or more verification methods may be used.

  In the present embodiment, the face recognition apparatus 100 is described as including the input unit 101, but the present invention is not limited to this. The input unit 101 may be installed outside the face recognition device 100. That is, any configuration may be used as long as it acquires a human face image and an operation signal based on an operation by the person.

  The camera 103 is an imaging unit. The camera 103 is composed of, for example, an industrial television (ITV) camera. The camera 103 captures a predetermined range of moving images (a plurality of continuous images). Thereby, the camera 103 captures an image including the face of the pedestrian. The camera 103 digitally converts the captured image using an A / D converter (not shown) and sequentially transmits it to the control unit 102. Note that a configuration may be employed in which a photograph is scanned by a scanner instead of a camera, and registration information is added, collated, and searched.

  The operation unit 104 is an operation means for causing an operator (including a pedestrian) to input an operation. The operation unit 104 includes, for example, an input key or a card reader. The operation unit 104 transmits, to the control unit 102, input information input using an input key or information read from a card (for example, an IC card) held by a pedestrian using a card reader.

  As a result, the face recognition apparatus 100 performs either 1: N verification that performs person authentication only by face authentication, or 1: 1 and 1: n verification that performs person authentication by face authentication and identification information verification. The person can be authenticated. 1: N verification is a method of performing authentication to determine whether or not a face that matches registered information exists among a plurality of people (N people).

  The 1: 1 and 1: n verification is a method for authenticating one person or a small number of persons (n persons) smaller than N persons. When 1: 1 and 1: n verification is used, the face recognition apparatus 100 performs face authentication and authentication using identification information input by an input key or a card reader.

  Furthermore, the input unit 101 may include a display unit that sequentially displays the status of face authentication and the like for pedestrians. Further, the operation unit 104 and the display unit may be configured integrally. In this case, the operation unit and the display unit function as a touch panel.

  The control unit 102 is a unit that performs various processes based on information input by the input unit 101. The control unit 102 includes a face detection unit 105, a motion determination unit 106, a recognition unit 107, a registration information storage unit 108, an output unit 109, and a gate control unit 110. Further, the control unit 102 includes an interface 102A and an interface 102B.

  The interface 102A and the interface 102B receive signals from the outside of the control unit 102. The interface 102A is connected to the camera 103. Images including at least the face of a person are continuously input from the camera 103 to the interface 102A. That is, the interface 102A functions as an image input unit.

  The interface 102B is connected to the operation unit 104. An operation signal from the operator is input from the operation unit 104 to the interface 102B. That is, the interface 102B functions as an operation input unit.

Next, a face recognition device, a face recognition method, and a traffic control device according to a second embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 7 is a block diagram for schematically explaining a configuration example of a face recognition apparatus 200 according to the second embodiment of the present invention. The face recognition device 200 includes an input unit 201, a control unit 202, an input interface 202A, an input interface 202B, a camera 203, an operation unit 204, a face detection unit 205, a motion determination unit 206, a recognition unit 207, a registration information storage unit 208, and an output. A unit 209, a gate control unit 210, and the like.

  The configurations of the input unit 201, the input interface 202A, the input interface 202B, the camera 203, the operation unit 204, the face detection unit 205, the motion determination unit 206, the registration information storage unit 208, the output unit 209, and the gate control unit 210 are as follows. The input unit 101, the input interface 102A, the input interface 102B, the camera 103, the operation unit 104, the face detection unit 105, the motion determination unit 106, the registration information storage unit 108, the output unit 109, and the gate control unit 110 in the first embodiment. Since the configuration is the same as that of FIG.

  The recognition unit 207 of the face recognition apparatus 200 according to the second embodiment performs sequential matching based on images continuously acquired by the camera 203.

  That is, the recognition unit 207 performs collation based on the image acquired by the camera 203 even when the collation intention is not confirmed by the motion determination unit 206. In this case, the recognition unit 207 performs collation using the second collation method.

  However, the second matching method is less accurate than the first matching method. For this reason, the recognition unit 207 performs collation by the second collation method after setting a threshold used when comparing the similarity to a value higher than normal (scrutiny threshold). The scrutinization threshold is a threshold that is higher than the threshold used for normal verification. That is, in order to determine that the verification is OK, a higher degree of similarity is required. The scrutiny threshold value is stored in advance in a ROM (not shown) or a non-volatile memory that the control unit 102 has.

  FIG. 8 is a flowchart for explaining processing in the face recognition apparatus 200 shown in FIG. Steps S31 to S46 are the same as steps S11 to S26 shown in FIG.

  When the motion determination unit 206 determines that the face area of the person is not at the predetermined position and the input of identification information by the operation unit 204 is not confirmed, the motion determination unit 206 does not confirm the verification intention (verification intention is not confirmed). (Step S46).

  When the face recognition apparatus 200 determines that the verification intention has not been confirmed, the recognition unit 207 performs a verification process based on the image of the current frame. That is, the recognizing unit 207 selects the second collation method as the collation method used for the collation processing (step S47). In this case, the recognizing unit 207 sets a higher threshold than usual so as not to cause erroneous processing.

  The recognizing unit 207 authenticates the person by the selected second collation method (step S48). That is, the recognition unit 207 extracts feature information based on the face image detected by the face detection unit 205. The recognition unit 207 collates the extracted feature information with the feature information stored in the registered information storage unit 208. That is, the recognition unit 207 determines whether or not feature information that matches or approximates the feature information extracted from the face image is stored in the registered information storage unit 208.

  As a result of the authentication by the recognition unit 207, when it is determined to be OK (step S48, YES), the output unit 209 outputs a passable signal to the external device or the gate control unit 210 (step S43).

  If the result of authentication by the recognition unit 207 is NG (step S48, NO), the face recognition device 200 moves to step S31 and acquires an image of the next frame.

  As described above, according to the face recognition apparatus 200 according to the second embodiment, when it is determined that the verification is successful before the motion determination unit 206 confirms the verification intention, the pedestrian does not need to stop in front of the camera 203. . For this reason, the face recognition apparatus 200 can output a collation result earlier than the collation performed after confirming the collation intention by the operation | movement determination part 206. FIG.

  Further, when the verification is not successful before the action determination unit 206 confirms the verification intention, the face recognition apparatus 200 resets the threshold value to a normal value and then performs face recognition according to the first embodiment. Collation can be performed by the same processing as that of the apparatus 100.

  As a result, it is possible to provide a face recognition device, a face recognition method, and a traffic control device that can recognize a person with higher accuracy in a short processing time.

Next, a face recognition device, a face recognition method, and a traffic control device according to a third embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 9 is a block diagram for schematically explaining a configuration example of a face recognition apparatus 300 according to the third embodiment of the present invention. The face recognition apparatus 300 includes an input unit 301, a control unit 302, an input interface 302A, an input interface 302B, a camera 303, an operation unit 304, a face detection unit 305, a motion determination unit 306, a recognition unit 307, a registration information storage unit 308, and an output. 309, a gate control unit 310, and the like.

  The configurations of the input unit 301, the input interface 302A, the input interface 302B, the camera 303, the operation unit 304, the face detection unit 305, the registration information storage unit 308, the output unit 309, and the gate control unit 310 are the same as in the first embodiment. The configuration is the same as the input unit 101, input interface 102A, input interface 102B, camera 103, operation unit 104, face detection unit 105, registration information storage unit 108, output unit 109, and gate control unit 110 in FIG. The detailed explanation is omitted.

  If the recognition unit 307 of the face recognition apparatus 300 according to the third embodiment determines that the verification is NG in the verification process, the verification unit 307 performs a verification process retry (second verification process).

  That is, the recognizing unit 307 first performs the first matching process as in the first embodiment. In the first verification process, when it is determined that the verification is NG, the operation determination unit 306 determines whether the person moving into the current frame image and the person moving into the previous frame are the same person. . In other words, the motion determination unit 306 tracks the face area of the person even after confirming the verification intention.

  The motion determination unit 306 may be configured to determine whether or not a person is the same person by the same processing as in the first embodiment, or pattern matching using an image up to the previous frame. It may be configured to determine whether or not they are the same person.

  The recognition unit 307 includes a setting storage unit 307a. The setting storage unit 307a functions as a setting storage unit. The setting storage unit 307a stores information for selecting a collation method used for the second collation process. That is, the recognition unit 307 refers to the information stored in the setting storage unit 307a when performing the second matching process.

  For example, when the information stored in the setting storage unit 307a is information indicating the first verification method, the recognition unit 307 selects the first verification method and performs the second verification process. For example, when the information stored in the setting storage unit 307a is information indicating the second collation method, the recognition unit 307 selects the second collation method and performs the second collation process.

  Note that the information stored in the setting storage unit 307a can be changed as appropriate. The first matching method has higher matching accuracy than the second matching method. On the other hand, the first verification method takes time. For example, a setting that uses the first collation method for the second collation is conceivable so that the collation does not continuously fail. Further, when it is necessary to reduce the time required for the second verification, a setting that uses the second verification method for the second verification can be considered.

  When the recognition unit 307 determines that the person in the image used for the previous collation process and the person who has moved into the current frame are the same person by the operation determination unit 306, the recognition unit 307 stores the setting storage unit 307a. refer. Thereby, the recognition unit 307 selects a collation method used for the second collation process. The recognition unit 307 performs the second collation using the selected collation method.

  FIG. 10 is a flowchart for explaining processing in the face recognition apparatus 300 shown in FIG.

  The face recognition device 300 performs the first matching process as described in steps S11 to S24 of FIG. 6 (step S51).

In the first verification, when it is determined that the verification is NG (step S52, NO), the camera 303 of the face recognition device 300 captures and acquires an image of the imaging region (step S53). The camera 303 inputs the acquired image to the face detection unit 305 of the control unit 302. The face detection unit 305 detects an area (face area) where a person's face is captured in an image (input image) input from the camera 303 (step S54).
When the face detection unit 305 detects a face area, the motion determination unit 306 compares the coordinates and size of the face area detected with the previous frame (step S13). That is, the motion determination unit 306 compares the face area detected in the image used for the first collation with the coordinates and size.

  The motion determination unit 306 determines whether the face areas detected in the two frames are those of the same person based on the comparison result. Therefore, the motion determination unit 306 determines whether or not the difference between the coordinates and size at which the face area is detected in the previous frame is within a predetermined threshold (step S56).

  When it is determined that the difference exceeds the threshold (NO in step S56), the motion determination unit 306 determines that the person in the face area detected in the previous frame and the current frame is a different person (step S57). . In this case, the face recognition apparatus 300 proceeds to the first verification process. That is, the face recognition device 300 proceeds to step S51.

  If it is determined in step S56 that the difference is within the threshold (YES in step S56), the operation determination unit 306 determines that the persons in the face area detected in the previous frame and the current frame are the same person. (Step S58).

  When the recognition unit 307 determines that the person in the image used in the previous collation process and the person who has moved into the current frame are the same person by the operation determination unit 306, the collation method at the time of retry Is confirmed (step S59). That is, the recognition unit 307 refers to the setting storage unit 307a.

  When the information stored in the setting storage unit 307a is information indicating the first collation method, the recognition unit 307 selects the first collation method (step S60). If the information stored in the setting storage unit 307a is information indicating the second collation method, the recognition unit 307 selects the second collation method (step S61). The recognition unit 307 performs the second matching process using the selected matching method (step S62).

  When it is determined that the authentication is OK in the first verification process (step S52, YES), or when it is determined that the authentication is OK in the second verification process (step S62, YES), the output unit 309 outputs the passable signal to the outside. The data is output to the device or the gate control unit 310 (step S63).

  If it is determined that the authentication is not successful in the second verification process (step S62, NO), the output unit 309 outputs to the external device or the display unit that the verification has failed (step S64). Thereby, the face recognition device 300 notifies the pedestrian that the verification is NG.

  As described above, according to the face recognition device 300 according to the third embodiment, when the verification fails, the second verification is performed continuously. Further, the face recognition device 300 can set a collation method used for the second collation based on the selection in advance. Therefore, the second verification can be performed in a state suitable for the operation mode.

  As a result, it is possible to provide a face recognition device, a face recognition method, and a traffic control device that can recognize a person with higher accuracy in a short processing time.

  In the present embodiment, the configuration is described in which the second verification is performed when the first verification fails, but the present invention is not limited to this. The number of times that the verification is repeated may be any number. Moreover, the face recognition apparatus can set the collation method used for collation for every collation. Further, the face recognition apparatus may be configured to select a collation method used for collation based on the number of collations.

Next, a face recognition device, a face recognition method, and a traffic control device according to a fourth embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 11 is a block diagram for schematically explaining a configuration example of a face recognition apparatus 400 according to the third embodiment of the present invention. The face recognition device 400 includes an input unit 401, a control unit 402, an input interface 402A, an input interface 402B, a camera 403, an operation unit 404, a face detection unit 405, a motion determination unit 406, a recognition unit 407, a registration information storage unit 408, A registration unit 409, an output unit 410, a gate control unit 411, and the like are provided.

  The configurations of the input unit 401, the input interface 402A, the input interface 402B, the camera 403, the operation unit 404, the face detection unit 405, the motion determination unit 406, the output unit 410, and the gate control unit 411 are the same as those in the first embodiment. Since the configuration is the same as the input unit 101, the input interface 102A, the input interface 102B, the camera 103, the operation unit 104, the face detection unit 105, the motion determination unit 106, the output unit 109, and the gate control unit 110, detailed description thereof is provided. Is omitted.

  The face recognition apparatus 400 according to the fourth embodiment extracts feature information from a face image acquired by the camera 403 using the same method as that used to extract feature information when registering registration information. Further, the face recognition device 400 updates the registered registration information based on the face image acquired by the camera 403.

  The registration information storage unit 408 stores the feature information of the face of the person to be recognized together with information indicating a calculation method of the feature information used for extracting the feature information (a collation method at the time of registration).

  The recognition unit 407 extracts feature information from the detected face image based on the determination result by the motion determination unit 406. In this case, the recognizing unit 407 refers to information (matching method information) indicating the matching method of the feature information stored in the registered information storage unit 408. The recognition unit 407 selects one of a plurality of verification methods based on the referenced verification method information. That is, the recognizing unit 407 selects a collation method for extracting feature information from a face image by the same method as that used for extracting feature information stored in the registered information storage unit 408.

  The recognizing unit 407 collates the facial feature information registered in advance with the facial feature information extracted from the input image using the selected collation method, and determines whether or not they match.

  The re-registration unit 409 replaces or adds facial feature information stored in the registration information storage unit 408 as necessary.

  In addition, the collation process by the recognition unit 407 can be realized by using any method of the first embodiment, the second embodiment, and the third embodiment described above. In order to further improve the accuracy of collation, it is desirable to calculate feature information using the same method as the “collation method at registration” stored in the registration information storage unit 408. The recognizing unit 407 calculates feature information using the same method as the collation method at the time of registration, and calculates the similarity with the previously registered feature information.

  As a collation method at the time of registration, in general, all registrants often use the same collation method. For this reason, for example, when all the registrants are registered by the first collation method, the recognition unit 407 preferentially selects the first collation method. In addition, when the verification method at the time of registration differs depending on the person, the recognition unit 407 is configured to select the verification method according to the operation as described in the first embodiment or the second embodiment. There may be.

  Furthermore, the recognition unit 407 may be configured to perform similarity calculation only on feature information registered by a matching method selected according to an operation. Further, the recognition unit 407 may be configured to perform similarity calculation based on feature information prioritized by a re-registration unit 409 described later.

  The registration information storage unit 408 stores the feature information of the face of the person to be recognized together with information indicating a calculation method of the feature information used for extracting the feature information (a collation method at the time of registration). The registration information storage unit 408 stores information indicating the verification method used when the user performs registration for the first time in association with the feature information.

  For example, when storing feature information extracted by the first matching method based on a plurality of face images acquired by photographing a face from different directions, the registered information storage unit 408 is information indicating the first matching method. Are stored in association with the feature information. In addition, for example, when storing feature information extracted by the second matching method based on one face image, the registration information storage unit 408 stores information indicating the second matching method in association with the feature information. To do.

  The registration information storage unit 408 also stores the number of face images used for calculating the feature information as necessary. As a result, the face recognition device 400 can use the number of learned face images as reference information when re-registering feature information.

  The re-registration unit 409 updates or adds facial feature information stored in the registration information storage unit 408. For example, when the time has elapsed since the facial feature information was stored in the registered information storage unit 408, the accuracy of matching may be reduced. This is because aging occurs on the face. In addition, there is a possibility that collation accuracy may be reduced due to a difference in face direction, standing position, and illumination conditions during collation.

  The re-registration unit 409 explicitly updates the registration information based on the operation of the administrator.

  Alternatively, the re-registration unit 409 updates the registration information when correct identification information and a password are input from the operation unit 404 by the person who is the object of authentication after the authentication process is completed. In this case, the re-registration unit 409 updates the registration information using the face image used in the previous authentication process. Note that the re-registration unit 409 may be configured to automatically update the registration information without confirming the identification information and password after the authentication process is completed.

  FIG. 12 is a flowchart for explaining processing in the face recognition apparatus 400 shown in FIG. Steps S71 to S80 are the same processes as steps S11 to S20 shown in FIG.

  The recognizing unit 407 determines whether or not a predetermined number of face images of the person to be authenticated have been photographed before the action determining unit 406 confirms the intention of collation (step S81). When the recognition unit 407 determines that a predetermined number or more of face images have been acquired before confirming the intention to collate (step S81, YES), the recognition unit 407 refers to the registration information storage unit 408 and confirms the priority collation method (step). S82). If the priority collation method is the first collation method, the recognition unit 407 selects the first collation method (step S83).

  In addition, when the recognition unit 407 determines that a predetermined number or more of face images have not been acquired before confirming the intention to collate (NO in step S81), or as a result of confirmation in step S82, the collation method to be prioritized is determined. In the case of the second collation method, the recognition unit 407 selects the second collation method (step S84).

  The recognition unit 407 authenticates the person using the selected collation method (step S85). That is, the recognition unit 407 extracts feature information based on the face image detected by the face detection unit 405. The recognition unit 407 collates the extracted feature information with the feature information stored in the registered information storage unit 408. That is, the recognition unit 407 determines whether or not feature information that matches or approximates feature information extracted from the face image is stored in the registered information storage unit 408.

  As a result of the authentication by the recognition unit 407, when it is determined to be OK (step S85, YES), the re-registration unit 409 updates (re-registers) the registration information stored in the registration information storage unit 408 as necessary. (Step S86). The output unit 410 outputs a passable signal to the external device or the gate control unit 411 (step S87).

  As a result of authentication by the recognition unit 407, if it is determined as NG (NO in step S85), the output unit 410 outputs information indicating that the verification has failed to an external device or a display unit (step S88). Thereby, the face recognition apparatus 400 notifies a pedestrian that collation is NG.

  As described above, when the face recognition device 400 according to the fourth embodiment stores registration information, the face recognition device 400 stores information indicating a matching method used for extracting feature information in association with each other. When performing the authentication process, the face recognition device 400 selects a collation method based on information indicating the collation method associated with the registration information. Thereby, the precision of collation can be raised.

  In addition, when the authentication process is normally performed, the face recognition device 400 updates or adds registration information as necessary. The face recognition apparatus 400 can replace the stored feature information with feature information extracted by a predetermined collation method. For this reason, the face recognition apparatus 400 can transfer the stored feature information to, for example, feature information extracted by a highly accurate first collation method.

  As a result, it is possible to provide a face recognition device, a face recognition method, and a traffic control device that can recognize a person with higher accuracy in a short processing time.

  In addition, this invention is not limited to the said embodiment as it is, It can implement by changing a component in the range which does not deviate from the summary in an implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine the component covering different embodiment suitably.

FIG. 1 is a block diagram for schematically explaining a configuration example of a face recognition apparatus 100 according to the first embodiment of the present invention. FIG. 2 is an explanatory diagram for schematically explaining processing of the operation determination unit of the face recognition apparatus shown in FIG. FIG. 3 is an explanatory diagram for explaining the first collation method. FIG. 4 is an explanatory diagram for explaining the second collation method. FIG. 5 is an explanatory diagram for explaining the second collation method. FIG. 6 is a flowchart for explaining processing in the face recognition apparatus 100 shown in FIG. FIG. 7 is a block diagram for schematically explaining a configuration example of a face recognition apparatus according to the second embodiment of the present invention. FIG. 8 is a flowchart for explaining processing in the face recognition apparatus shown in FIG. FIG. 9 is a block diagram for schematically explaining a configuration example of a face recognition apparatus according to the third embodiment of the present invention. FIG. 10 is a flowchart for explaining processing in the face recognition apparatus shown in FIG. FIG. 11 is a block diagram for schematically explaining a configuration example of a face recognition apparatus according to the fourth embodiment of the present invention. FIG. 12 is a flowchart for explaining processing in the face recognition apparatus shown in FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Input image, 11 ... Frame, 100 ... Face recognition apparatus, 101 ... Input part, 102 ... Control part, 103 ... Camera, 104 ... Operation part, 105 ... Face detection part, 106 ... Motion determination part, 107 ... Recognition part , 108 ... registered information storage unit, 109 ... output unit, 110 ... gate control unit, 111 ... door, 200 ... face recognition device, 201 ... input unit, 202 ... control unit, 203 ... camera, 204 ... operation unit, 205 ... Face detection unit, 206 ... motion determination unit, 207 ... recognition unit, 208 ... registration information storage unit, 209 ... output unit, 210 ... gate control unit, 300 ... face recognition device, 301 ... input unit, 302 ... control unit, 303 DESCRIPTION OF SYMBOLS ... Camera, 304 ... Operation part, 305 ... Face detection part, 306 ... Motion determination part, 307 ... Recognition part, 307a ... Setting storage part, 308 ... Registration information storage part, 309 ... Output part, 310 ... Gate control part, 40 ... face recognition device, 401 ... input unit, 402 ... control unit, 403 ... camera, 404 ... operation unit, 405 ... face detection unit, 406 ... motion determination unit, 407 ... recognition unit, 408 ... registration information storage unit, 409 ... Re-registration unit, 410 ... output unit, 411 ... gate control unit.

Claims (12)

Registration information storage means for storing registration information in advance;
Image input means for continuously inputting an image including at least a face of a person;
Face detection means for detecting the face area of the person for each image from successive images input from the image input means;
An operation determination unit that determines the presence or absence of the person's authentication intention based on the position and size of the face area detected for each image by the face detection unit;
A first collation unit that performs collation by a first collation method based on an image of a face area detected by the face detection unit and registration information stored by the registration information storage unit;
A second collation unit that performs collation by a second collation method based on the image of the face area detected by the face detection unit and the registration information stored by the registration information storage unit;
Selecting means for selecting one of the first collating means and the second collating means;
Recognizing means for determining whether or not the person is a pre-registered person by the matching means selected by the selecting means;
A face recognition apparatus comprising:   The selection means selects the first verification method when it is determined that there are a predetermined number or more of images in which the face area of the person is detected before the action determination means confirms the person's authentication intention, The second verification method is selected when it is determined that there are less than a predetermined number of images in which the face area of the person has been detected before the action determination means confirms the person's intention to authenticate. Item 2. The face recognition device according to Item 1. The selection unit selects the second verification method when the action determination unit does not confirm the person's authentication intention,
The recognizing means determines whether the person is a person registered in advance by the second collating means using a scrutiny threshold;
The face recognition apparatus according to claim 1. When the recognition unit determines that the person appearing in the previous frame is not a person registered in advance, the action judging means is a person appearing in the current frame and a person appearing in the previous frame. And whether they are the same person,
The recognizing unit performs a retry to determine again whether or not the person is a pre-registered person when the operation determining unit determines that the person is the same person.
The face recognition apparatus according to claim 1. Further comprising setting storage means for preliminarily storing information for selecting a collation method used for retry by the recognition means;
The selection means selects either the first collation means or the second collation means based on the information stored in the setting storage means;
The face recognition apparatus according to claim 4. The registration information storage means stores collation method information indicating a collation method used for extracting feature information from an image in association with registration information,
When the selection unit determines that a predetermined number or more of the images in which the person's face area has been detected before the operation determination unit confirms the person's authentication intention, the selection unit stores the registration information storage unit. Select the same verification method as the verification method indicated by the verification method information.
The face recognition apparatus according to claim 1.   The motion determination means includes the position and size of the face area detected by the face detection means from the image input from the image input means and the position and size of the face area detected from the image of the previous frame. The face recognition apparatus according to claim 1, wherein when the difference is determined to be less than a predetermined value, it is determined that the person has an intention to authenticate. An operation input means for inputting an operation by the person;
The face recognition apparatus according to claim 1, wherein the motion determination unit determines that the person has an intention to authenticate when it is determined that an operation is input by the operation input unit.   The recognizing unit extracts feature information based on a plurality of images when the first collating method is selected by the selecting unit, and collates with registered information stored in the registered information storing unit. The face recognition apparatus according to claim 1.   The recognizing unit extracts feature information based on at least one image when the selection unit selects the second collation method, and collates with registration information stored in the registration information storage unit. The face recognition apparatus according to claim 1, wherein: A face recognition method used in a face recognition apparatus comprising: a registration information storage unit that stores registration information in advance; and an image input unit that continuously inputs an image including at least a face of a person,
Detecting the face area of the person for each image from the continuous images input from the image input means;
Determining the presence or absence of the person's authentication intention based on the position and size of the face area detected for each image;
A first collation method for performing collation based on the detected plurality of face area images and registration information stored in the registration information storage unit; and at least one detected face area image; Select one of the second verification methods for verification based on the registration information stored by the registration information storage means,
Determining whether the person is a pre-registered person by the selected matching method;
A face recognition method characterized by the above. A traffic control device for controlling the opening and closing of doors provided in an area where traffic is restricted,
Registration information storage means for storing in advance registration information of persons permitted to pass;
Image input means for continuously inputting an image including at least a face of a person;
Face detection means for detecting the face area of the person for each image from successive images input from the image input means;
An operation determination unit that determines the presence or absence of the person's authentication intention based on the position and size of the face area detected for each image by the face detection unit;
A first collation unit that performs collation by a first collation method based on an image of a face area detected by the face detection unit and registration information stored by the registration information storage unit;
A second collation unit that performs collation by a second collation method based on the image of the face area detected by the face detection unit and the registration information stored by the registration information storage unit;
Selecting means for selecting one of the first collating means and the second collating means;
Recognizing means for determining whether or not the person is a pre-registered person by the matching means selected by the selecting means;
A gate control means for controlling to open a door provided in an area where traffic is restricted when the recognition means determines that the person is a person registered in advance;
A traffic control device comprising:

Images