JP2008108243A - Person recognition device and person recognition method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a person recognition device and a person recognition method, allowing efficient access control or person monitoring for a person of a recognition object, based on photographing conditions by a plurality of cameras. <P>SOLUTION: The first face detecting part detects a face of a passenger, based on an image photographed by the first camera set to be easily recognized by the passenger. The second face detecting detects the face of the passenger, based on an image photographed by the second camera set difficult to be recognized by the passenger. A classification part classifies the passenger, based on a detection result of the face by the first face detecting part and a detection result of the face by the second face detecting part, and regulates a threshold value for authentication used for face collation processing by a face collation part, based on a classification result. The face collation part calculates the similarity between feature information of the face obtained from a face image of the passenger and feature information of the face of a registrant stored in a face feature control part, and determines whether the passenger is the registrant or not, according to whether the calculated similarity is the regulated threshold value for the authentication or more, or not. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a person recognition apparatus and a person recognition method applied to, for example, an entrance / exit management system or a person monitoring system using biometric authentication.

  Conventionally, as a technique applied to entrance / exit management or person monitoring, a technique has been proposed in which a pedestrian (passerby) is photographed, the photographed image is recorded, and the pedestrian is identified from the photographed image. Has been. The most popular of these technologies is a system that constantly records video taken by a surveillance camera. However, in a system that constantly records video captured by a surveillance camera, it is not easy to search for an image of a specific person from videos recorded over a long period of time. For this reason, a system that records video captured by the camera only when the human sensor detects a person, or a system that records video only when a human-like image is detected from video captured by the camera is proposed. Has been.

  For example, Japanese Patent Laid-Open No. 2003-169320 (Patent Document 1) is a system in which first and second surveillance cameras are installed and a person who is acting with awareness of these surveillance cameras is monitored. This is based on the premise that a suspicious person is likely to act with the surveillance camera in mind. In the system described in Japanese Patent Laid-Open No. 2003-169320, an image including a person determined to be conscious of the first monitoring camera and the second monitoring camera is recorded, or an alarm is output.

  However, in the technique described in Japanese Patent Laid-Open No. 2003-169320, depending on the installation state of the camera, it is often determined that a general person who does not need to hide his face is likely to be a suspicious person, and misinformation is likely to increase. it is conceivable that. In addition, the technique described in Japanese Patent Application Laid-Open No. 2003-169320 has a problem that when a suspicious person does not project a face on a surveillance camera, it is impossible to record an image or output an alarm.

  Japanese Patent No. 3088880 (Patent Document 2) describes a system for notifying an administrator and a passer-by when a person hiding his face is detected from an image captured by a camera. . This is based on the premise that there is a high possibility that a person who does not show his / her face on the surveillance camera (person who hides his / her face) is a suspicious person. Japanese Patent No. 3088880 discloses a system that extracts a region that is likely to be a person's head from an image captured by a monitoring camera, and determines whether or not the person's face is hidden based on whether or not the person's face has been detected from that region. Is described.

  However, in Japanese Patent No. 3088880, a person whose face is hidden is treated as a suspicious person. For this reason, in the technique of Japanese Patent No. 3088880, there is a problem that it is determined that all persons who happen to be faceless in the direction of the camera and persons whose face cannot be detected by sunglasses or masks are likely to be suspicious. There is.

  Japanese Unexamined Patent Application Publication No. 2004-118359 (Patent Document 3) describes a technique for stabilizing face collation for passers-by based on images taken by a plurality of cameras installed under different conditions. Japanese Patent Application Laid-Open No. 2004-118359 describes a technique for collating images of a plurality of cameras installed under various conditions using human faces under conditions according to the installation conditions of each camera. ing. However, Japanese Patent Laid-Open No. 2004-118359 does not describe the point of determining a person who seems to be a suspicious person.

In the conventional techniques as described above, there is a problem that it is difficult to reliably detect a person who seems to be a suspicious person or to perform an efficient authentication process according to each person's suspicious person.
JP 2003-169320 A Japanese Patent No. 3088880 JP 2004-118359 A

  An object of one aspect of the present invention is to provide a person recognition apparatus and a person recognition method capable of performing efficient access control or person monitoring for a person.

  A person recognition apparatus according to an aspect of the present invention includes a first image acquisition unit that acquires an image taken by a camera installed in a state that is easily recognized by a person, and a camera installed in a state that is difficult to be recognized by a person. A second image acquisition unit that acquires an image captured by the first image acquisition unit, a first face detection unit that detects a human face from the image acquired by the first image acquisition unit, and the second image acquisition unit. A second face detection unit that detects the face of the person from the acquired image; an association unit that associates the person photographed by the first camera with the person photographed by the second camera; and the correspondence A classification unit that classifies the person based on a result of association by the attaching unit, a detection result of the face by the first face detection unit, and a detection result of the face by the second face detection unit;

  According to another aspect of the present invention, there is provided a person recognition method that acquires an image taken by a first camera installed in a state that is easily recognized by a person, and uses a second camera installed in a state that is difficult for a person to recognize. Acquiring a photographed image, detecting a human face from an image photographed by the first camera, detecting the human face from an image photographed by the second camera, and the first camera; The person photographed by the second camera and the person photographed by the second camera are associated with each other, the result of the association, the face detection result from the image photographed by the first camera, and the second camera The person is classified based on a face detection result from the captured image.

  According to one aspect of the present invention, it is possible to provide a person recognition device and a person recognition method capable of performing efficient access control or person monitoring for a person.

Embodiments of the present invention will be described below with reference to the drawings.
First, the first embodiment will be described.
FIG. 1 schematically shows a configuration example of a person (passerby) recognition device 10 as an access control device according to the first embodiment.
A passer-by identification apparatus 10 shown in FIG. 1 extracts face feature information of a person from an image taken by a camera, and performs face matching that matches the extracted face feature information and registrant face feature information (face matching). Functions as a device. Further, the passer-by identification apparatus 10 shown in FIG. 1 permits access (passage in the example shown in FIG. 1) of a person who has been determined by facial matching that the facial feature information matches the facial feature information of the registrant. It also functions as an access control device.

  In addition, the passer-by identification device 10 shown in FIG. 1 is, for example, access control that allows only a specific person (registrant) to enter the security area, or an important customer or a caution from the passer-by It is assumed that the present invention is applied to person detection for detecting a specific person such as a person. For example, the passer-by identification apparatus 10 shown in FIG. 1 is an entrance management system that manages a person who wants to enter a specific building or security area, or a large number of persons such as commercial facilities, entertainment facilities, or transportation facilities. It is assumed that the present invention is applied to a monitoring system that monitors a person in a place where a person passes.

  As shown in FIG. 1, the passer-by identification apparatus 10 includes a first image acquisition unit 111, a second image acquisition unit 112, a first person detection unit 121, a second person detection unit 122, and a first person. It includes a face detection unit 131, a second face detection unit 132, a person association unit 140, a classification unit 150, a face feature management unit 160, a face matching unit 170, a traffic control unit 180, a history management unit 190, and the like.

  The first image acquisition unit 111 includes, for example, a camera, an A / D converter, an output interface, and the like. The camera captures an image including at least the face of the passerby M. The A / D converter digitizes the image taken by the camera. The output interface outputs the image digitized by the A / D converter to the person detection unit 121. The camera of the first image acquisition unit 111 (hereinafter also referred to as the first camera) is installed so that it can be easily recognized by the passerby M who is the recognition target person. Further, a display unit 111 a is provided in the vicinity of the camera of the first image acquisition unit 111. The display unit 111a is installed in the vicinity of the camera of the first image acquisition unit 111 so that the camera of the first image acquisition unit 111 is easily noticed by the recognition target person M. The display unit 111a is configured by, for example, a liquid crystal display.

  The second image acquisition unit 112 includes, for example, a camera, an A / D converter, an output interface, and the like. The camera captures an image including at least the face of the passerby M. The A / D converter digitizes the image taken by the camera. The output interface outputs the image digitized by the A / D converter to the person detection unit 122. The camera of the second image acquisition unit 112 (hereinafter, also referred to as a second camera) is installed so that it is difficult for the passer-by M who is a recognition target to be recognized. Furthermore, an acrylic plate 112a that functions as a blind is provided on the front surface of the camera of the second image acquisition unit 112. The acrylic plate 112a is installed so as to hide the camera so that the second image acquisition unit 112 is not easily detected by the passerby M. That is, the camera (second camera) of the second image acquisition unit 112 may be a hidden camera. For example, the second camera may be a small camera that is hard to be found by a passer-by other than hiding with the acrylic plate 112a.

  That is, the camera of the first image acquisition unit 111 is easily recognized by the passer-by M, while the camera of the second image acquisition unit 112 is not easily recognized by the passer-by M. It has become. As a result, the first image acquisition unit 111 can capture the face image of the passer-by M who is conscious of the presence of the camera, and the second image acquisition unit 112 is in the state of being unaware of the presence of the camera. The face image of the person M can be taken.

  The first person detection unit 121 and the second person detection unit 122 detect image areas (person areas) that are likely to be persons from the images acquired by the first image acquisition unit 111 and the second image acquisition unit 112, respectively. . Information indicating the person-like image area detected by the first person detection unit 121 and the second person detection unit 122 is output to the first face detection unit 131 and the second face detection unit 132, respectively.

  As a technique for detecting a person region by the first and second person detection units 121 and 122, Reference 1 (Hiroaki Nakai: “A moving object detection technique using a posteriori probability”, Information Processing Society of Japan Research Report, SIG-CV90- 1, 1994) can be applied. In the technique described in Document 1, a change area is extracted as a person area using a difference between background images for an input image. Further, in Document 1, characteristic information such as the shape, area, or color distribution information of the extracted person area (change area) can be acquired as the characteristics of the person (passerby) in the person area.

  In addition, in order to improve the detection accuracy of a person, the passer-by identification device 10 may be added with a configuration for predicting the position or moving direction of the passer-by M. In this case, the passer-by identification device 10 can perform control to efficiently photograph the passer-by M or to detect the person area efficiently based on the prediction result of the passer's position or moving direction. Become. For example, as a configuration for predicting the position or moving direction of the passerby M, a distance sensor for detecting the position of the person, a human sensor for detecting the person, or another for detecting the movement of the person Such a surveillance camera (for example, a camera capable of wide-angle shooting) can be considered.

  The first and second face detection units 131 and 132 perform face area detection processing. That is, the first face detection unit 131 and the second face detection unit 132 are regions that are likely to be faces in the images within the person region detected by the first person detection unit 121 and the second person detection unit 122, respectively. Is detected. For example, the first and second face detection units 131 and 132 detect a position that gives the highest correlation value as a face region by obtaining a correlation value while moving a prepared template in the image. Note that the first and second face detection units 131 and 132 may use a method of extracting a face region using the eigenspace method or the subspace method.

  The first and second face detection units 131 and 132 further detect feature information of the face of the person from the detected face area image. That is, the first and second face detection units 131 and 132 detect the position of a facial part such as an eye, nose, or mouth as a characteristic part of the face from the detected face area image. The first and second face detection units 131 and 132 calculate the facial feature information of the person based on the position of the facial part detected as the facial feature part.

  For example, methods for detecting the position of a facial part such as eyes, nose or mouth are described in Reference 2 (Kazuhiro Fukui, Osamu Yamaguchi: “Face feature point extraction by combination of shape extraction and pattern matching”), IEICE Transactions (D), vol.J80-D-II, No. 8, pp2170-2177 (1997)) and the like can be applied.

  Further, as a detection method for detecting the eyes, nose or mouth, Reference 3 (Mayumi Yuasa, Ayako Nakajima: “Digital Make System Based on High-Precision Facial Feature Point Detection”, Proc. (2004)) can also be applied.

  In general, when extracting one piece of feature information from one image, a position indicating a region where the correlation value is maximized is obtained by obtaining a correlation value with a template of feature information to be extracted for the entire image. The size is obtained as a result of extracting feature information. In addition, when extracting a plurality of feature information from a plurality of images that are continuous in time series, the feature in each image is obtained by obtaining a local maximum correlation value in each image while considering the overlap of the plurality of feature information. Information candidates are narrowed down. A plurality of feature information is selected by considering the relationship (temporal transition) between successive images for feature information candidates in each image.

  As the facial feature information extracted by the first and second face detection units 131 and 132, for example, information using shading information of the face region is obtained. In this case, the first and second face detection units 131 and 132 cut out the face area into a fixed size and a fixed shape based on the detected positions of the face parts, and the shading of the cut out face area is changed. Get information. Here, as the facial feature information, information representing the gray value of an area of m pixels × n pixels as an m × n-dimensional vector (feature vector) is used.

  Such similarity between feature vectors is calculated by a method called a simple similarity method, for example. In the simple similarity method, the length of the first vector and the length of the second vector are normalized to be “1”, and the inner product of the normalized first and second vectors is calculated. Thus, the similarity indicating the similarity between the vectors is obtained. When one feature vector obtained from one image is collated with a predetermined feature vector (a feature vector registered in advance), the first and second face detection units 131 and 132 are configured to detect one feature image from one image. What is necessary is just to calculate a feature vector.

  However, highly accurate recognition processing can be performed by calculating feature information from moving images as a plurality of continuous images. Therefore, here, a method for calculating feature information from a moving image will be described. In addition, a moving image shall be acquired as an image of each continuous frame for every predetermined time. The first and second face detection units 131 and 132 calculate a partial space as face feature information obtained from a moving image. The partial space is information calculated based on the correlation of feature vectors obtained from the images of each frame.

  That is, the first and second face detection units 131 and 132 calculate feature vectors from the image of the face area of m × n pixels in each frame image. The face detection units 131 and 132 calculate a correlation matrix (or covariance matrix) for each feature vector obtained from each image, and obtain an orthonormal vector (eigenvector) by a well-known KL expansion. The subspace is expressed using a set of eigenvectors by selecting k eigenvectors corresponding to eigenvalues in descending order of eigenvalues. Here, when the correlation matrix is Cd and the eigenvector matrix is Φ, the following relationship of Equation 1 holds.

Cd = ΦdΛdΦdT (Equation 1)
The matrix Φd of eigenvectors can be calculated by the above equation 1. This information becomes a partial space (input subspace) as facial feature information. In addition, the facial feature management unit 160 may register a partial space (registered partial space) calculated by the above-described calculation method from facial images taken in advance for registration as facial feature information. .

  The person association unit 140 performs processing for associating a person photographed by the first image acquisition unit 111 with a person photographed by the second image acquisition unit 112. That is, the person associating unit 140 includes a person detection result by the first person detection unit 121, a person detection result by the second person detection unit 122, a face detection result by the first face detection unit 131, or Based on the detection result of the face by the second face detection unit 132, the person photographed by the first image acquisition unit 111 and the person photographed by the second image acquisition unit 112 are associated with each other. In other words, the person association unit 140 identifies the same person as the person detected from the image captured by the first image acquisition unit 111 (or the second image acquisition unit 112) as the second image acquisition unit 112 (or Detection is performed from a person detected from an image captured by the first image acquisition unit 111).

  The person associating unit 140 includes, for example, the shape, area, color distribution information, and the like in the person image area detected by the first person detection unit 121 and the person image area detected by the second person detection unit 122. To associate each person with The color distribution information is identified by a histogram for each color of R, G, and B. That is, the image of the person taken by the first image acquisition unit 111 and the image of the person taken by the second image acquisition unit 112 are similar in shape, area, color distribution, etc. It is corresponded as.

  The person association unit 140 may be provided with an auxiliary detection unit 140a in order to increase the person association accuracy. As the auxiliary detection unit 140a, for example, a distance sensor, a human sensor, a temperature sensor, a weight sensor, or the like may be provided.

  For example, in the distance sensor as the auxiliary detection unit 140a, the distance from the sensor installation position to the person can be estimated. For this reason, the position of each person can be tracked based on the distance to the person detected by the distance sensor. Based on the tracking result of the position of each person, the person association unit 140 detects the person detected from the image captured by the image acquisition unit 111 and the person detected from the image captured by the image acquisition unit 112. Can be associated with each other.

  Further, the presence sensor as the auxiliary detection unit 140a can detect the presence of a person. The temperature sensor as the auxiliary detection unit 140a can detect the presence of a person based on the temperature (body temperature) of the person. The weight sensor as the auxiliary detection unit 140a can detect the presence of a person based on the weight of the person. Each person can be tracked based on the detection result by the sensor as the auxiliary detection unit 140a. Therefore, based on the tracking result of each person, the person association unit 140 detects the person detected from the image captured by the image acquisition unit 111 and the person detected from the image captured by the image acquisition unit 112. Can be associated with each other.

  Furthermore, as the auxiliary detection unit 140a for increasing the accuracy of person association, a surveillance camera capable of photographing the entire passage at a wide angle may be provided. In this case, the monitoring camera as the auxiliary detection unit 140a can track each person from the captured image of the entire passage. Based on the tracking result of each person, the person association unit 140 determines the person detected from the image captured by the image acquisition unit 111 and the person detected from the image captured by the image acquisition unit 112. It is possible to associate them.

  The classification unit 150 classifies each person based on the state (behavior pattern) of the person associated by the person association unit 140. In the classification unit 150, based on the state of the person photographed by the first image acquisition unit 111 and the state of the person photographed by the second image acquisition unit 112 (person associated by the person association unit 140). It is determined how the behavior pattern of the person is classified. For example, the classification unit 150 classifies each person into several patterns based on the face detection result by the face detection unit 131 and the face detection result by the face detection unit 132.

  That is, whether the classification unit 150 detects a human face detected from the image captured by the first image acquisition unit 111 (a state in which the face is hidden from the camera of the first image acquisition unit 111). And whether or not a human face detected from the image captured by the second image acquisition unit 112 is detected (a state in which the face is hidden from the camera of the second image acquisition unit 112). The person is classified into four patterns. Such a classification result by the classification unit 150 is an element that determines the processing content by the face matching unit 170 or the traffic control unit 180.

  For example, as described above, the camera of the first image acquisition unit 111 is installed so that the passerby M can easily recognize it. For this reason, it can be estimated that a person whose face is detected from the image acquired by the first image acquisition unit 111 is unlikely to be a suspicious person. Based on such an assumption, the classification unit 150 performs face matching on a person whose face is detected from the image acquired by the first image acquisition unit 111, and uses the image acquired by the first image acquisition unit 111. A person whose face is not detected is not allowed to pass without performing face matching.

  Further, the camera of the second image acquisition unit 112 is installed so that it is difficult for the passerby M to recognize it. For this reason, the person whose face is detected from the image acquired by the second image acquisition unit 112 is a person who knows the existence of the camera and faces the camera, or the face without knowing the existence of the camera. It is estimated that it is one of the persons being photographed. Based on such estimation, the classification unit 150 detects a face from the image acquired by the second image acquisition unit 112 among the persons whose faces are detected from the image acquired by the first image acquisition unit 111. The person is controlled so that the passage is easily permitted. On the other hand, the classification unit 150 is a person whose face is not detected from the image acquired by the second image acquisition unit 112 among the persons whose face is detected from the image acquired by the first image acquisition unit 111. Is controlled so as to determine whether or not traffic is allowed by normal face matching. An example of the classification process of each person by the classification unit 150 will be described in detail later.

  The face feature management unit 160 stores registrant face feature information as dictionary data. For example, the facial feature management unit 160 registers the partial space as described above as facial feature information of the registrant. However, the registrant's face feature information registered in the face feature management unit 160 includes the registrant's face image (moving image or one image), m × n feature vectors obtained from each face image, or K It may be a correlation matrix just before -L expansion. The facial feature information of the registrant is stored by the facial feature management unit 160 in association with an ID number as identification information for identifying the registrant.

  The face feature management unit 160 may store one face feature information for each registrant, or may store a plurality of face feature information for each registrant. When storing a plurality of facial feature information for each registrant, the face matching unit 170 uses, for example, one facial feature information for each registrant selected according to the situation. Alternatively, the face matching process may be performed using feature information of a plurality of faces of each registrant.

  The face collation unit 170 uses the facial feature information detected by the first facial detection unit 131 or the second facial detection unit 132 and the facial feature information of the registrant stored in the facial feature management unit 160. It is to be verified. The face matching unit 170 calculates the similarity between the two as a matching result. The face matching unit 170 determines whether or not the person is allowed to pass based on whether or not the calculated similarity is equal to or greater than an authentication threshold.

  For example, in the face matching unit 170, the facial feature information obtained by the first face detection unit 131 or the second face detection unit 132 is stored in the partial space (input partial space) and the facial feature management unit 160. The degree of similarity with one or more subspaces (registered subspaces) is obtained. Thereby, the face collation part 170 determines whether a passerby is a registrant by comparing the calculated similarity with the threshold value for authentication.

  The authentication threshold value can be adjusted based on a predetermined authentication threshold value. The threshold value for authentication is adjusted based on the classification result by the classification unit 150 in the processing described later.

  When there are a plurality of passersby in one image acquired by the first image acquisition unit 111 or the second image acquisition unit 112, the face verification unit 170 has detected the above-described face verification process. By repeating the process for the number of persons, face collation processing is performed for all passersby present in the acquired image.

  As a calculation method for obtaining the similarity between two subspaces, a method such as a subspace method or a composite similarity method can be applied. For example, the similarity between two subspaces can be found in Reference 4 (Kenichi Maeda, Sadaichi Watanabe: “Pattern matching method introducing local structure”, IEICE Transactions (D), vol. J68-D, No. .3, pp 345-352 (1985)). In Reference 3 above, an angle formed by two partial spaces is defined as a similarity. Here, when the correlation matrix is Cin and the eigenvector is Φin, the relationship of Equation 2 below holds.

Cin = ΦinΛinΦinT (Formula 2)
The eigenvector Φin can be calculated by the above equation 2. Thereby, the similarity (0.0-1.0) of two partial spaces represented by two (PHI) in and (PHI) d is computable.

  In addition, when a plurality of faces exist in one acquired image, the face collation unit 170 sequentially detects the facial feature information of each person that has been stored in the facial feature management unit 160. The degree of similarity with feature information is calculated brute force. Thereby, it is possible to obtain face matching results for all persons existing in one acquired image. For example, when an X passerby walks (that is, when facial feature information of X name is detected from one acquired image), facial feature information for Y names is stored in the facial feature management unit 160. If stored, the face collation result of all X names can be obtained by performing similarity calculation “X × Y” times.

  Further, when the input subspace obtained from the acquired m images does not collate with any registered subspace (that is, when the person being photographed is not determined as any of the registrants), The input subspace is updated with the image of the previous frame and the images of the past plural frames. Such an update of the input subspace is performed by adding the correlation matrix for the image of the next acquired frame to the sum of the correlation matrices created from the images of a plurality of past frames and calculating the eigenvector again. good. That is, when face matching is performed using images (moving images) obtained by continuously capturing the face images of passers-by, the accuracy is gradually increased by performing matching while updating the input subspace with sequentially input images. Calculation is possible.

  The pass control unit 180 controls the pass of the passer-by M based on the result of face collation by the face collation unit 170 or the classification result by the classification unit 150. The passage control unit 180 controls the passage of the passer-by M by outputting a signal for controlling opening / closing of an automatic door, an electronic lock door, a gate, and the like.

  For example, the passage control unit 180 is configured to pass an automatic door, an electronic lock door, a person who is determined to be allowed to pass (a person who is determined to match the registrant) based on the result of the comparison in the face matching unit 170, Alternatively, a control signal for opening the gate is output. In addition, the traffic control unit 180 includes an automatic door, an electronic lock, and an electronic door to prevent traffic of a person who is determined not to pass (a person who is determined not to match the registrant and a person whose face cannot be matched). A control signal for closing a door or a gate is output (or a control signal for opening is not output).

  The history management unit 190 stores history information (passage history) regarding the passerby M. The history management unit 190 may be realized by a management server or the like that can communicate with the passer-by identification device 10. In the history management unit 190, the determination result for each passer-by, the date and time of passage, the captured image, and the like are recorded as history information. Further, history information may be recorded in the history management unit 190 only for the passerby M determined to be a suspicious person.

Next, an installation example of the first image acquisition unit 111 and the second image acquisition unit 112 will be described.
As described above, the first image acquisition unit 111 and the second image acquisition unit 112 include a camera, an A / D converter, an output interface, and the like. The first image acquisition unit 111 and the second image acquisition unit 112 have different installation states and the like in order to photograph the passerby M under different conditions. As described above, the first image acquisition unit 111 is installed at a position where the passer-by M can easily recognize, and the display unit 111a is provided in the vicinity of the camera so that the passer-by M can easily recognize the presence of the camera. ing. The 2nd image acquisition part 112 is provided with the acrylic board 112a for concealing a camera so that the passer-by M cannot recognize easily. The first image acquisition unit 111 and the second image acquisition unit 112 may have various installation forms depending on the shape of the passage through which the passer-by M passes.

  FIGS. 2 and 3 are diagrams illustrating an installation example of the first image acquisition unit 111 and the second image acquisition unit 112. In the example shown in FIG. 2, for example, a passage P1 leading to an entrance (gate) G1 of an area (security area) where only a registrant can enter, and a first image acquisition unit 111 and a second image acquisition unit in the passage P1. 112 shows an example of installation.

  As shown in FIG. 2, the passage P1 is bent in front of the gate G1. Accordingly, it is considered that the person (passerby) M who intends to enter the security area turns left before the gate G1 and reaches the gate G1.

  The second image acquisition unit 112 is installed in a place where the passage P1 is bent. The camera (second camera) of the second image acquisition unit 112 is installed so that a passerby walking toward the gate G1 photographs a passerby who is walking when the passerby approaches a bending place. ing. In the example shown in FIG. 2, the second camera is installed outside the wall forming the passage P <b> 1 at a place where the passage P <b> 1 is bent. The second camera is hidden by an acrylic plate 112a installed along the wall. With such a configuration, the second camera is hardly recognized by the passerby M. That is, with the second camera, it is possible to photograph the passerby M who is highly likely not to be aware of the camera.

  The first image acquisition unit 111 is installed near the gate G1 of the passage P1. The camera (first camera) of the first image acquisition unit 111 is installed so that a passerby traveling from the passage P1 toward the gate G1 can be photographed. Further, in the vicinity of the gate G1, a display unit 111a that displays an image or the like that is being guided or photographed to a passer-by M that is approaching the gate G1 is installed. When the passer-by M pays attention to the image displayed on the display unit 111a, the first camera is further conscious of the passer-by M. That is, with the first camera, it is possible to photograph the passerby M who is highly likely to be conscious of the camera.

  FIG. 3 is a diagram illustrating a setting example of the passage P2 leading to the entrance (gate) G2 of the security area and the first image acquisition unit 111 and the second image acquisition unit 112 installed in the passage P2, for example. . In the example shown in FIG. 3, the passage P2 is linear. Therefore, it is considered that the person (passerby) M who intends to enter the security area travels substantially straight and reaches the gate G2.

  In the configuration example shown in FIG. 3, the second image acquisition unit 112 is installed so as to photograph the face of a passerby traveling straight through the passage P2 toward the gate G2. That is, the second camera is installed so as to photograph a passerby walking toward the gate G2. In the example shown in FIG. 3, the second camera is installed outside (or inside) the wall forming the passage P2 in front of the first camera installed in front of the gate G2 in the passage P2. ing. The second camera is hidden by the acrylic plate 112a so that it is difficult for the passerby M to recognize it. That is, with the second camera, it is possible to photograph the passerby M who is highly likely not to be aware of the camera.

  In the configuration example illustrated in FIG. 3, the first image acquisition unit 111 is installed in the vicinity of the passage P2 in front of the gate G2. The first camera is installed so as to be able to photograph a passerby (or a passerby who has reached the gate G2) immediately before reaching the gate G2 by going straight through the passage P2. Further, in the vicinity of the gate G2, a display unit 111a that displays an image or the like that is being guided or photographed for a passer-by M that approaches the gate G2. When the passer-by M pays attention to the image displayed on the display unit 111a, the first camera is further conscious of the passer-by M. That is, with the first camera, it is possible to photograph the passerby M who is highly likely to be conscious of the camera.

Next, a method for classifying each person by the classification unit 150 will be described.
As described above, the classification unit 150 classifies each person based on the detection results of the faces by the first face detection unit 131 and the second face detection unit 132, for example. How to classify each person and what kind of processing to perform according to the classification result is set appropriately according to the installation status of the passer-by identification device, the overall system operation mode, security policy, etc. It is what is done.

  That is, the classification unit 150 classifies each person according to a preset classification standard. In other words, the classification unit 150 determines the state of a person (for example, a face detection result) captured by the camera of the first image acquisition unit 111 and the camera of the second image acquisition unit 112 based on a predetermined classification criterion. The classification of the combination with the state of the person photographed in step (for example, the detection result of the face) is determined.

FIG. 4 is a diagram illustrating an example of classification criteria for each person by the classification unit 150.
Here, the camera (first camera) of the first image acquisition unit 111 is installed so as to be easily recognized by the passer-by, and the camera (second camera) of the second image acquisition unit 112 is recognized by the passer-by. It is assumed that it is installed so that it is hard to be done.

  According to “No. 3” and “No. 4” shown in FIG. 4, when the face of the person detected from the image acquired by the first image acquisition unit 111 cannot be detected, that is, the first person When the face of the person detected by the detection unit 121 cannot be detected by the first face detection unit 131 (first camera image: face detection NG), the classification unit 150 is a person who seems to be a suspicious person. It is classified as being, and it is determined that traffic to the person is impossible.

  Furthermore, according to “No. 4” illustrated in FIG. 4, a human face detected from the image acquired by the first image acquisition unit 111 cannot be detected, and the human face acquired by the second image acquisition unit 112 is acquired. When the face of the person associated with the person detected from the image cannot be detected (first camera image: face detection NG and second camera image: face detection NG), the classification unit 150 The person is classified as a person who seems to be a suspicious person whose face could not be detected at all, and it is determined that the person cannot pass through the person.

  In this case, history information of a person who is determined to be a suspicious person is recorded in the history management unit 190. The history information recorded in the history management unit 190 is information such as a determination result and a determination time because the face of the person has not been detected. However, since an image in which the person is detected exists, the image in which the person is detected may be recorded together with history information. In addition, since the person whose face could not be detected as described above is a person whose face cannot be confirmed later (the face is not recorded), it is assumed that he is the person who should be most wary.

  Further, according to “No. 3” illustrated in FIG. 4, the face of the person detected from the image acquired by the first image acquisition unit 111 cannot be detected, and acquired by the second image acquisition unit 112. When the face of the person associated with the person detected from the image can be detected (first camera image: face detection NG and second camera image: face detection OK), the classification unit 150 The person is classified as a person who seems to be a suspicious person whose face could be detected, and it is determined that the person cannot pass and the face image of the person is recorded in the history management unit 190. In this case, history information including a face image of a person determined to be a suspicious person is recorded in the history management unit 190. For this reason, the face of the person can be confirmed with the face image included in the history information.

  According to “No. 1” and “No. 2” shown in FIG. 4, when the face of a person detected from the image acquired by the first image acquisition unit 111 can be detected, that is, the first person detection When the face of the person detected by the unit 121 can be detected by the second face detection unit 131 (first camera image: face detection OK), the classification unit 150 is a person who is not likely to be a suspicious person. It is determined that face collation for determining whether or not the person can pass is executed.

  Furthermore, according to “No. 2” illustrated in FIG. 4, the face of the person detected from the image acquired by the first image acquisition unit 111 can be detected, and the image acquired by the second image acquisition unit 112 When the face of the person associated with the person detected from the above cannot be detected (first camera image: face detection OK and second camera image: face detection NG), the classification unit 150 The person is classified as a person who does not look like a suspicious person (a person in a general walking state), and the authentication threshold value used for face matching for the person is determined to be a predetermined value.

  According to “No. 1” illustrated in FIG. 4, the human face detected from the image acquired by the first image acquisition unit 111 can be detected and detected from the image acquired by the second image acquisition unit 112. When the face of the person associated with the person is detected (first camera image: face detection OK and second camera image: face detection OK), the classification unit 150 determines that the person The person is classified as a person who does not seem to be a suspicious person who knows the position of the second camera, and determines that the threshold value for authentication used for face matching for the person is relaxed (adjusted) from a predetermined value.

  That is, in the example shown in FIG. 4, it is determined whether or not it seems to be a suspicious person depending on whether or not a face can be detected from an image taken by the first camera. Such a setting is such that a person who does not hide his face (looking at the camera) with respect to the first camera installed so as to be easily recognized by a passerby is unlikely to be a suspicious person. It is based on the assumption that it is. Therefore, an operation mode in which at least a registrant is informed in advance to face the first camera is assumed.

  Further, in the example shown in FIG. 4, the authentication threshold for a person whose face can be detected from the image captured by the first camera is adjusted depending on whether the face can be detected from the image captured by the second camera. It has become so. In other words, for a person whose face can be detected from the image taken with the first camera and the image taken with the second camera, matching with the registrant succeeded by relaxing the authentication threshold used for face matching. (Passage is easily permitted). In such a setting, a person whose face is photographed by the second camera installed in a state that is difficult to be recognized by a passerby is likely to be a registrant who already knows the existence of the second camera. Based on the guess. Therefore, an operation mode in which the registrant is notified in advance of the position of the second camera is conceivable.

  Note that the above settings (classification method for each person) can be set as appropriate according to the system operation mode, the installation status of the second camera, the secrecy of the second camera, and the like. This is because it is presumed that the state of the passerby photographed by the first and second cameras differs depending on the operation mode of the passer's system, the installation status of the second camera, the secrecy of the second camera, and the like. It is. For example, depending on whether the second camera is in a state where it is easy to capture the face of a passerby in a general walking state, whether the second camera is easy to be found by a passer-by, etc. It is presumed that the state of the passerby whose face was photographed is different.

Next, an operation example of the passer-by identification apparatus 10 according to the first embodiment will be described.
FIG. 5 is a flowchart for explaining an operation example of the passer-by identification device 10.
Here, it is assumed that a classification method as shown in FIG. 4 is preset.

  First, it is assumed that the camera (first camera) of the first image acquisition unit 111 and the camera (second camera) of the second image acquisition unit 112 are capturing a predetermined area in the passage. Images (images of each frame) captured by the first camera are sequentially supplied to the first person detection unit 121 via the first image acquisition unit 111. Similarly, images (images of each frame) captured by the second camera are sequentially supplied to the second person detection unit 122 via the second image acquisition unit 112. As a result, the first person detection unit 121 and the second person detection unit 122 perform processing for detecting a person from the supplied images (steps S101 and S102).

  In this state, when the first person detection unit 121 detects a person (step S101, YES), the first face detection unit 131 detects a face region from the person image detected by the first person detection unit 121. The process which detects is performed (step S103). Similarly, when the second person detection unit 122 detects a person (NO in step S101 and YES in step S102), the second face detection unit 132 is detected by the second person detection unit 122. A process of detecting a face region from a person image is performed (step S104). When a person is detected by the first person detecting unit 121 or the second person detecting unit 122, the person associating unit 140 detects the person detected by the first person detecting unit 121 and the second person detecting unit 121. The person detected by the person detection unit 122 is associated with the person (step S105).

  The association process by the person association unit 140 is a process for associating a person photographed with the first camera with a person photographed with the second camera. For this reason, an image of a person photographed with only one camera cannot be associated with an image photographed with the other camera. However, depending on the operation mode, when a person is detected only from an image captured by one camera, it can be obtained as a result of the association process that the person has not been detected from an image captured by the other camera. You may do it.

  The classification unit 150 classifies the person (passerby) based on the result of the association by the person association unit 140 and the detection result of the person by the first person detection unit 121 and the second person detection unit 122. It is determined whether it can be determined (step S106). Here, in the example illustrated in FIG. 4, it is assumed that the person is a person photographed by the first camera and the second camera. Therefore, when the classification as shown in FIG. 4 is set, the classification unit 150 causes the person detected from the image captured by the first camera and the person detected from the image captured by the second camera. Are determined as to whether or not they are associated with each other as the same person. However, even if a person is detected only from an image captured by one camera, the classification unit 150 can detect the person from an image captured by one camera as long as the person can be classified. If it is detected, it is determined that the person can be classified.

  For example, the second camera is installed so that it is difficult for a passer-by to recognize it. For this reason, it may be difficult to reliably detect a person from an image captured by the second camera. Further, in the installation example shown in FIG. 2 or FIG. 3, the first camera is installed in the vicinity of the entrance, and the second camera is installed in front of the first camera. In such a case, even if the person detected from the image captured by the first camera is not detected from the image captured by the second camera, “No 2” shown in FIG. Similar to “No 3”, the person may be classified. In this case, the person association unit 140 indicates that the person associated with the person detected from the image captured by the first camera has not been detected from the image captured by the second camera. As a result, it may be supplied to the classification unit 150.

  If it is determined that the person cannot be classified by the above determination (step S106, NO), the processes of steps S101 to S106 are repeated until the classification by the classification unit 150 is possible.

  If it is determined that the person can be classified by the above determination (step S106, YES), the classification unit 150 performs a classification process on the person whose result of the association process is obtained by the person association unit 140. . That is, the classification unit 150 first determines whether or not a face is detected by the first face detection unit 131 from an image captured by the first camera (a person image detected by the first person detection unit 121). Is determined (step S107).

  When a face is not detected from the image captured by the first camera (step S107, NO), the classification unit 150 further determines the person (first person) from the image captured by the second camera. It is determined whether the face of the person associated with the person detected by the detection unit 121 has been detected by the second face detection unit 132 (step S108).

  When the face cannot be detected from the images (person images) taken by the first camera and the second camera by these determinations (step S108, NO), the classification unit 150 is based on the classification shown in FIG. Then, the person is classified as a person who seems to be a suspicious person whose face could not be detected. In accordance with such classification, the classification unit 150 records the history information about the person without performing face matching on the person as the processing content, and determines that the person cannot pass.

  In this case, the history management unit 190 records the date and time when the person was detected, the determination result for the person, the image where the person was detected, and the like as the history information regarding the person who seems to be a suspicious person whose face could not be detected ( Step S109). At the same time, the passage control unit 180 rejects the passage of the person by closing the automatic door, the electronic lock, or the gate (step S110). At this time, the passage control unit 180 may output a warning such as “Cannot pass because face cannot be confirmed” or “Please do not hide the face” by the display unit 111a or a speaker (not shown). As a result, it is possible to urge the face to face the camera or prevent fraud.

  If the face is not detected from the image captured by the first camera by the above determination and the face is detected from the image captured by the second camera (person image) (YES in step S108), the classification is performed. Based on the classification shown in FIG. 4, unit 150 classifies the person as a suspicious person whose face was detected. In accordance with such classification, the classification unit 150 records the history information including the face image related to the person without processing the face collation for the person as processing contents, and prohibits the passage of the person. to decide.

  In this case, the history management unit 190 detects the person's face image, the date and time when the person was detected, the determination result for the person, and the person was detected as history information about the person who seems to be a suspicious person. An image or the like is recorded (step S111). At the same time, the passage control unit 180 rejects the passage of the person by closing the automatic door, the electronic lock, or the gate (step S110). At this time, the passage control unit 180 outputs a warning such as “Don't pass because the face cannot be confirmed (by the first camera)” or “Please do not hide the face” from the display unit 111a or a speaker (not shown). You may do it. As a result, it is possible to prompt the user to turn his face to the camera (first camera) or to prevent fraud.

  When a face is detected from the image captured by the first camera (step S107, YES), the classification unit 150 further determines the person (in the person detection unit 121) from the image captured by the second camera. It is determined whether or not a face of a person associated with the detected person is detected (step S112).

  When a face is detected from an image photographed by the first camera by these determinations, and a face of the same person is also detected from an image photographed by the second camera (person image) (step S112, YES), the classification unit 150 classifies the person as a passerby who knows the presence of the second camera based on the classification shown in FIG. In accordance with such classification, the classification unit 150 relaxes the authentication threshold used for face matching for the person as a processing content, rather than a predetermined value (predetermined authentication threshold) (step S113). It is assumed that face matching is performed using the relaxed threshold for authentication (step S114).

  In this case, the face matching unit 170 uses the authentication threshold value obtained by relaxing the predetermined value specified by the classifying unit 150, and the face feature information and the face feature detected by the face detecting unit 131 or the face detecting unit 132. A face matching process is performed to match the facial feature information of each registrant stored in the management unit 160 (step S114).

  Further, when the face is detected from the image taken by the first camera by the above determination, and the face of the same person is not detected from the image (person image) taken by the second camera (step) (S112, NO), the classification unit 150 classifies the person as a general passerby based on the classification shown in FIG. In accordance with such classification, the classification unit 150 sets, as processing details, a threshold for authentication used for face matching for the person as a predetermined threshold for authentication, and performs face matching using the predetermined threshold for authentication. It is assumed (step S114).

  In this case, the face collation unit 170 uses the predetermined authentication threshold specified by the classification unit 150 and the face feature information detected by the face detection unit 131 or the face detection unit 132 and the face feature management unit 160. A face matching process is performed to match the facial feature information of each registrant stored in (step S114).

  In the processing procedure shown in FIG. 5, when face matching is performed, the face of the passerby M is detected from at least an image photographed by the first camera. The first camera is installed so that it can be easily recognized by a passerby. For this reason, it is estimated that the first camera is more likely to capture an image including a good face image than the second camera. For this reason, here, it is assumed that face matching processing is performed using a face image (face feature information) detected from an image taken by the first camera.

  That is, the face collation unit 170 calculates the similarity between the face feature information detected by the face detection unit 131 and the face feature information of each registrant stored in the face feature management unit 160. When the similarity with each registrant is calculated, the face matching unit 170 selects the maximum similarity among the calculated similarities, and the threshold (classification) according to the classification determined by the classification unit 150 is selected. It is determined whether or not the threshold value adjusted by the unit 150 is greater than or equal to.

  Here, in the example illustrated in FIG. 4, when the face of the person is detected by the second camera, the classification unit 150 uses the authentication threshold obtained by relaxing the predetermined authentication threshold as the face matching for the person. It is set as the threshold value for authentication used for. When the face of the person is not detected by the second camera, the classification unit 150 sets a predetermined authentication threshold as an authentication threshold used for face matching for the person. That is, for the person whose face is detected from the image photographed by the first camera, the classification unit 150 uses the authentication for face collation depending on whether or not the face is detected from the image photographed by the second camera. The threshold value is adjusted.

  If it is determined by the above determination that the maximum similarity is equal to or greater than the threshold value for authentication, the face matching unit 170 determines that the person whose face is photographed by the first camera is the registrant who has the maximum similarity. . That is, when the maximum similarity is equal to or greater than the authentication threshold, the face matching unit 170 determines that the face matching is successful. If it is determined by the above determination that the maximum similarity is less than the threshold value for authentication, the face matching unit 170 determines that the person whose face has been photographed by the first camera is not any registrant. That is, when the maximum similarity is less than the threshold for authentication, the face matching unit 170 determines that face matching has failed.

  When the face collation in step S114 is successful (step S115, YES), the passage control unit 180 permits passage of the person by opening an automatic door, an electronic lock, a gate, or the like (step S116). At this time, the traffic control unit 180 may output a guidance indicating that traffic is permitted by the display unit 111a or a speaker (not shown).

  If the face collation in step S114 fails (step S115, NO), the passage control unit 180 rejects the passage of the person by closing the automatic door, electronic lock, gate, or the like (step S110). ). At this time, the passage control unit 180 may output a guidance indicating that “passage is not possible because it is not possible to confirm the registrant” from the display unit 111a or a speaker (not shown).

  As described above, in the passer-by identification device as the first embodiment, the face detection result from the image taken by the camera installed so that the passer-by is easily recognized and the passer-by recognition are not easily installed. The passers are classified based on the detection results of the face from the image taken by the camera, and processing corresponding to the classification is performed. Thereby, the traffic control process according to the action of the passerby estimated from the state of the person photographed with each camera becomes possible.

  Note that the first embodiment has been described on the assumption that the two types of image acquisition units 111 and 112 are installed one by one. However, the image acquisition unit (camera) may increase the number of installed units according to the shape of the passage or the operation mode. In this case, the person association unit 140 can cope with this by increasing the association processing according to the number of installed image acquisition units (cameras).

Next, a second embodiment will be described.
FIG. 6 schematically shows a configuration example of a person (passerby) recognition device 20 as an access control device according to the second embodiment.

  The passer-by identification device 20 shown in FIG. 6 functions as an access control device having a face matching function, like the passer-by identification device 10 described in the first embodiment. Moreover, the application example similar to the passer-by identification device 10 described in the first embodiment is also assumed for the operation mode of the passer-by identification device 20 shown in FIG.

  As shown in FIG. 6, the passer-by identification apparatus 20 includes a first image acquisition unit 211, a second image acquisition unit 212, a first person detection unit 221, a second person detection unit 222, and a first person. A face detection unit 231, a second face detection unit 232, a person association unit 240, a classification unit 250, an impersonation determination unit 255, a face feature management unit 260, a face matching unit 270, a traffic control unit 280, a history management unit 290, and the like Have.

  In addition, the 1st image acquisition part 211, the display part 211a, the 2nd image acquisition part 212, the acrylic board 212a, the 1st person detection part 221, the 2nd person detection part 222, the 1st face detection part 231, The second face detection unit 232, the person association unit 240, the classification unit 250, the face feature management unit 260, the traffic control unit 280, and the history management unit 290 are the first image acquisition described in the first embodiment. Unit 111, display unit 111 a, second image acquisition unit 112, acrylic plate 112 a, first person detection unit 121, second person detection unit 122, first face detection unit 131, and second face detection unit 132. The person association unit 140, the classification unit 150, the face feature management unit 160, the traffic control unit 180, and the history management unit 190 have the same functions, and thus detailed description thereof will be omitted.

  Similar to the first image acquisition unit 111, the first image acquisition unit 211 is installed so that the camera can be easily recognized by the passerby M. Similar to the second image acquisition unit 112, the second image acquisition unit 212 is installed so that the camera cannot be easily recognized by the passerby M. Further, with respect to the installation examples of the first image acquisition unit 211 and the second image acquisition unit 212, an installation example similar to that of the first image acquisition unit 111 and the second image acquisition unit 112 can be assumed.

  Next, the impersonation determination unit 255 will be described.

  The impersonation determination unit 255 determines whether a passerby is impersonating (determining whether impersonation is present). That is, the impersonation determination unit 255 determines whether or not the passerby associated with the person association unit 240 is impersonating. Various application methods are possible for the impersonation determination process by the impersonation determination unit 255. However, in this embodiment, the impersonation determination unit 255 determines impersonation based on a change between a face image photographed by the first camera and a face image of the same person photographed by the second camera. For this reason, in this embodiment, the impersonation determination unit 255 performs impersonation determination for a person whose face is photographed by both the first camera and the second camera.

  The face matching unit 270 is a face detected from the image acquired by the first image acquisition unit 211 (or the second image acquisition unit 212), similarly to the face matching unit 170 described in the first embodiment. And the feature information of the registrant's face stored in the face feature management unit 260 are collated. Further, the face collating unit 270 includes face feature information (hereinafter also referred to as first feature information) detected by the first face detecting unit 231 from the image acquired by the first image acquiring unit 211 and the second. The image acquisition unit 212 has a function of calculating the similarity with the face feature information detected by the second face detection unit 232 (hereinafter also referred to as second feature information) from the image acquired by the image acquisition unit 212. Note that the method described in the first embodiment is applied to the method of calculating the similarity between the first feature information and the second feature information by the face matching unit 270.

  Next, the impersonation determination process by the impersonation determination unit 255 will be described.

  For example, the impersonation determination unit 255 determines the presence / absence of impersonation by comparing the similarity between the first feature information and the second feature information with a predetermined impersonation determination threshold. For example, the impersonation determination unit 255 determines that there is impersonation if the similarity between the first feature information and the second feature information is less than a predetermined impersonation determination threshold. This assumes a case where a person who is unaware of the second camera is photographed in a state where the first camera has become another person. In such a case, the face photographed by the second camera and the face photographed by the first camera are significantly different (for example, the first feature information and the second feature information to the extent that they cannot be determined as the same person). It is estimated that the degree of similarity becomes lower. For this reason, the impersonation determination unit 255 determines whether or not impersonation is based on whether the similarity between the first feature information and the second feature information is less than a predetermined threshold (Ta). .

  In this case, the impersonation determination unit 255 causes the face matching unit 270 to calculate the similarity between the first feature information and the second feature information. As a result, the impersonation determination unit 255 determines whether or not impersonation occurs depending on whether the similarity between the first feature information and the second feature information calculated by the face matching unit 270 is less than a predetermined threshold (Ta). Determine.

  In the present embodiment, it is assumed that the first and second cameras capture the face of a passerby while walking, and that the second camera is not conscious of the passerby. For this reason, the first feature information and the second feature information may greatly fluctuate even for the same person (passer without impersonation) due to a change in the posture or expression of the passerby. Therefore, the threshold value Ta needs to be set so that the same person is not determined to be impersonated according to the operation mode of the passer-by identification device 20. For example, the threshold Ta is used to determine whether or not the facial feature information detected from the image acquired by the first image acquisition unit 211 and the facial feature information of the registrant are the same person. A value smaller than a predetermined threshold (relaxed threshold) is set.

  In addition, when the suspicious person is aware of the second camera, the face of the person who is impersonated like the first camera may be photographed on the second camera. In such a case, it is presumed that the similarity between the two becomes extremely high (substantially the same). In contrast, in the present embodiment, it is assumed that the first and second cameras capture the face of a walking passerby. There may be a situation in which there is a low possibility that the first feature information and the second feature information are substantially the same (the degree of similarity between them is extremely high). Assuming such a situation, a person who has extremely high similarity between the first feature information and the second feature information (for example, the similarity is equal to or greater than a predetermined threshold Tb) may be determined to be impersonated. good. In this case, the impersonation determination unit 255 may determine whether there is no impersonation depending on whether or not the threshold Ta is greater than or less than the threshold Tb in combination with the above-described conditions.

  Next, a method for classifying each person by the classification unit 150 and the impersonation determination unit 255 will be described.

  FIG. 7 is a diagram showing an example of the classification criteria for each person and the processing contents for each classification person.

  The classification example shown in FIG. 7 is the same as the classification example shown in FIG. 4 except that a classification corresponding to the presence or absence of impersonation is added. Therefore, the classification example shown in FIG. 7 is other than the case where a face is detected from both the image acquired by the first image acquisition unit 111 and the image acquired by the second image acquisition unit 112 and it is determined that there is impersonation. Is the same as the classification example shown in FIG.

  In other words, when it is determined that there is impersonation, the impersonation determination unit 255 does not collate with the registrant's face feature information stored in the face feature management unit 260 by the face collation unit 270, and performs the above traffic control. The unit 280 rejects the person's passage, and controls the history management unit 290 to record images determined to be spoofed (images taken by the first and second cameras). In this case, the impersonation determination unit 255 may further issue an alarm with a speaker (not shown) or notify the outside with a warning through a communication interface (not shown).

Next, an operation example of the passer-by identification device 20 according to the second embodiment will be described.
FIG. 8 is a flowchart for explaining an operation example of the passer-by identification device 20. In the operation shown in FIG. 8, it is assumed that a classification method as shown in FIG. 7 is set in advance.

  Note that steps S201 to S216 in the flowchart shown in FIG. 8 can be realized by the same processing as steps S101 to S116 in the flowchart shown in FIG. That is, the operation example shown in FIG. 8 is a process obtained by adding impersonation determination processing (step S221) to the operation example shown in FIG. Therefore, detailed description of the same processing as the step shown in FIG. 5 is omitted.

  In other words, when a face is detected from an image captured by the first camera and a face of a person determined to be the same person is detected from an image captured by the second camera (YES in S207 and S212). YES), the impersonation determination unit 255 performs an impersonation determination process based on the image captured by the first camera and the image captured by the second camera (step S221). In this impersonation determination process, as described above, face feature information (first feature information) detected from an image photographed by the first camera and a face detected from the image photographed by the second camera It is determined whether or not impersonation is based on whether or not the degree of similarity to the feature information (second feature information) is less than a predetermined threshold Ta for impersonation determination.

  For example, when the degree of similarity between the first feature information and the second feature information is greater than or equal to the threshold value Ta, the impersonation determination unit 255 determines that there is no impersonation. On the other hand, when the similarity between the first feature information and the second feature information is less than the threshold value Ta, the impersonation determination unit 255 determines that there is impersonation.

  When it is determined that there is no spoofing based on the above determination (step S221, NO), the face matching unit 270 uses the first feature information based on a threshold value obtained by relaxing the predetermined threshold value for authentication, as in steps S113 and S114. Collation processing (face collation processing) with the facial feature information of the registrant is performed (steps S213 and S214). In this case, the traffic control unit 280 performs traffic control for the person according to the result of the face matching process by the face matching unit 270 (steps S210, S215, and S216).

  When it is determined that there is impersonation by the above determination (step S221, YES), the history management unit 290 includes images determined as impersonation (each image acquired by the first and second image acquisition units). History information is recorded (step S211). In this case, the traffic control unit 280 rejects traffic for the person (step S210).

  As described above, in the passer-by identification apparatus 20 according to the second embodiment, the detection result of the face from the image taken by the camera installed so that the passer-by is easily recognized, and the passer-by is less likely to be recognized by the passer-by. A passer-by is classified based on a face detection result from an image photographed by a camera installed in the camera. At the same time, in the passer-by identification device 20, facial feature information detected from an image taken by a camera installed so as to be easily recognized by a passer-by, and an image taken by a camera installed so as not to be recognized by a passer-by. It is determined whether or not the passerby is impersonating by comparing with the facial feature information detected from. This makes it possible to perform traffic control processing according to the behavior of the passerby estimated from the state of the person photographed by each camera while eliminating impersonation.

  In addition, the passer-by identification device 20 described in the second embodiment may increase the number of installed image acquisition units (cameras) according to the shape of the passage or the operation mode. Also in this case, the impersonation determination unit 255 can determine whether or not the passerby is impersonating by comparing the facial feature information detected from the images acquired by the respective image acquisition units. is there.

Next, a third embodiment will be described.
FIG. 9 schematically shows a configuration example of a person (passerby) recognition apparatus 30 as a person monitoring apparatus according to the third embodiment.

  The passer-by identification device 30 shown in FIG. 9 has a face matching function, similar to the passer-by identification device 10 described in the first embodiment. In the third embodiment, it is assumed that the passer-by identification device 30 functions as a monitoring device that monitors the passer-by. For example, the passer-by identification device 30 shown in FIG. 9 is applied to a monitoring device that recognizes the face of a passer-by and notifies the recognition result to an external device or the like. However, the passer-by identification device 30 can be applied to the access control device in the operation mode as described in the first and second embodiments.

  As shown in FIG. 9, the passer-by identification device 30 includes a first image acquisition unit 311, a second image acquisition unit 312, a first person detection unit 321, a second person detection unit 322, Face detection unit 331, second face detection unit 332, person association unit 340, classification unit 350, face feature management unit 360, suspicious person list 361, important person (VIP) list 362, face search unit 370, output unit 380 And a history management unit 390 and the like.

  In addition, the 1st image acquisition part 311, the display part 311a, the 2nd image acquisition part 312, the acrylic board 312a, the 1st person detection part 321, the 2nd person detection part 322, the 1st face detection part 331, The second face detection unit 332, the person association unit 340, and the history management unit 390 are the first image acquisition unit 111, the display unit 111a, the second image acquisition unit 112, and the acrylic described in the first embodiment. Functions similar to those of the board 112a, the first person detection unit 121, the second person detection unit 122, the first face detection unit 131, the second face detection unit 132, the person association unit 140, and the history management unit 190 Since it has, detailed description shall be abbreviate | omitted.

  Similar to the first image acquisition unit 111, the first image acquisition unit 311 is installed so that the camera can be easily recognized by the passerby M. Similar to the second image acquisition unit 112, the second image acquisition unit 312 is installed so that the camera is difficult for the passerby M to recognize. The first image acquisition unit 311 and the second image acquisition unit 312 can be assumed to be installed similarly to the first image acquisition unit 111 and the second image acquisition unit 112. However, the first image acquisition unit 311 and the second image acquisition unit 312 are installed not only to photograph a person approaching the entrance, but also to photograph a face of a passerby passing near the entrance. Forms are also possible. An installation example of the first image acquisition unit 311 and the second image acquisition unit 312 will be described in detail later.

  Similar to the classification unit 150 described in the first embodiment, the classification unit 350 classifies each person based on the state (behavior pattern) of the person associated by the person association unit 340. For example, the classification unit 350 classifies each person into several patterns based on the face detection result by the face detection unit 331 and the face detection result by the face detection unit 332. A classification example as the third embodiment will be described in detail later.

  The face feature management unit 360 stores registrant information including face feature information of each person to be searched. Further, the face feature management unit 360 has a suspicious person list 361 and a VIP list 362. The suspicious person list 361 stores facial feature information of a person registered as a suspicious person. The VIP list 362 stores facial feature information of persons registered as important persons (VIP). However, the facial feature management unit 360 may distinguish the suspicious person and the VIP as attribute information for each person (facial feature information) without separating the suspicious person and the VIP in a list.

  The face search unit 370 searches the face feature management unit 360 for face feature information that has a maximum similarity to the passer's face feature information and that is equal to or greater than a search threshold. Here, the face search unit 370 determines one person who is most similar to the passerby depending on whether or not the similarity that maximizes the similarity with the feature information of the passer's face is equal to or greater than a search threshold. Shall be searched. However, the face search unit 370 may obtain all persons whose similarity is equal to or higher than the search threshold as search results.

  That is, the face search unit 370 resembles the face feature information detected by the first face detection unit 331 or the second face detection unit 332 and the feature information of each face stored in the face feature management unit 360. Calculate the degree. The face search unit 370 determines whether or not the maximum similarity among the calculated similarities is equal to or greater than a search threshold. When it is determined that the maximum similarity is equal to or greater than the search threshold, the face search unit 370 determines that the passerby is the person who has the maximum similarity (that is, the person who has the maximum similarity is the search result). And).

  Here, the search threshold value used in the search process by the face search unit 370 is a value adjusted according to the classification result by the classification unit 350. Further, a search threshold for the VIP list 362 (search threshold for similarity to the facial feature information stored in the VIP list 362) or a search threshold for the suspicious person list 361 (suspicious person list 361). The threshold value for search for the similarity to the facial feature information stored in (2) can be selectively adjusted. For example, when the search threshold for the VIP list 362 is relaxed, it is easier to search the VIP list 362 for a person who is determined to match the passerby. When the search threshold for the suspicious person list 361 is relaxed, it is easier to search the person who is determined to match the passerby from the suspicious person list 361.

  The output unit 380 functions as a communication interface that outputs information according to the search result by the face search unit 370 or the classification result by the classification unit 350 to an external device such as a monitoring device. In this case, the output unit 380 monitors image information such as images captured by the first and second cameras or audio information such as an alarm as information indicating the search result by the face search unit 370 or the classification result by the classification unit 350. Output to the device. Thereby, the administrator who is monitoring by the monitoring device can confirm the search result and the classification result in real time.

  The output unit 380 may display a search result or a warning on the display unit 311a, or may issue a warning through a speaker (not shown). Thereby, the passerby can recognize the search result and the classification result in real time.

  Next, an installation example of the first image acquisition unit 311 and the second image acquisition unit 312 as the third embodiment will be described.

  FIGS. 10 and 11 are diagrams illustrating an installation example of the first image acquisition unit 311 and the second image acquisition unit 312.

  FIG. 10 shows an installation example of the linear passage P3 in which the gate G3 is provided in the middle, and the first image acquisition unit 311 and the second image acquisition unit 312 in the passage P3. In the example shown in FIG. 10, the camera (first camera) of the first image acquisition unit 311 is provided in front of the gate G3 in the a direction indicated by the dotted arrow in FIG. The camera (second camera) of the acquisition unit 312 is provided at a location past the gate G3.

  That is, in the example shown in FIG. 10, the first camera is installed so as to photograph the face of a passerby approaching the gate G3. In addition, a display unit 311a for displaying guidance for passers-by is provided in the vicinity of the first camera. As a result, a passerby approaching the gate G3 is aware of the first camera or visually observes the guidance displayed on the display unit 311a. That is, the first camera can easily photograph the face of a passerby approaching the gate G3 while being conscious of the first camera.

  The second camera is installed so as to photograph the face of a passerby who has passed through the gate G3. The second camera is hidden by the acrylic plate 312a. Thereby, the passerby who passed gate G3 passes passage P3, without being conscious of the 2nd camera. That is, a passerby who is not conscious of the second camera is photographed by the second camera.

  FIG. 11 shows a linear path P4 in which a gate G4 is provided in the middle, and an installation example of the first image acquisition unit 311 and the second image acquisition unit 312 in the path P4.

  In the example shown in FIG. 11, the camera (first camera) of the first image acquisition unit 311 is provided in front of the gate G4 with respect to the direction a shown in FIG. The (second camera) is provided in front of the gate G4 in the b direction opposite to the a direction shown in FIG.

  That is, the first camera is installed so as to photograph the face of a passerby approaching the gate G4 from the direction a. In addition, a display unit 311a for displaying guidance for passers-by is provided in the vicinity of the first camera. As a result, a passerby who approaches the gate G4 from the direction a is aware of the first camera or visually observes the guidance displayed on the display unit 311a. In other words, the first camera can easily photograph a passerby approaching from the a direction toward the gate G4 while being conscious of the first camera.

  The second camera is installed so as to photograph the face of a passerby approaching the gate G4 from the b direction opposite to the a direction. The second camera is hidden by the acrylic plate 312a. Thereby, the passerby who approaches the gate G4 from the direction b passes through the passage P4 without being aware of the second camera. That is, a passerby who is approaching the gate G4 from the direction opposite to the direction a and is not aware of the second camera is photographed by the second camera.

  As in the installation example described above, in the passer-by identification apparatus 30 as the third embodiment, the first camera and the second camera can be installed by various installation methods depending on the operation mode and the like. That is, since the passer-by identification device 30 is supposed to monitor a passer-by, the first and second cameras can be installed at various locations as long as the passer-by can be photographed.

Next, a method for classifying each person by the classification unit 350 will be described.
As in the first embodiment, the classification unit 350 classifies each person based on the face detection results by the first face detection unit 331 and the second face detection unit 332, for example. How to classify each person and what kind of processing to perform according to the classification result is set appropriately according to the installation status of the passer-by identification device, the overall system operation mode, security policy, etc. It is what is done. That is, the classification unit 350 classifies each person according to a preset classification standard.

FIG. 12 is a diagram illustrating an example of classification criteria for each person by the classification unit 350 according to the third embodiment.
Here, the camera of the first image acquisition unit 311 (hereinafter also referred to as the first camera) is installed so as to be easily recognized by passers-by, and the camera of the second image acquisition unit 312 (hereinafter also referred to as the second camera). ) Is installed in such a way that it is difficult for a passer-by to recognize it.

  According to “No. 3” and “No. 4” shown in FIG. 12, when a human face cannot be detected from the image acquired by the first image acquisition unit 311, that is, detected by the human detection unit 321. If the face detection unit 331 cannot detect the face of the person (first camera image: face detection NG), the classification unit 350 classifies the person as a person who seems to be a suspicious person. Such classification is based on the assumption that the person who intentionally prevents the first camera from taking a face is a person who seems to be a suspicious person.

  In addition, according to “No. 4” illustrated in FIG. 12, a human face detected from the image acquired by the first image acquisition unit 311 cannot be detected, and the human face acquired by the second image acquisition unit 312 is acquired. When the face of the person associated with the person detected from the image cannot be detected (first camera image: face detection NG and second camera image: face detection NG), the classification unit 350 The person is classified as a person who seems to be a suspicious person whose face could not be detected at all.

  In this case, the classification unit 350 causes the output unit 380 to output an alarm to the monitoring device, and causes the history management unit 390 to record history information including an image of a person determined to be a suspicious person. The history information recorded in the history management unit 390 is, for example, a determination result, a determination time, an image in which the person is detected, and the like. For a person whose face could not be detected as described above, if a face image can be acquired later, it can be registered in the suspicious person list together with facial feature information obtained from the face image.

  Further, according to “No. 3” illustrated in FIG. 12, a human face cannot be detected from the image acquired by the first image acquisition unit 311, and the image acquired by the second image acquisition unit 312 When the face of the person associated with the person can be detected (first camera image: face detection NG and second camera image: face detection OK), the classification unit 350 detects the face of the person. Classify the person as a suspicious person.

  In this case, face search based on face feature information obtained from the face image detected from the image acquired by the second image acquisition unit 312 is possible. For this reason, in the example illustrated in FIG. 12, the processing content is set to focus on the suspicious person list 361 and output the search result. That is, in the case of “No. 3” illustrated in FIG. 12, the classification unit 350 sets the search threshold for the suspicious person list 361 based on a predetermined search threshold in order to perform a face search mainly on the suspicious person list 361. Is also made smaller (the threshold is relaxed). Thereby, in the face search process by the face search unit 370, the suspicious person stored in the suspicious person list 361 is easily extracted as the search result. In the case of “No. 3” illustrated in FIG. 12, the classification unit 350 may make the search threshold for the VIP list 362 larger (stricter threshold) than the predetermined search threshold. In this case, it is difficult to extract VIPs stored in the VIP list 362 as search results.

  Further, as a method of searching for a face with emphasis on the suspicious person list 361, only the face feature information stored in the suspicious person list 361 may be searched as a search target. Furthermore, also in this case, the classification unit 350 may make the search threshold for the suspicious person list 361 smaller (relax the threshold) than the predetermined search threshold.

  Further, according to “No. 1” and “No. 2” shown in FIG. 12, when a human face detected from the image acquired by the first image acquisition unit 311 can be detected, that is, the person detection unit When the face detected by the H.321 can be detected by the face detection unit 331 (first camera image: face detection OK), the classification unit 350 is a person who seems to be a heavy person or a general passerby. Classify as

  In addition, according to “No. 2” illustrated in FIG. 12, a human face detected from the image acquired by the first image acquisition unit 311 can be detected, and the image acquired by the second image acquisition unit 312 can be detected. When the face of the person associated with the person detected from the above cannot be detected (first camera image: face detection OK and second camera image: face detection NG), the classification unit 350 The person is classified as a person who seems to be a general passerby. In this classification, a general passerby who does not intend to hide the face is photographed with the first camera but is not photographed with the second camera unless the installation position of the second camera is informed. This is based on the assumption that it is difficult to do this (or that it is uncertain that the face is shot).

  In this case, face search based on facial feature information obtained from the face image detected from the image acquired by the first image acquisition unit 311 is possible. For this reason, in the example shown in FIG. 12, as the processing content, each list (suspicious person list 361 and VIP list 362) is searched for a face, and the search result is output. That is, in the case of “No. 2” illustrated in FIG. 12, the classification unit 350 is obtained from the image acquired by the first image acquisition unit 311 using the feature information of each face stored in each list as a search target. The face search unit 370 searches for a face whose similarity with the facial feature information is greater than or equal to a predetermined search threshold.

  According to “No. 1” illustrated in FIG. 12, a human face detected from the image acquired by the first image acquisition unit 311 can be detected, and also from the image acquired by the second image acquisition unit 312. When the face of the person can be detected (first camera image: face detection OK and second camera image: face detection OK), the classification unit 350 knows the position of the second camera. It is classified as a person who seems to be an important person (VIP).

  Here, it is assumed that the important person is previously informed of the position of the second camera. That is, in the above classification, an important person who does not intend to hide the face and is informed of the installation position of the second camera is likely to photograph the face with the first camera and the second camera. Based on the guess.

  In this case, in the example shown in FIG. 12, the face search is focused on the VIP list 362 as the processing content, and the search result is output. That is, in the case of “No. 1” illustrated in FIG. 12, the classification unit 350 makes the search threshold for the VIP list 362 smaller than the predetermined search threshold in order to focus face search on the VIP list 362. (Reducing the threshold). Thereby, in the face search process by the face search unit 370, an important person stored in the VIP list 362 is easily extracted as a search result.

  In the case of “No. 1” shown in FIG. 12, the classification unit 350 may make the search threshold for the suspicious person list 361 larger (stricter threshold) than the predetermined search threshold. In this case, as a result of the face search, it is difficult to extract suspicious persons stored in the suspicious person list 361.

  In addition, as a method of searching for a face focusing on the VIP list 362, only the face feature information stored in the VIP list 362 may be searched as a search target. Furthermore, also in this case, the classification unit 350 may make the search threshold used for the face search for the VIP list 362 smaller than the predetermined search threshold (relax the threshold).

  As described above, in the example illustrated in FIG. 12, the face search process for the facial feature information detected from the image captured by the first camera or the second camera is performed from the image captured by the first camera. Based on the detection result of the face and the detection result of the face from the image taken by the second camera, a list to be intensively searched (for example, search by reducing the search threshold) is selected. ing. That is, in the example shown in FIG. 12, search for each list is performed based on the face detection result from the image captured by the first camera and the face detection result from the image captured by the second camera. It is set to adjust the threshold value. Thereby, efficient face search processing and person monitoring processing according to the state of the passerby (the direction of the face toward the first and second cameras) can be realized.

  Note that the above settings (classification method for each person) can be set as appropriate according to the system operation mode, the installation status of the second camera, the secrecy of the second camera, and the like. This is because it is presumed that the state of the passerby photographed by the first and second cameras differs depending on the operation mode of the passer's system, the installation status of the second camera, the secrecy of the second camera, and the like. It is. For example, whether the second camera is in a state where it is easy to capture the face of a passerby in a general walking state, or whether the second camera is easily found by a passer-by, or the like, It is estimated that the classification for passers-by whose faces are photographed by the camera is different.

Next, an operation example of the passer-by identification device 30 as the third embodiment will be described.
FIG. 13 is a flowchart for explaining an operation example of the passer-by identification device 30. In the operation illustrated in FIG. 13, it is assumed that the classification and processing content illustrated in FIG. 12 are set in advance. Note that steps S301 to S308 and S312 of the flowchart shown in FIG. 13 can be realized by the same processing as steps S101 to S108 and S112 of the flowchart shown in FIG.

  That is, when a face cannot be detected from images (person images) taken by the first camera and the second camera (NO in step S307 and NO in step S308), the classification unit 350 performs the process shown in FIG. The person is classified as a person who seems to be a suspicious person whose face could not be detected. In response to such classification, the classification unit 350 determines that the processing content records history information about the person and outputs an alarm to the monitoring device that a person who seems to be a suspicious person whose face cannot be detected is detected. To do.

  In this case, the history management unit 390 records the date and time when the person was detected, the determination result for the person, the image where the person was detected, and the like as the history information regarding the person who seems to be a suspicious person whose face could not be detected ( Step S309). At the same time, the output unit 380 outputs a warning to the monitoring device that a suspicious person whose face cannot be detected has been detected (step S310). At this time, the output unit 380 may output a warning such as “Cannot confirm face” or “Do not hide face” from the display unit 311a or a speaker (not shown).

  Further, when a face is not detected from the image captured by the first camera and a face is detected from an image (person image) captured by the second camera (NO in step S307 and step S308). Based on the classification shown in FIG. 12, the classification unit 350 classifies the person as a person who seems to be a suspicious person whose face could be detected. In accordance with such classification, the classification unit 350 determines that the processing content is focused on the suspicious person list 361 and the search result is output. Here, the classification unit 350 relaxes a search threshold for the suspicious person list 361 in order to perform a face search mainly on the suspicious person list 361 (step S311). In this case, the classification unit 350 may tighten the search threshold for the VIP list 362. In addition, the classification unit 350 may search only the suspicious person list 361.

  In this case, the face search unit 370 performs similarity between the face feature information detected from the image acquired by the second image acquisition unit 312 and the face feature information stored in the suspicious person list 361 as the face search process. Each similarity is compared with the relaxed search threshold (step S314). As a result, the face search unit 370 supplies the suspicious person whose similarity is equal to or higher than the search threshold with the similarity reduced to the output unit 380 as a search result.

  The output unit 380 that has received the search result of such face search processing outputs the search result to the monitoring device (step S316). This search result includes, for example, an image in which the person's face is detected, information indicating a suspicious person whose similarity is less than or equal to a search threshold, and the like. At this time, the output unit 380 may output a warning such as “Can't confirm face with (first camera)” or “Do not hide face” by display unit 311a or a speaker (not shown).

  When a face is detected from the image captured by the first camera and a face of the same person is detected from an image (person image) captured by the second camera (YES in step S307), and Based on the classification shown in FIG. 12, the classification unit 350 classifies the person as an important person who knows the presence of the second camera. In accordance with such classification, the classification unit 350 determines that the processing content is a face search focusing on the VIP list 362 and outputs the search result. Here, the classification unit 350 relaxes a search threshold for the VIP list 362 in order to search for a face in the VIP list 362 (step S313). In this case, the classification unit 350 may tighten the search threshold for the suspicious person list 361. Further, the classification unit 350 may search only the VIP list 362.

  In this case, the face search unit 370 stores the face feature information detected from the image acquired by the first image acquisition unit 311 or the second image acquisition unit 312 and the VIP list 362 as the face search process. The similarity with the facial feature information is compared, and each similarity is compared with the relaxed search threshold (step S314). As a result, the face search unit 370 supplies, to the output unit 380, a VIP whose search level is equal to or higher than the search threshold with the similarity reduced.

  The output unit 380 that has received the search result of such face search processing outputs the search result to the monitoring device (step S316). This search result includes, for example, an image in which the face of the person is detected, information indicating a VIP whose similarity is less than or equal to a search threshold value, and the like. At this time, the output unit 380 outputs guidance such as “a face can be confirmed (by the first camera)” or “guidance information corresponding to a VIP that is equal to or greater than the threshold” by the display unit 311a or a speaker (not shown). You may do it.

  Further, when a face is detected from the image captured by the first camera and a face of the same person is not detected from the image captured by the second camera (person image) (in step S307). YES and NO in step S312), classification unit 350 classifies the person as a general passer-by based on the classification shown in FIG. In accordance with such classification, the classification unit 350 determines that the processing content is a face search for each list (suspicious person list 361 and VIP list 362) and outputs the search result.

  In this case, as the face search process, the face search unit 370 calculates the similarity between the face feature information detected from the image acquired by the first image acquisition unit 311 and the face feature information stored in each list. Each similarity is calculated and compared with a predetermined search threshold (step S314). Thereby, the face search unit 370 supplies a person whose similarity is equal to or higher than the search threshold to the output unit 380 as a search result.

  The output unit 380 that has received the search result of such face search processing outputs the search result to the monitoring device (step S316). This search result includes, for example, an image in which the face of the person is detected, information indicating a person whose similarity is equal to or higher than a predetermined search threshold, and the like. At this time, the output unit 380 outputs guidance such as “the face can be confirmed (by the first camera)” or “information indicating a person who has exceeded the threshold” by the display unit 311a or a speaker (not shown). May be. However, when a suspicious person stored in the suspicious person list 361 is detected as a search result, the output unit 380 may issue an alarm or the like to urge the person to be alert.

  As described above, in the passer-by identification apparatus as the third embodiment, the face detection result from the image taken by the camera installed so that the passer-by is easily recognized and the passer-by identification are not easily recognized. A passer is classified based on a face detection result from an image photographed by the camera, and face search processing or monitoring processing is performed according to the classification. Thereby, an efficient person monitoring process according to a passer's action presumed from the state of the person imaged with each camera is attained.

  Note that the third embodiment has been described on the assumption that the two types of image acquisition units 111 and 112 are installed one by one. However, the image acquisition unit (camera) may increase the number of installed units according to the situation of the area to be monitored or the operation mode. In this case, the person association unit 340 can cope with this by increasing the association processing according to the number of installed image acquisition units (cameras).

1 is a block diagram schematically showing a configuration example of a passer-by identification device according to a first embodiment of the present invention. The schematic diagram for demonstrating the example of installation of the 1st image acquisition part and 2nd image acquisition part as 1st Embodiment. The schematic diagram for demonstrating the example of installation of the 1st image acquisition part and 2nd image acquisition part as 1st Embodiment. The figure explaining the classification example of the passerby which concerns on 1st Embodiment. The flowchart for demonstrating the example of the process sequence in the passer-by identification apparatus which concerns on 1st Embodiment. The block diagram which shows roughly the structural example of the passer-by identification apparatus which concerns on 2nd Embodiment. The figure explaining the classification example of the passerby which concerns on 2nd Embodiment. The flowchart for demonstrating the example of the process sequence in the passer-by identification apparatus which concerns on 2nd Embodiment. The block diagram which shows roughly the structural example of the passer-by identification apparatus which concerns on 3rd Embodiment. The schematic diagram for demonstrating the example of installation of the 1st image acquisition part as a 3rd embodiment, and a 2nd image acquisition part. The schematic diagram for demonstrating the example of installation of the 1st image acquisition part as a 3rd embodiment, and a 2nd image acquisition part. The figure explaining the classification example of the passerby which concerns on 3rd Embodiment. The flowchart for demonstrating the example of the process sequence in the passer-by identification apparatus which concerns on 3rd Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10, 20, 30 ... Person recognition apparatus (passerby recognition apparatus), 111, 211, 311 ... 1st image acquisition part, 111a, 211a, 311a ... Display part, 112, 212, 312 ... 2nd image acquisition part 112a, 212a, 312a ... acrylic plates, 121, 221, "321 ... first person detection unit, 122, 222, 322 ... second person detection unit, 131, 231, 331 ... first face detection unit, 132, 232, 332 ... second face detection unit, 140a ... auxiliary detection unit, 150, 250, 350 ... classification unit, 160, 260, 360 ... face feature management unit, 170, 270 ... face collation unit, 180, 280 ... traffic control unit, 190, 290 ... history management unit, 255 ... determination unit, 361 ... suspicious person list, 362 ... important person list, 370 ... face search unit, 380 ... output unit, 390 ... history management .

Claims (20)

A person recognition device for recognizing a person from a face image,
A first image acquisition unit that acquires an image captured by a first camera installed in a state that is easily recognized by a person;
A second image acquisition unit for acquiring an image taken by a second camera installed in a state that is difficult to be recognized by a person;
A first face detection unit for detecting a human face from the image acquired by the first image acquisition unit;
A second face detection unit for detecting the face of the person from the image acquired by the second image acquisition unit;
An associating unit for associating a person photographed by the first camera with a person photographed by the second camera;
A classification unit that classifies the person based on a result of association by the association unit, a face detection result by the first face detection unit, and a face detection result by the second face detection unit;
Have The classification unit determines whether the person is a suspicious person;
The person recognition apparatus according to claim 1, wherein: The classification unit determines whether the person is a suspicious person depending on whether the face of the person associated with the person photographed by the second camera by the association unit is detected by the first face detection unit. Determine whether or not
The person recognition apparatus according to claim 2, wherein: Furthermore, a memory for storing the registrant's face image,
When the classification unit determines that the person is not a suspicious person, the face image of the registrant stored in the memory and the face image of the person detected by the first face detection unit or the A face matching unit that performs face matching based on whether the similarity with the face image of the person detected by the second face detection unit is equal to or higher than a threshold for authentication,
The person recognition apparatus according to claim 2, wherein: Furthermore, a memory for storing the registrant's face image,
Face matching is performed based on whether the similarity between the face image stored in the memory and the face image detected by the first face detection unit or the second face detection unit is greater than or equal to an authentication threshold. A face matching unit to perform,
The classification unit is further adapted to classify the person based on the association result by the association unit, the face detection result by the first face detection unit, and the face detection result by the second face detection unit. Adjusting a threshold for authentication used for face matching by the face matching unit.
The person recognition apparatus according to claim 1, wherein: The classification unit relaxes an authentication threshold for the person when the face of the person associated by the association unit is detected by the first face detection unit and the second face detection unit,
The person recognition apparatus according to claim 5, wherein: Further, the person impersonates the person associated by the association unit based on the face image detected by the first face detection unit and the face image detected by the second face detection unit. An impersonation determination unit that determines whether or not
The person recognition apparatus according to claim 1, wherein: The impersonation determination unit is configured such that the face image of the person detected by the first face detection unit associated with the association unit and the face image of the person detected by the second face detection unit are the same person It is determined that the person is impersonating when it is determined that
The person recognition apparatus according to claim 7, wherein: Further, a memory for storing each registrant's face image and information indicating the classification in association with each other,
A face search unit that searches a face image of a registrant whose similarity with the face image detected by the first face detection unit or the second face detection unit is a search threshold or more from the memory. Have
The classification unit according to the classification of the person based on a result of association by the association unit, a face detection result by the first face detection unit, and a face detection result by the second face detection unit Adjusting a search threshold used for search by the face search unit;
The person recognition apparatus according to claim 1, wherein: The memory stores a suspicious person's face image,
The classification unit searches for the suspicious person's face image when the face of the person detected by the second face detection unit associated by the association unit is not detected by the first face detection unit. Relax the threshold for
The face search unit searches for a face image of a suspicious person whose similarity with the face image detected by the second face detection unit is equal to or higher than a search threshold relaxed by the classification unit;
The person recognition apparatus according to claim 9, wherein: The memory stores a face image of a specific person,
The classification unit sets a search threshold for the face image of the specific person when the face of the person associated by the association unit is detected by the first face detection unit and the second face detection unit. Relax,
The face search unit includes a face of a specific person whose similarity to the face image detected by the first face detection unit or the second face detection unit is equal to or more than a search threshold that is relaxed by the classification unit. Search for images,
The person recognition apparatus according to claim 9, wherein: Further, a memory for storing each registrant's face image and information indicating the classification in association with each other,
Using the face image of a registrant whose classification matches the classification of the person by the classification unit as a search target, the similarity with the face image detected by the first face detection unit or the second face detection unit is searched. A face search unit for searching for a registrant's face image that is equal to or greater than a threshold for use.
The person recognition apparatus according to claim 1, wherein: The memory stores a suspicious person's face image,
The classification unit sets the person as a suspicious person when the face of the person detected by the second face detection unit associated with the association unit is not detected by the first face detection unit. Classify and
When the face is not detected by the first face detection unit, the face search unit sets the similarity to the face image detected by the second face detection unit with the face image of the suspicious person as a search target. Search for face images that are greater than or equal to the search threshold,
The person recognition apparatus according to claim 12, wherein: The memory stores a face image of a specific person,
The classification unit classifies the person as a specific person when the face of the person associated by the association unit is detected by the first face detection unit and the second face detection unit,
The face search unit searches a face image of a specific person stored in the memory as a search target, and searches for a similarity to the face image detected by the first face detection unit or the second face detection unit. Search for face images that are above the threshold for
The person recognition apparatus according to claim 12, wherein: A person recognition method used in a person recognition device for recognizing a person from a face image,
Obtain an image taken with a first camera installed in a state that is easily recognized by a person,
Acquire an image taken with a second camera installed in a state that is difficult for humans to recognize,
Detecting a human face from an image taken by the first camera;
Detecting the face of the person from an image taken by the second camera;
Associating a person photographed by the first camera with a person photographed by the second camera;
Classifying the person based on a result of the association, a face detection result from an image taken by the first camera, and a face detection result from an image taken by the second camera;
A person recognition method characterized by the above. The classification is based on whether or not the face of a person detected from an image captured by the second camera associated with the association is detected from an image captured by the first camera. Whether to classify as a suspicious person,
The person recognition method according to claim 15, wherein: Furthermore, according to the classification of the person based on the result of the association, the detection result of the face from the image captured by the first camera, and the detection result of the face from the image captured by the second camera Adjust the threshold for authentication used for face matching,
The degree of similarity between the face image of the registrant stored in the memory and the face image detected from the image taken with the first camera or the image taken with the second camera Face matching is performed based on whether or not is equal to or more than the adjusted authentication threshold value.
The person recognition method according to claim 15, wherein: Furthermore, the person's face image detected from the image photographed by the first camera associated with the association and the person's face image detected from the image photographed by the second camera. To determine whether the person is impersonating,
The person recognition method according to claim 15, wherein: Further, for search used for face search according to the classification of the person based on the association result, the face detection result by the first face detection unit, and the face detection result by the second face detection unit Adjust the threshold of
A face image of a registrant whose similarity with a face image detected from an image taken by the first camera or a face image detected from an image taken by the second camera is equal to or greater than a search threshold. Is searched from a memory in which each registrant's face image and information indicating the classification are stored in association with each other.
The person recognition method according to claim 15, wherein: Further, a face image of a registrant whose classification matches the classification of the person according to the classification is used as a search target, and a face image detected from an image photographed by the first camera or photographed by the second camera. Search for a registrant's face image whose similarity with the detected face image is greater than or equal to the search threshold from a memory in which each registrant's face image is associated with information indicating the classification. To
The person recognition method according to claim 15, wherein:

Images