JP2007249953A - Person identification device and person identification method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a person identification device and person identification method which can efficiently detect a group composed of a plurality of persons and has high convenience or security property. <P>SOLUTION: The person identification device obtains information including biometric information of a person, detects the biometric information of at least one person from the obtained information, collates each detected biometric information with the biometric information of at least one registrant associated with group information and stored in a storage unit to thereby identify the person having the biometric information detected from the obtained information, classifies a plurality of successively identified persons into group candidates based on predetermined conditions, divides the respective persons of the group candidates into groups based on the group information of each person stored in the storage unit, and outputs a grouping result to an external device. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention acquires, for example, biological information such as a person's face, retina, mouth, iris, fingerprint, palm print, ear, and vein shape of a part of a human body, and at least one biological information registered in advance The present invention relates to a person recognition device and a person recognition method for collation.

  2. Description of the Related Art Conventionally, for example, in a person recognition device, biological information such as a recognition target person's face, retina, mouth, iris, fingerprint, palm print, ear, and vein shape of a part of a human body is acquired, and the acquired biological information is registered in advance. It is determined whether or not the recognition target person is a person (registrant) registered in advance by collating with at least one or more biometric information. In addition, an access control system that manages entry / exit to a security area or access to a specific device, or a person monitoring system that monitors a person, has developed a control that performs control according to the determination result by the person recognition device. Has been. The person recognition method used for the person recognition device includes a method of collating biometric information acquired from a person who is in a stationary state with biometric information registered in advance, or a person who is recognized in an operation state such as walking. There are various application examples such as a method of collating acquired biometric information with biometric information registered in advance.

For example, in a passer-by identification apparatus that collates biometric information acquired from a recognition target person (passerby) who is passing a passage or the like with biometric information registered in advance, recognition processing is performed on the recognition target person. The recognition process can be executed without making the user aware of the situation. In such a passer-by recognition apparatus that recognizes a passer-by, it is assumed that a plurality of passers-by who are walking continuously are recognized. In such a case, the conventional person recognition apparatus performs recognition processing for each person, even for a plurality of persons walking continuously. Conventionally, a technique for measuring the number of consecutive persons has been disclosed in order to prevent companion (for example, Patent Document 1).
Japanese Patent No. 3617373 JP 2001-256696 A JP 2004-362283 A

  An object of one aspect of the present invention is to provide a highly convenient person recognition device and person recognition method that can efficiently determine a group of a plurality of persons.

  A person recognition apparatus according to an aspect of the present invention recognizes a person and stores at least one biometric information of a registrant and group information indicating a group of each registrant in association with each other. Information acquisition means for acquiring information including biological information of a person, detection means for detecting biological information for at least one person from the information acquired by the information acquisition means, and each biological body detected by the detection means Recognizing means for recognizing a person of the biometric information detected by the detecting means by collating information with each biometric information stored in the storage means, and a plurality of persons sequentially recognized by the recognizing means Are classified into group candidates based on the above conditions, and each person of the group candidate is grouped based on the group information of each person stored in the storage means It has a loop determination means, and output means for outputting a result of the grouping by the group judgment unit to an external device.

  A person recognition apparatus according to an aspect of the present invention recognizes a person and stores at least one biometric information of a registrant and group information indicating a group of each registrant in association with each other. And, for each of a plurality of monitoring areas in one area, information acquisition means for sequentially acquiring information including biological information of a person existing in each monitoring area, and at least from each information acquired by the information acquisition means Detection means for detecting biological information for one person, and biological information detected by the detection means by collating each biological information detected by the detection means with each biological information stored in the storage means Recognition means for recognizing a person, information on each monitoring area acquired by the information acquisition means, and the recognition means for the biological information detected by the detection means from the information Based on the recognition result by, has a group determining means for determining a group behavior of a plurality of persons in the area, and output means for outputting a determination result by the group judgment unit to an external device.

  A person recognition method according to an aspect of the present invention is a method used in a person recognition apparatus for recognizing a person, acquiring information including biological information of the person, and obtaining at least one biological information from the acquired information. The detected biometric information is collated with the biometric information of at least one registrant stored in the storage unit in association with the group information, thereby detecting the biometric information detected from the acquired information. And sequentially classifying a plurality of persons recognized as group candidates based on a predetermined condition, and grouping the persons of the group candidates based on group information of each person stored in the storage means The grouping result is output to an external device.

  A person recognition method according to an aspect of the present invention is a method used in a person recognition apparatus for recognizing a person, and for each of a plurality of monitoring areas in one area, a living body of a person existing in each monitoring area. Information including information is sequentially acquired, biometric information for at least one person is detected from each of the acquired information, and at least one person stored in the storage unit in association with each detected biometric information and group information The person of each biometric information detected from the acquired information is recognized by collating with the biometric information of the registrant, and the information of each acquired monitoring area and the recognition result for the biometric information detected from the information Based on the above, the group behavior of a plurality of persons in the area is determined, and the determination result of the group behavior is output to an external device.

  An object of one aspect of the present invention is to provide a highly convenient person recognition device and person recognition method that can efficiently determine a group of a plurality of persons.

Hereinafter, embodiments according to the present invention will be described with reference to the drawings.
In each of the following embodiments, the recognition target is obtained by collating the facial feature information obtained from the facial image (biological information) of the recognition subject with at least one registered facial feature information. A person recognition device for recognizing a person will be described. The person recognition apparatus of each embodiment described below can be applied to a method for recognizing a person using various biological information other than the face. For example, a method for recognizing a person based on biological information such as a retina, mouth, iris, fingerprint, palm print, ear, vein shape of a part of a human body (such as a hand or a finger) is applied to a person recognition device. Is possible.

  In each embodiment, a passer-by recognition apparatus as a person recognition apparatus that recognizes a person (passer-by) who passes through a specific place will be described. The passer-by identification apparatus according to each embodiment is, for example, an entrance / exit management system that manages entry / exit of a person to / from an area (security area) in which only a specific person is permitted to enter / exit, access to a specific device, etc. An access control system for controlling a person, a person monitoring system for a supervisor to monitor a person passing through a specific place, or a monitoring video recording system for storing a monitoring video in a storage device based on a specific condition It is assumed that it will be applied.

  In addition, it is assumed that the passer-by identification apparatus according to each embodiment is applied to a system that performs entrance / exit management or monitoring of a person in a place where there are very many passers-by as recognition targets. For example, the passer-by identification device according to each embodiment has an entrance / exit management system that manages passers-by entering and exiting the entrance of a building or a company building, or a large number of passers-by such as commercial facilities, entertainment facilities, or transportation facilities. It is assumed that the present invention is applied to a person monitoring system at an entry / exit place.

First, the first embodiment will be described.
FIG. 1 schematically shows a configuration example of a passer-by recognition system 100 including a passer-by recognition device 102 as a person recognition device according to the first embodiment. FIG. 1 shows a configuration example when the passer-by identification system 100 is operated as an entrance / exit management system for controlling gate passage (entrance / exit) and a person monitoring system.
As shown in FIG. 1, the passer-by identification system 100 includes a camera 101, a passer-by identification device 102, a gate device 103, a monitoring device 104, and the like.

  The camera 101 is a device for acquiring information (image) including a face image as biological information. The camera 101 captures an image of a specific place (area to be monitored). The camera 101 captures a face image (an image including at least a face) of a passerby M who wants to pass through the gate device 103. The camera 101 is a camera that captures moving images (continuous images at a predetermined frame rate). For example, the camera 101 is configured by a television camera or the like with an imaging element such as a CCD sensor. The camera 101 supplies the captured image to the passer-by identification device 102. In addition, as an apparatus for acquiring information including biological information, an information input apparatus such as a scanner may be used instead of a camera.

  Further, depending on the operation mode of the passer-by identification system 100, a sensor 101a for detecting a person may be installed in the vicinity of the camera 101 as an auxiliary means for detecting a passer-by. As the sensor 101a installed in the vicinity of the camera 101, for example, a distance sensor for detecting the distance from the person photographed by the camera 101 to the camera 101, and the moving speed of the person photographed by the camera 101 are used. There are speed measuring instruments to measure. Information detected by the sensor as described above may be supplied to the passer-by identification device 102 as auxiliary information for an image captured by the camera 101.

  The passer-by identification device 102 recognizes a person in a face image detected from an image captured by the camera 101. Further, the passer-by identification device 102 outputs a person recognition result to the gate device 103. The passer-by identification apparatus 102 acquires (inputs) an image taken by the camera 101, extracts facial feature information from the input image, and stores (registers) the extracted facial feature information in advance in a storage device. It matches with the registrant's facial feature information. In addition, the passer-by identification device 102 sequentially performs group determination on a person who is recognized based on a collation result for an image input from the camera 101 and a predetermined condition. The passer-by identification device 102 outputs the comparison result and the group determination result as described above to the gate device 103 and the monitoring device 104.

  The gate device 103 and the monitoring device 104 are external devices that operate in accordance with the output of processing results and the like by the passer-by identification device 102. The gate device 103 or the monitoring device 104 is installed according to the operation mode of the system. The gate device 103 or the monitoring device 104 may be omitted depending on the operation mode of the system.

  The gate device 103 is a device for controlling the passage of a person (passerby). The gate device 103 includes, for example, a gate (not shown), a control mechanism (not shown) for controlling opening and closing of the gate, and an interface (not shown) for performing data communication with the passer-by identification device 102. Etc. In the configuration example shown in FIG. 1, the gate device 103 controls the passage (entrance / exit) of passersby by controlling the opening and closing of a gate (not shown). That is, when the passer-by identification system is operated as an entrance / exit management system that controls the passage of gates, the gate device 103 passes according to the processing result (recognition result for passers-by) of the passer-by identification device 102. To control the traffic of the person.

  The monitoring device 104 is a device for a monitor to monitor a predetermined area. The monitoring device 104 has a configuration for informing the monitoring staff of information. For example, the monitoring device 104 includes a display device (not shown) for displaying information, an alarm device (not shown) for issuing an alarm, an operation unit (not shown) for a monitor to input an operation instruction, In addition, an interface (not shown) for performing data communication with the passer-by identification device 102 is configured. In the configuration example shown in FIG. 1, the monitoring device 104 monitors a person (passerby) in an area (an area near the gate device 103) taken by the camera 101. That is, when the passer-by recognition system is operated as a monitoring system that monitors a person in a predetermined area, the monitoring device 104 is configured to provide information such as a processing result (recognition result for a passer-by) of the passer-by recognition device 102. Is informed (displayed or warned by voice, etc.) to the supervisor.

Next, a configuration example of the passer-by identification device 102 will be described.
The passer-by identification device 102 includes an input unit (image input unit) 111, a control unit 112, an output unit 113, a storage unit 114, and the like. Furthermore, the control unit 112 includes a feature detection unit 121, a recognition unit 122, a group determination unit 123, and the like.

  The input unit 111 is an interface for inputting information such as an image. The input unit 111 also has a function of performing image processing on the input image information. That is, the input unit 111 captures image information (an image captured by the camera 101) supplied from the camera 101. The input unit 111 performs predetermined image processing on the image information acquired from the camera 101, and supplies the image information subjected to the predetermined image processing to the control unit 112. For example, when the image information acquired from the camera 101 is an analog signal, the input unit 111 supplies the image information digitized by A / D converting the captured image information to the control unit 112. The input unit 111 may store image information subjected to predetermined image processing in a buffer memory (not shown) so as to be readable from the control unit 112.

  The control unit 112 includes an arithmetic processor (not shown) such as a CPU, an internal memory (not shown) such as a working memory, a program memory, and a data memory, and an internal interface (not shown) for inputting / outputting data to / from each unit. ) Etc. In the control unit 112, various functions are realized by the arithmetic processor executing various programs stored in a program memory or the like. As such a function, the said control part 112 has functions, such as a feature detection part 121, the recognition part 122, and the group determination part 123 as shown in FIG.

  The feature detection unit 121 performs processing for detecting facial feature information (biological information) from image information captured from the camera 101 by the input unit 111. The feature detection unit 121 detects a face area image from the input image captured by the input unit 111 (face detection process), and extracts facial feature information from the face area image detected by the face detection process. Processing (feature extraction processing).

  The recognition unit 122 determines whether or not the person photographed by the camera 101 is a person (registrant) whose facial feature information (biological information) is stored in the storage unit 114. I do. That is, the recognizing unit 122 compares the facial feature information detected by the feature detecting unit 121 with the facial feature information stored in the storage unit 114, and the matching result obtained by the matching processing. Then, a determination process is performed to determine whether or not the facial feature information detected by the feature detection unit 121 matches any facial feature information stored in the storage unit 114.

The group determination unit 123 performs a process of determining a group for the person recognized by the recognition unit 122. That is, the group determination unit 123 divides each person recognized by the recognition unit 122 into group candidates, and determines a group for each group candidate person.
The processing in the feature detection unit 121, the recognition unit 122, and the group determination unit 123 will be described in detail later.

  The said memory | storage part 114 memorize | stores the information regarding a registrant. The storage unit 114 stores facial feature information (or a face image) as biometric information of each registrant and group information indicating a group to which each registrant belongs. In the first embodiment, it is assumed that the storage unit 114 stores information as described above in advance. The information stored in the storage unit 114 will be described in detail later.

  The output unit 113 outputs information such as a control signal or display data to the gate device 103 or the monitoring device 104 (external device) based on the recognition result of the recognition unit 122 or the group determination result of the group determination unit 123. It is supposed to be. For example, the output unit 113 indicates to the gate device 103 whether each passerby can pass (gate opening / closing) according to the recognition result by the recognition unit 122 or the determination result by the group determination unit 123. Output a control signal. The output unit 113 outputs information (data such as an image or sound) such as a recognition result by the recognition unit 122 and a group determination result by the group determination unit 123 to the monitoring device 104. In this case, the output unit 113 may output video information photographed by the camera 101 together with the above information.

Next, the feature detection unit 121 will be described in detail.
The feature detection unit 121 performs a face detection process for detecting a face area image from an input image and a feature extraction process for extracting face feature information from the face area image.
First, the face detection process in the feature detection unit 121 will be described in detail.
In the face detection process, first, a face region is detected from an image input by the input unit 111 (an image photographed by the camera 101). As a method for detecting a face region, for example, a correlation value is obtained while moving a template prepared in advance in an image, and a region giving the highest correlation value is set as a face region. As a method for detecting a face area, a face area extraction method using an eigenspace method or a subspace method may be used.

  In the face detection process, next, the position of a specific part in the face such as eyes, nose, mouth, etc. is detected in the image of the detected face region. As the detection method, for example, Document [2] (Kazuhiro Fukui, Osamu Yamaguchi: “Face feature point extraction by combination of shape extraction and pattern matching”, IEICE Transactions (D), vol. J80-D- II, No. 8, pp 2170-2177 (1997)) and the like can be used.

  Note that in a person recognition device that recognizes a person using biometric information obtained from an image of a specific part of the face instead of the entire face, the image of the desired part detected by the face detection process is zoomed up, It is also possible to perform processing for extracting biometric information from the zoomed-up image. This can be realized, for example, by controlling the camera 101 so as to zoom up the position of a desired part detected by the face detection process. In this case, the camera 101 captures an image obtained by zooming up the position of a desired part, and supplies the image to the person recognition device. For example, in a person recognition apparatus that recognizes a person using biological information obtained from an eye image such as an iris or a retina, the eye area detected by the face detection process is zoomed in, and the eye area is zoomed. It is possible to obtain an uploaded image.

In addition to the above eye and nose detection, the detection of the mouth area is described in, for example, Document [3] (Mayumi Yuasa, Ayako Nakajima: “Digital makeup system based on high-precision facial feature point detection”, 101st image sensing. The technology of the Symposium Proceedings, pp 219-224 (2004)) can also be used.
Also, a general recognition technique using pattern matching or the like can be used for a fingerprint or vein pattern.
Any of the above-described techniques can acquire two-dimensional array image information and detect biological information from the image information.

  Further, according to the above-described processing, in order to extract only one face area image (biological information) from one image, a correlation value with the template is obtained for the entire image, and the correlation value is maximum. What is necessary is just to determine the position and size of one area which become as one face area. On the other hand, in order to extract images (biological information) of a plurality of face regions from one image, a plurality of local maximum values of correlation values for one whole image are obtained, and these correlation values are localized. Each region having a maximum value is narrowed down to face region candidates in consideration of overlapping with other regions. Furthermore, when images (moving images) are continuously obtained as input images, it is possible to narrow down a plurality of face area candidates by considering the relationship (temporal transition) with past input images. .

  FIG. 2 is a diagram illustrating an example of an image having a plurality of face regions. When one image as shown in FIG. 2 is input, in the face detection process, a plurality of areas A to D indicated by broken lines in FIG. 2 are detected as face areas in the input image. That is, in the system configuration as shown in FIG. 1, when a plurality of passersby walk to the gate device 103 at the same time, the camera 101 captures the passersby simultaneously in one image. For example, when four passers-by are present at the same time in the imaging region, the camera 101 captures an image as shown in FIG. In this case, in the face detection process, the areas A to D are detected as the face area of each passerby.

Next, the feature extraction process in the feature detection unit 121 will be described in detail.
The feature extraction processing in the feature detection unit 121 is processing for extracting facial feature information (feature amount) from the face area image detected by the face detection processing. In the feature extraction process, for example, based on the face area image detected by the face detection process and the position of a specific part, the face area image has a predetermined size and a predetermined shape (for example, m (Pixel × n pixel image), and shade information of each pixel in the face area image is used as facial feature information (feature amount). Also, the facial feature information representing the gray value of each pixel in the image of the face area of m pixels × n pixels is represented as a feature vector composed of m × n-dimensional information. For the feature vector as the facial feature information as described above, for example, the similarity with another feature vector is calculated by a method called a simple similarity method. In the simple similarity method, the lengths of the respective vectors are normalized so as to be “1”, and the inner product between these normalized vectors is calculated. Thereby, the similarity indicating the similarity between two vectors is obtained.

  When the recognition process is performed on one image, the feature information of the person's face is extracted as a feature vector from one image by the above-described method. On the other hand, when performing recognition processing with a plurality of continuous images (moving images), a technique called a subspace method that generates facial feature information from a plurality of continuous images can be applied. In the subspace method, first, feature vectors composed of gray values in an image of a face area of m × n pixels are generated for a plurality of continuous input images by the same method as described above. When a plurality of feature vectors are obtained from a plurality of continuous input images, the subspace method obtains a correlation matrix (or covariance matrix) of these feature vectors. When a correlation example is obtained, the subspace method obtains an orthonormal vector (eigenvector) by performing a well-known KL expansion on the correlation matrix.

  A partial space is calculated as facial feature information obtained from successive images using the orthonormal vector obtained as described above. The subspace is expressed using a set of k eigenvectors in descending order of eigenvalues. The subspace is calculated as a matrix Φ of eigenvectors using Equation 1 below using a correlation matrix Cd calculated from a plurality of feature vectors obtained from a plurality of input images.

Cd = ΦdΛdΦdT (Equation 1)
The partial space is the facial feature information of the person to be recognized. For example, as in the example shown in FIG. 2, when a plurality of face areas (face areas of four passers) A to D are detected, the feature extraction process is performed for each face area A to D. The processing as described above is repeatedly executed. Thus, four partial spaces A to D are calculated as the feature information of the faces of the four passers-by. In addition, the partial space calculated by the above-described method may be registered in the storage unit 114 as the registrant's facial feature information.

Next, the registration information stored in the storage unit 114 will be described in detail.
The storage unit 114 stores information related to a registrant (for example, a person permitted to pass through a gate) to be collated with an input image (facial feature information obtained from the input image). The storage unit 114 stores in advance information about the registrant's face obtained from an image including the registrant's face (registration face image) as information about the registrant.
FIG. 3 is a diagram illustrating an example of information (registration information) related to a registrant stored in the storage unit 114. In the example illustrated in FIG. 3, identification information, facial feature information, facial images, group information, and the like are stored in the storage unit 114 for each registrant.

  The identification information is information for identifying the registrant. For example, the identification information includes a registrant's name and ID information given to the registrant. The face feature information is information (registrant face feature information) for collation with face feature information obtained from an input image. The face feature information is information generated from a registration face image by the technique used in the feature extraction process described above. The face image is an image including the registrant's face for registration (the entire registration image or an image of a face area detected from the registration image).

  The storage unit 114 may store a partial space generated from a plurality of consecutive registration face images as face feature information, or may be generated from each registration face image. A feature vector may be stored, a face image before execution of feature extraction processing (an image of a face area detected from an input image) may be stored, or KL expansion is performed. You may make it memorize | store the correlation matrix before performing. Here, it is assumed that a partial space obtained from a plurality of consecutive images is registered in the storage unit 114 as facial information of the registrant.

  The storage unit 114 may store feature information of one face for one registrant, or may store feature information of a plurality of faces for one registrant. You may do it. In the case of storing a plurality of facial feature information for one registrant, the recognizing unit 122 recognizes, for each registrant, all feature information of the registrant and facial feature information obtained from the input image. The face feature information selected based on the shooting conditions of the input image and the face feature information obtained from the input image may be matched.

  The group information is information indicating a group to which the registrant belongs. The storage unit 114 stores information indicating which group each registrant belongs to in association with each registrant. Further, each group stores information indicating the characteristics of the group such as importance, alertness (degree of caution), and necessity of assistance (necessity of assistance) in association with a group number. In the example shown in FIG. 3, as information indicating the characteristics of the group, a group of “important customers” (group number 1), a group of “president's family” (group number 2), and a group of “groups requiring attention” (group number 3). ) Is stored.

Next, the recognition unit 122 will be described in detail.
The recognizing unit 122 recognizes whether the person of the face image included in the image captured by the camera 101 is any registrant stored in the storage unit 114. The recognition unit 122 calculates the similarity between the facial feature information obtained from the input image captured by the camera 101 and the facial feature information of each registrant stored in the storage unit 114. When the similarity with the facial feature information of each registrant is calculated, the recognizing unit 122 determines which registrant matches the person of the face image included in the input image based on the calculated similarity. Determine. For example, when the maximum similarity is greater than or equal to a predetermined value, the recognition unit 122 recognizes that the person of the face image included in the input image is the registrant who has the maximum similarity (included in the input image). Recognize that the person in the face image has the highest similarity) If the maximum similarity is less than a predetermined value, the recognizing unit 122 recognizes that the facial feature information detected from the input image does not match any registrant's facial feature information (in the input image). Recognize that the person in the included face image is not a registrant).

  A method such as a mutual subspace method or a compound similarity method can be applied as a method for calculating the similarity between facial feature information. For example, the mutual subspace method is described in the literature [4] (Kenichi Maeda, Sadaichi Watanabe: “Pattern Matching Method Introducing Local Structure”, IEICE Transactions (D), vol. 3, pp 345-352 (1985)). This mutual subspace method is a method for calculating the similarity between subspaces. In the mutual subspace method, an “angle” formed by two subspaces is defined as a similarity. Therefore, when the recognition unit 122 calculates the similarity by the mutual subspace method, the facial feature information of the registrant stored in the storage unit 114 and the input image input from the camera 101 (sequential multiple images) Both of the facial feature information obtained from the image of FIG. Here, the partial space obtained from the input image is referred to as an input partial space. In addition, the partial space as the registrant's facial feature information stored in the storage unit 114 is referred to as a dictionary partial space.

  A relational expression as shown in Equation 2 below is established between the correlation matrix Cin and the eigenvector Φin obtained from a plurality of continuous input images. Therefore, the input subspace Φin is calculated by the relational expression between the correlation matrix Cin and the state-owned vector Φin shown in Equation 2 below.

Cin = ΦinΛinΦinT (Formula 2)
Accordingly, the similarity between the person of the face image included in the input image (face feature information obtained from the input image group) and the registrant (face feature information of the registrant) is the input subspace Φin and the dictionary subspace Φd. As a similarity (0 to 1).

  When there are a plurality of human face images in each input image (one frame image), the recognition unit 122 stores the facial feature information obtained from each face image and the storage unit 114. The similarity with the facial feature information of each registrant is calculated. That is, the recognizing unit 122 calculates the similarity between the facial feature information obtained from the facial images included in the input image and the facial feature information of each registrant stored in the storage unit 114. Calculate by hit. As a result, the recognition unit 122 can obtain the recognition result of the face images of all persons included in the input image and the face images of the registrant. For example, when the camera 101 captures the faces of X persons, if there is registration information of Y registrants, the recognizing unit 122 performs X × Y similarity processing. Thereby, the said recognition part 122 can obtain the recognition result with respect to all X persons.

If the input subspace obtained from the m input images does not match any registrant's dictionary subspace, the recognition unit 122 updates the input subspace with the next acquired input image (m + 1 frame image). The recognition process (similarity calculation process) may be executed again using the input subspace. In this case, the input subspace can be updated by adding the feature vector obtained from the acquired input image to the sum of correlation matrices created in a plurality of past frames, and calculating the eigenvector again.
That is, when the camera 101 continuously captures images (when the camera 101 continuously captures a passer's face image), the passer-by identification device 102 uses images sequentially acquired from the camera 101. It is possible to execute the authentication process while updating the input subspace. In this case, since the number of input images for configuring the input subspace gradually increases, it can be expected that the accuracy gradually increases.

Next, the group determination unit 123 will be described in detail.
The group determination unit 123 performs a group determination process (combination determination process) for determining a group for a plurality of persons recognized by the recognition unit 122. The group determination unit 123 determines group candidates obtained by grouping a plurality of persons recognized by the recognition unit 122 according to a predetermined condition. Further, the group determination unit 123 determines a group (group name, number of people, etc.) based on group information of each person (registrant) as a group candidate.

  In the group candidate determination, a plurality of persons who are group candidates are selected based on a predetermined condition. For example, as a group candidate, a plurality of persons detected from one image (a plurality of persons photographed by the camera 101 at the same time) can be used as group candidates, or from images captured by the camera 101 within a predetermined time. The detected person is used as a group candidate. In an operation mode in which a plurality of cameras 101 are installed at a plurality of locations, a person detected from images captured at a plurality of locations in the same time zone may be determined as a group candidate.

  Further, the group is determined based on the group information of each person (registrant) determined as a group candidate. That is, each group candidate is finally determined as a group based on the group information. As a result of such group determination, information such as which group is how many is obtained. The group determination unit 123 supplies the result of group determination as described above to the output unit 113.

FIG. 4 is a diagram illustrating an example of a result of the group determination process.
In the example shown in FIG. 4, it is shown that three persons in group 1 and two persons in group 2 are recognized from an image taken by a camera installed in gate 1. For example, when four face images detected from one image as shown in FIG. 2 are recognized as the face images of the persons (registrants) A to D shown in FIG. 3, referring to FIG. In the group determination process, the persons A, B, and C are determined as the group 1, and the persons C and D are determined as the group 2. In this case, as a result of the group determination, as shown in FIG. 4, there are three persons in group 1 and two persons in group 2.

  The group determination result shown in FIG. 4 is an example when all persons are recognized as registrants. If a person who is not registered in advance and tries to pass through the gate device 103, the person is not recognized as a registered person. In this case, a person who is not recognized as a registered person is counted up as an unregistered person. Note that the number of unregistered persons counted up as unregistered persons is output by the output unit 113 as a result of group determination.

  When the group determination result as shown in FIG. 4 is displayed on the monitoring device 104, the monitoring device 104 has three persons belonging to the group 1 and two persons belonging to the group 2 as the camera. It is possible to quickly and easily recognize that the image was taken at 101. As a result, the monitor can smoothly respond to the persons in each group. In addition, the monitor can smoothly perform an appropriate response according to the number of people in each group.

Next, the output unit 113 will be described in detail.
The output unit 113 is an interface for outputting a processing result in the passer-by identification device 102 to the outside. The information output to the outside by the output unit 113 can be set as appropriate according to the overall system configuration or operation mode. The output unit 113 displays information such as an image captured by the camera 101, an image of a face area obtained from the feature detection unit 121 or facial feature information, and a group determination result by the group determination unit 123 as an external device ( Output to the gate device 103 and the monitoring device 104). For example, the output unit 113 outputs a control signal instructing opening / closing of the gate according to the recognition result of the recognition unit 122 to the gate device 103. Further, the output unit 113 displays information such as an image captured by the camera 101, an image of a face area obtained from the feature detection unit 121 or facial feature information, and a group determination result by the group determination unit 123. The data is output to the monitoring device 104.

Next, a process flow in the passer-by identification system 100 according to the first embodiment will be described.
FIG. 5 is a flowchart for explaining processing in the passer-by identification system 100 according to the first embodiment.
First, it is assumed that the camera 101 is shooting an image of a predetermined shooting area. In the configuration example shown in FIG. 1, the passage formed by the gate device 103 is photographed. An image taken by the camera 101 is supplied to the input unit 111 of the passer-by identification device 102. The input unit 111 of the passer-by identification device 102 sequentially inputs images taken by the camera 101 as input images (step S101). In the control unit 112 of the passer-by identification device 102, the feature detection unit 121 performs processing for detecting a human face in the input image (step S102). When the feature detection unit 121 detects a human face from the input image, the feature detection unit 121 performs face detection processing and feature extraction processing for extracting facial feature information. Thereby, the feature detection unit 121 detects the feature information of the face of the person included in the input image (step S103).

  When generating the input subspace as facial feature information, the feature detection unit 121 generates an input subspace from input images that are sequentially input. When a plurality of person face images are included in one input image, the feature detection unit 121 detects face feature information from all the person face images detected from the input image.

  When the feature detection unit 121 detects facial feature information from the input image, the recognition unit 122 detects the facial feature information detected from the input image of any registrant stored in the storage unit 114. It is determined whether it matches the facial feature information. That is, the recognition unit 122 performs a person recognition process for determining which registrant is the person of the face image included in the input image (step S104). In the recognition process by the recognizing unit 122, information indicating which registrant is the person of the facial feature information detected from the input image or an unregistered person is obtained as a person recognition result. The result of the recognition process by the recognition unit 122 is supplied to the output unit 113 and also to the group determination unit 123. The output unit 113 outputs a control signal indicating opening / closing of the gate to the gate device 103 according to the recognition result, or outputs the recognition result to the monitoring device 104 together with the input image.

  The group determination unit 123 stores the recognition result supplied from the recognition unit 122 in an internal memory (not shown). In addition, as a result of the recognition process supplied to the group determination unit 123, a face image of the person is included together with information indicating a person recognition result (whether the person is a registered person or an unregistered person). Information such as the shooting time of the detected input image is also included.

  The group determination unit 123 performs group determination for the person recognized by the recognition unit 122. Here, a plurality of persons photographed at the same time (a plurality of persons from which one input image is detected) or a person photographed within a predetermined time is assumed as a group candidate. In addition, the recognition result of each person is sequentially supplied from the recognition unit 122 to the group determination unit 123. When the recognition result is supplied by the recognition unit 122, the group determination unit 123 determines whether another human face is detected from the input image from which the recognition result is obtained. That is, the group determination unit 123 determines whether or not faces of a plurality of persons (a plurality of persons photographed at the same time) are detected from one input image (step S105). Further, the group determination unit 123 determines whether or not a plurality of human faces are detected from the input image taken within a predetermined time (step S106).

  When it is determined that a plurality of human faces have been detected from one input image or an input image taken within a predetermined time (YES in step S105 or YES in step S106), the group determination unit 123 A person is set as a group candidate (step S107). When it is determined that the recognized person is a group candidate, the group determination unit 123 determines a group to which the person belongs based on the group information of the person (step S108). When the group is determined, the group determination unit 123 counts up the number of people in each group in the group candidate (step S109). When the recognized person is counted up based on the group information, the group determination unit 123 outputs information indicating the number of persons for each group in the group candidate to the output unit 113 as a result of the group determination (step S110).

  For example, as shown in FIG. 2, when four persons A, B, C, and D shown in FIG. 3 are recognized in order from one input screen, the group determination unit 123 determines the persons A, B, C, and D. The four candidates are group candidates. Furthermore, the group determination unit 123 counts up the number of persons in each person's group every time a recognition result of each person is obtained. That is, according to the registration information shown in FIG. 3, the group determination unit 123 counts up the group 1 when the person A is recognized, and counts up the group 1 when the person B is recognized. When C is recognized, group 1 and group 2 are counted up, and when person D is recognized, group 1 is counted up. As a result of such group determination, information as shown in FIG. 4 is obtained.

  Further, when it is determined that a plurality of human faces are detected from the input image captured within a predetermined time (NO in step S106), the group determination unit 123 determines group candidates for the person recognized by the recognition unit 122. It is determined that there is none (step S111). In this case, the passer-by identification device 102 returns to step S101 and repeats the above process.

  In the first embodiment as described above, the facial feature information of the registrant and the group information are stored in the storage unit in association with each other, and the facial feature information of the passer-by from the image captured by the camera 101 is stored. And detecting the passerby by comparing the detected face feature information of the passer with the face feature information of the registrant stored in the storage unit, and further storing the recognized passer The grouping is performed based on the group information of each registrant stored in the section, and the grouping result is output to the external device.

  As a result, according to the first embodiment, the passer-by identification device determines whether or not “what kind of person in the group” appears in a predetermined place (area photographed by the camera). Can notify external devices. As a result, the external device can notify the supervisor of “how many people in what group” appear in a predetermined location. As a result of this, for example, when a group of important customers appears, the supervisor can quickly prepare to respond to the visitor. In addition, when a group of people who need attention appears, the supervisor can quickly take measures such as deploying security guards for warning or starting recording of surveillance video. In addition, for a group including people who need attention, even a registrant can perform control such as closing the gate of the gate device.

Next, a second embodiment will be described.
FIG. 6 schematically shows a configuration example of a passer-by recognition system 200 including a passer-by recognition device 202 as a person recognition device according to the second embodiment.
As shown in FIG. 6, the passer-by identification system 200 includes a camera 201, a passer-by identification device 202, a gate device 203, a monitoring device 204, and the like. The passer-by identification device 202 includes an input unit (image input unit) 211, a control unit 212, an output unit 213, a storage unit 214, and the like. Further, the control unit 212 includes a feature detection unit 221, a recognition unit 222, a group determination unit 223, and the like.

  The camera 201, the gate device 203, the monitoring device 204, the input unit (image input unit) 211, the output unit 213, the feature detection unit 221 and the recognition unit 222 in the passer-by identification system 200 illustrated in FIG. The configuration of the camera 101, the gate device 103, the monitoring device 104, the input unit (image input unit) 111, the output unit 113, the feature detection unit 121, and the recognition unit 122 in the passer-by identification system 100 shown in FIG. Detailed description will be omitted.

  The passer-by identification system 200 according to the second embodiment and the passer-by identification system 100 described in the first embodiment are different in information for grouping persons. In the passer-by identification system 100 according to the first embodiment, group information indicating the group name of each registrant is registered as information for grouping each person. This group information is information for directly grouping the persons. For this reason, in the passer-by identification system 100, it is necessary to register each registrant in groups and store the information in the storage unit.

  In contrast, in the passer-by identification system 200 according to the second embodiment, it is assumed that the attribute information of each registrant is registered as information for grouping each person. Therefore, the passer-by identification system 200 determines each person's group based on the attribute information of each registrant. Various information can be registered as the attribute information. Examples of the attribute information include information such as age, sex, importance, alertness, and necessity of assistance. If information to be grouped among such attribute information is set, the passer-by identification device 202 outputs information grouped according to the set information as a result of group determination. For example, if age or gender is set as grouping information, the passer-by identification apparatus 202 outputs a group determination result grouped by age or gender. In addition, if importance level, vigilance level, or necessity of assistance are set as grouping information, the passer-by identification device 202 determines the result of group determination divided into groups according to importance level, warning level, or necessity of assistance. Is output.

Hereinafter, each part of the passer-by identification system 200 will be described in detail.
First, the storage unit 214 stores identification information, facial feature information, facial images, and attribute information (age, sex, importance, alertness, necessity of assistance, etc.) as registration information for each registrant. . The identification information, face feature information, and face image are the same as the identification information, face feature information, and face image information stored in the storage unit 114 described in the first embodiment. That is, the storage unit 214 is different in that person attribute information is stored instead of the group information stored in the storage unit 114 described in the first embodiment.
FIG. 7 is a diagram showing an example of registration information for each registrant stored in the storage unit 214. In the example shown in FIG. 7, for each registrant, as attribute information associated with identification information, facial feature information, and facial image information, age, gender, importance, alertness, necessity of assistance, etc. Is stored.

  The group determination unit 223 performs group candidate determination and group determination in the same manner as the group determination unit 123 described in the first embodiment. The group candidate is determined by the same process as the group determination unit 123 described above. In the group determination in the group determination unit 223, each person (registrant) of the group candidate is grouped based on predetermined information among the attribute information of each person (registrant), and the number of each grouped group is counted. Up. In addition, it is assumed that the group determination unit 223 is preset with specific information (one or more) among the attribute information stored in the storage unit 214 as information for grouping. However, the grouping information may be set as appropriate by the operator, or may be set as appropriate by the monitor using an operation unit (not shown) of the monitoring device 204.

FIG. 8 is a diagram illustrating an example of a result of group determination processing in the group determination unit 223.
In the example shown in FIG. 8, it is shown that two important persons and one person who needs assistance are recognized from an image captured by a camera installed in the gate 1. Here, if the registration information as shown in FIG. 7 is stored in the storage unit 214, the result of the group determination as shown in FIG. 8 is the case where the persons B, C, and D are detected as group candidates. Is obtained. This is because in the example shown in FIG. 7, the importance is important in the attribute information of the person C and the person D, and the attribute information of the person B is essential.

  Further, when the group determination result as shown in FIG. 8 is displayed on the monitoring device 204, the monitoring device 204 has two important persons and one auxiliary person photographed by the camera 201. Can be easily recognized. As a result, the monitor can smoothly and appropriately respond to the important person, and can appropriately respond to the person who needs assistance, such as wheelchair preparation or road guidance.

Next, the flow of processing in the passer-by identification system 200 according to the second embodiment will be described.
FIG. 9 is a flowchart for explaining processing in the passer-by identification system 200 according to the second embodiment.
Note that the processes in steps S201 to S207 and S209 to S211 shown in FIG. 9 are the same as the processes in steps S201 to S207 and S209 to S211 shown in FIG.

  That is, in the passer-by identification device 202, when the person recognized by the recognition unit 222 is set as a group candidate, the group determination unit 123 determines the person based on predetermined grouping information among the attribute information of the person. Are determined (step S208). When the group of each person is determined, the group determination unit 123 counts up the number of persons for each group in the group candidate (step S209). Here, the number of people in each group is counted up for each group candidate. The group determination unit 123 outputs information indicating the number of people in each group in the group candidate to the output unit 113 as a result of group determination (step S210).

  For example, when the three persons B, C, and D shown in FIG. 7 are determined as group candidates, the group determination unit 123 divides the groups based on the attribute information of each person, and counts up the number of persons in each group. To do. That is, according to the registration information shown in FIG. 7, when the person B is recognized, the group determination unit 123 counts up the number of people who need assistance (groups that need assistance), and when the person C is recognized, The number of important persons (group of important persons) is counted up, and when the person D is recognized, the number of important persons (group of important persons) is counted up. As a result of such group determination, information as shown in FIG. 8 is obtained.

  In the passer-by identification system according to the second embodiment as described above, the registrant's facial feature information and various attribute information are stored in the storage unit in association with each other, and the image captured by the camera 101 is stored. Detecting the passer's face feature information from the detected passer's face feature information and comparing the registrant's face feature information stored in the storage unit to recognize the passer-by, The recognized passers-by are grouped based on the attribute information of each registrant stored in the storage unit, and the grouped result is output to an external device.

  As a result, according to the second embodiment, the passer-by identification device monitors whether a “group of what kind of persons” appears in a predetermined place (area photographed by the camera). Can be notified to external devices. As a result, the external device can notify the supervisor of “a group of what kind of persons” appears in a predetermined place. As a result, for example, when a group of important persons appears, the supervisor can quickly prepare for the reception of the visitor. In addition, when a group of people who need attention appears, the supervisor can quickly take measures such as deploying security guards for warning or starting recording of surveillance video. Further, for a group including persons having specific attribute information such as a person requiring attention, even a registrant can perform control such as closing the gate of the gate device 203.

Next, a third embodiment will be described.
FIG. 10 schematically shows a configuration example of a passer-by identification system 300 according to the third embodiment.
In the third embodiment, as shown in FIG. 10, the camera 301, a plurality of passer-by identification devices 302A and 302B, a plurality of gate devices 303A and 303B, a monitoring device 304, and a server device 305 are configured. Assume that a passer-by identification system 300 is present. Each passer-by identification device 302 (302A, 302B) includes an input unit (image input unit) 311 (311A, 311B), a control unit 312 (312A, 312B), an output unit 313 (313A, 313B), and the like. ing. Further, each control unit 312 (312A, 312B) includes a feature detection unit 321 (321A, 321B), a recognition unit 322 (322A, 322B), a group determination unit 323 (323A, 323B), and the like.
Note that the configuration of the passer-by identification system 300 as described above can be modified as appropriate according to the operation mode and the like. For example, each configuration of the server device 305 may be provided in any passer-by identification device 302.

  Further, in the passer-by identification system 300 shown in FIG. 10, the camera 301 (301A, 301B), the gate device 303 (303A, 303B), the input unit 311 (311A, 311B), the output unit 313 (313A, 313B), feature detection The unit 321 (321A, 321B), the recognition unit 322 (322A, 322B), and the group determination unit 323 (323A, 323B) are respectively the camera 101, the gate device 103, and the input unit 111 in the passer-by identification system 100 shown in FIG. The output unit 113, the feature detection unit 121, the recognition unit 122, and the group determination unit 123 have the same configuration, and thus detailed description thereof is omitted.

Hereinafter, each part of the passer-by identification system 300 will be described in detail.
In the passer-by identification system 300, passer-by identification devices 302A and 302B are connected to the server device 305. The server device 305 is a data server that stores and manages various data. As shown in FIG. 10, the server device 305 includes a storage unit 331, a history database 332, and an association determination unit 333. Further, the server device 305 has an interface (not shown) for performing data communication with the passer-by identification device 302A, the passer-by recognition device 302B, and the monitoring device 304.

  The storage unit 331 stores registration information (identification information, facial feature information, face image, group information, etc.) related to each registrant similar to the storage unit 114 described in the first embodiment. To do. However, it is assumed that the registration information stored in the storage unit 331 is appropriately rewritable. The registration information stored in the storage unit 331 is information used in person recognition processing or group determination processing in the two passer-by identification devices 302A and 302B. Therefore, the registration information stored in the storage unit 331 is appropriately read out by the passer-by identification device 302A or 302B.

  The history database 332 stores information such as recognition results by the passer-by identification device 302A or 302B as history information. The history information stored in the history database 332 includes, for example, information such as a person recognition result (information indicating a recognized person, information indicating a place where the image was taken (gate), date and time of recognition (photographing)), and the like. Remembered.

  The relevance determination unit 333 determines relevance (group-likeness) between persons based on information stored in the history database 332 (information such as recognition results obtained by the passer-by identification devices 302A and 302B). To do. The determination result by the relevance determination unit 333 is appropriately reflected in the group information of each registrant stored in the storage unit 331.

  That is, the relevance determination unit 333 determines relevance between the persons recognized by the passer-by identification devices 302A and 302B based on preset conditions. The conditions for determining the relationship between a plurality of recognized persons include, for example, “whether they are photographed at the same time” or “persons photographed continuously within a predetermined time” Can be set. In addition, the relevance determination unit 333 determines the degree of relevance of each recognized person with another person as an evaluation value of relevance, and evaluates the relevance evaluation value (value indicating relevance) as data. A table (correlation map) 333a is accumulated for each person, and a group of each person is determined based on the accumulated evaluation value.

  For example, the evaluation value of the relationship between persons photographed “simultaneously” and the evaluation value of the relationship between persons photographed “continuously within a predetermined time” are different values (weighted values). You may make it set with. In this case, the relevancy determination unit 333 determines that “the possibility of grouping is high (the evaluation value is set to the highest value because the relevance is high)” between persons photographed “simultaneously” with a certain camera. However, it is determined that there is a possibility of grouping (there is low relevance, so the evaluation value is low) between persons photographed “continuous within a predetermined time” with a certain camera. It is determined that “a person photographed after elapse of a predetermined time or more” is “not a group (there is no evaluation because there is no relationship)”.

  Furthermore, the relevance evaluation value may be set stepwise in accordance with the time interval at which each person is continuously photographed. For example, the relevance evaluation value is set to the first value (level 3; for example, the evaluation value is “3”) for persons photographed at the same time, and is taken within the first time (for example, 1 minute). The second value (level 2; for example, the evaluation value is “2”) is used for the distance between persons, and the above-mentioned number is set for the distance between persons photographed from the first time to the second time (for example, 5 minutes). A third value lower than the value of 2 (level 1; for example, the evaluation value is “1”), and a person who is photographed at intervals of the second time or longer is a fourth lower than the third value. (Level 0; for example, the evaluation value is “0”).

FIG. 11 is a diagram illustrating an example of history information as a history of recognition results by the passer-by identification device. FIG. 12 is a diagram illustrating an evaluation example of relevance to the history information illustrated in FIG. 11.
In the example shown in FIG. 12, the relevance determination unit 333 compares each recognition result with the previous recognition result for the history information as shown in FIG. is doing. That is, the relevance determination unit 333 evaluates the relevance between each person by sequentially comparing the recognition result obtained by each passer-by identification device 302 with the immediately preceding recognition result. Here, the case where each recognition result of the history information shown in FIG. 11 is evaluated based on the above-described conditions will be described.

  First, the relevance determination unit 333 compares the recognition result of the history number 2 with the recognition result of the history number 1 immediately before that. When the recognition result of history number 2 is compared with the recognition result of history number 1 immediately before, the relevance determination unit 333 determines that the shooting interval (date and time difference) is within the first time (1 minute). Therefore, the evaluation value of the relationship between the person B and the person A is “2”.

  The relevancy determination unit 333 compares the recognition result of the history number 3 with the recognition result of the history number 2 immediately before. Comparing the recognition result of the history number 3 with the recognition result of the history number 2 immediately before, the relevance determination unit 333 determines that the person C, the shooting interval is within the second time (1 minute). The evaluation value of the relationship between D, E and person B (C and B, D and B, E and B) is determined to be “1”, respectively. In addition, the relevance determination unit 333 recognizes that the persons C, D, and E are simultaneously photographed in the recognition result of the history number 3, and thus the persons C, D, and E (C and D, D and E, and E and C ) Is evaluated as “3”.

The relevancy determination unit 333 compares the recognition result of the history number 4 with the recognition result of the history number 3 immediately before. Comparing the recognition result of the history number 4 with the recognition result of the history number 3 immediately before, the relevance determination unit 333 determines that the person F has the shooting interval equal to or longer than the second time (5 minutes). And the persons C, D, and E (F and C, F and D, and F and E) are determined to be “0”.
When the relevance determination results as described above are arranged, a data table (correlation map) 333a showing the correlation as shown in FIG. 12 is generated. According to the correlation map as shown in FIG. 12, the relevance of the persons A to F can be evaluated. Furthermore, according to the correlation map 333a in which the evaluation values of relevance as described above are accumulated for a predetermined period, information indicating the correlation between persons in the predetermined period (information indicating grouping) is created.

  In the example illustrated in FIG. 12, a case is described in which association is performed for a person who is recognized as a registrant whose registration information is stored in the storage unit 331 in advance. Such determination of relevance can also be applied to an unregistered person (a person who is determined not to be a registered person). In this case, for a person determined to be an unregistered person, an ID number is assigned to the person, and the feature information of the person's face is associated with the ID number as registration information of the temporary registrant. Remember. Each recognition device performs a person recognition process using the registration information of the temporary registrant registered as described above. As a result, when the person (temporary registrant) is photographed by any camera next time, the passer-by identification system recognizes the person. If the temporary registrant is recognized as described above, the passer-by identification system can determine the relevance of the unregistered person with the other person as described above.

  Further, the relationship between each person obtained as described above may be registered as loop information of each registrant based on specific conditions. For example, the relevance determination unit 333 of the server device 305 determines whether or not the relevance evaluation value in the correlation map 333a exceeds a predetermined threshold. When it is determined by this determination that the persons whose relevance evaluation value exceeds a predetermined threshold value, the relevance determining unit 333 determines the persons as a group, and the groups of those persons (registrants). Update information. In this case, it is possible to update the group information of each registrant based on the evaluation value of relevance without involving human resources.

  In addition, the relevance determining unit 333 of the server device 305 indicates to the monitoring device 304 or the like “a possibility that the relevance evaluation value exceeds a predetermined threshold in the correlation map as described above. You may make it display the guidance to the effect of "high". In this case, the supervisor or the system administrator may visually confirm the guidance as described above and instruct the operation unit 304a whether or not to register those persons as a group based on the confirmation result. . In this case, the relevance determination unit 333 of the server device 305 updates the group information of each person in accordance with an instruction from the monitor or system administrator. In this case, since the group information of each registrant is updated after confirmation by the supervisor or the system administrator, it is possible to manage the group reliably.

Next, the flow of relevance determination processing by the relevance determination unit 333 will be described.
FIG. 13 is a flowchart for explaining an example of the relevance determination process performed by the relevance determination unit 333. The relevancy determination process may be performed each time a recognition result is received from each passer-by identification device 302, or the recognition result by each passer-by recognition device 302 is stored in the history database 332. Then, the recognition result accumulated in the history database 332 may be periodically read and executed.

  First, it is assumed that the relevance determination unit 333 of the server device 305 acquires the recognition result by each passer-by identification device 302 (step S301). Then, the relevance determination unit 333 determines whether there are a plurality of persons recognized at the same time (step S302). When it is determined that a plurality of persons are recognized at the same time by this determination (step S302, YES), the relevance determination unit 333 evaluates that the relevance between the persons recognized at the same time is the first and the values ( Step S303). The evaluation value of the relevance is accumulated (voted) in the correlation map 333a indicating the relevance between the persons.

  Furthermore, the relevancy determination unit 333 compares the acquired recognition result with the immediately preceding recognition result, thereby determining whether or not the interval of the date and time (shooting date and time) in the recognition result is within the first time. Determination is made (step S304). That is, the relevance determination unit 333 determines whether there is a person who is continuously recognized within the first time. When it is determined that the interval between the acquired recognition result and the immediately preceding recognition result is within the first time (step S304, YES), the relevance determining unit 333 determines the acquired recognition result person and the immediately preceding recognition result. It is evaluated that the relationship with the person is the second value (step S305). Note that the second value is smaller than the first value. Such an evaluation value of relevance is accumulated (voted) in the correlation map 333a indicating the relevance between the persons (the acquired recognition result person and the previous recognition result person).

  When it is determined that the interval between the acquired recognition result and the previous recognition result is not within the first time (NO in step S304), the relevance determination unit 333 compares the acquired recognition result with the previous recognition result. By doing so, it is determined whether or not the interval of the date and time (shooting date and time) in the recognition result is within the second time (step S306). In other words, the relevancy determination unit 333 determines whether there is a person who has been continuously recognized for the first time or longer and within the second time. When it is determined that the interval between the acquired recognition result and the immediately preceding recognition result is within the second time (step S306, YES), the relevance determination unit 333 determines that the person of the acquired recognition result and the immediately preceding recognition result It is evaluated that the relationship with the person is the third value (step S307). Note that the third value is smaller than the second value. Such an evaluation value of relevance is accumulated (voted) in the correlation map 333a indicating the relevance between the persons (the acquired recognition result person and the previous recognition result person).

  When it is determined that the interval between the acquired recognition result and the immediately preceding recognition result is not within the second time (step S306, NO), the relevance determining unit 333 determines whether the acquired recognition result person and the immediately preceding recognition result are It is evaluated that there is no relationship with the person (step S308). In this case, the evaluation value is not accumulated (voted) in the correlation map 333a indicating the relationship between the persons (the acquired person as the recognition result and the person as the previous recognition result).

Further, the relevance determination unit 333 determines whether the evaluation value exceeds a predetermined value in the correlation map 333a (step S309). When it is determined that the evaluation value exceeds the predetermined value by this determination (step S309, YES), the relevance determination unit 333 displays the group information of each person whose relevance evaluation value exceeds the predetermined value. Update processing is performed (step S310).
As described above, in the process of step S310, the relevance determination unit 333 updates the group information of each person whose relevance evaluation value stored in the storage unit 331 exceeds a predetermined value. Alternatively, group information of each person whose relevance evaluation value exceeds a predetermined value is notified to the monitoring device 304, and the group information is updated in response to a group information update request from the monitoring device 304. You may make it do.
Further, the processing in steps S309 and S310 may be performed every time an evaluation value is accumulated in the correlation map 333a, or may be performed periodically, or in response to an instruction from a supervisor or system administrator. You may make it perform according to it.

In the passer-by identification system according to the third embodiment, passer's face feature information detected from an image captured by the camera is collated with face information of each registrant's face stored in the storage unit in advance. Thus, a passerby is recognized, and a plurality of passersby are associated based on the recognition result or the history of the recognition result. The passer-by identification system determines each passer-by group based on the association as described above.
Thereby, according to the said 3rd Embodiment, even if it is persons who are not previously registered as a group, a relevance can be evaluated, Furthermore, based on the relevance between each person, People can be grouped.

Next, a fourth embodiment will be described.
FIG. 14 schematically shows a configuration example of a passer-by identification system 400 according to the fourth embodiment.
In the fourth embodiment, as shown in FIG. 14, the camera 401, a plurality of passer-by identification devices 402A and 402B, a plurality of gate devices 403A and 403B, a monitoring device 404, and a server device 405 are configured. Assume that a passer-by identification system 400 is present. Each passer-by identification device 402 (402A, 402B) includes an input unit (image input unit) 411 (411A, 411B), a control unit 412 (412A, 412B), an output unit 413 (413A, 413B), and the like. ing. Further, each control unit 412 (412A, 412B) includes a feature detection unit 421 (421A, 421B), a recognition unit 422 (422A, 422B), a group determination unit 423 (423A, 423B), and the like.
Note that the configuration of the passer-by identification system 400 as described above can be modified as appropriate according to the operation mode and the like. For example, each configuration of the server device 405 may be provided in any passer-by identification device 402.

  Further, in the passer-by identification system 400 shown in FIG. 14, the camera 401 (401A, 401B), gate device 403 (403A, 403B), input unit 411 (411A, 411B), output unit 413 (413A, 413B), feature detection The unit 421 (421A, 421B), the recognition unit 422 (422A, 422B), and the group determination unit 423 (423A, 423B) are the camera 101, the gate device 103, and the input in the passer-by identification system 100 shown in FIG. Since the configuration is the same as that of the unit 111, the output unit 113, the feature detection unit 121, the recognition unit 122, and the group determination unit 123, detailed description thereof is omitted.

Hereinafter, each part of the passer-by identification system 400 will be described in detail.
In the passer-by identification system 400, passer-by identification devices 402A and 402B are connected to the server device 405. The server device 405 is a data server that stores and manages various data. As shown in FIG. 14, the server device 405 includes a storage unit 431, a history database 432, and a group management unit 433. Further, the server device 405 has an interface (not shown) for performing data communication with the passer-by identification device 402A, the passer-by recognition device 402B, and the monitoring device 404.

  The storage unit 431 stores registration information (identification information, facial feature information, face image, group information, etc.) related to each registrant similar to the storage unit 114 described in the first embodiment. To do. The registration information stored in the storage unit 431 is information used in a person recognition process or a group determination process in the passer-by identification devices 402A and 402B. Therefore, the registration information stored in the storage unit 431 is appropriately read out by the passer-by identification device 402A or 402B.

  The history database 432 stores information such as recognition results by the passer-by identification device 302A or 302B as history information. The history information stored in the history database 332 includes, for example, a person recognition result (information indicating a recognized person, information indicating a shooting location (gate), date and time of recognition (shooting)) and each person. Information such as the group determination result is stored.

  The group management unit 433 manages the state of each group based on information stored in the history database 332 (person recognition results and group determination results obtained by the passer-by identification devices 402A and 402B). . The group management unit 433 manages the traffic state of each person at each gate for each group. The group management unit 433 has a data table 433a that shows the traffic state of each person at each gate for each group. That is, the group management unit 433 determines which gate, which group of people, and how many people have passed, based on the person recognition result and group determination result by each passer-by identification device 402. This determination result is stored in the data table 433a as group management information.

  Further, the group management unit 433 appropriately outputs information (group management information) stored in the data table 433a to the monitoring device 404 as display data. For example, the group management unit 433 may output group management information to the monitoring device 404 each time the data table 433a is updated, or monitor the group management information in response to a request from the monitoring device 404. You may make it output to the apparatus 404. FIG.

FIG. 15 is a diagram illustrating an example of group management information stored in the data table 433a.
In the example illustrated in FIG. 15, information indicating a person who has passed through each gate at each time is stored for each group. For example, according to the example shown in FIG. 15, the persons A, B, and C of group 1 pass through gate 1 at 10:00, pass through gate 2 at 10:30, and further pass through gate 2 at 10:40. Is going through. Among the persons A, B, and C who passed the gate 2 at 10:40, the persons A and C passed through the gate 1 at 11:00. However, the person B is not recognized by at least 11:30. This means that the person B who was acting with the persons A and C is moving away from the persons A and C in the group 1 after passing through the gate 2. Therefore, according to the determination result as shown in FIG. 15, it can be easily estimated that the person B is acting away from the group 1.

  Further, according to the example shown in FIG. 15, the persons D and E of the group 2 pass through the gate 1 at 10:10. Further, the persons D, E, and F of the group 2 pass through the gate 2 at 10:50. This means that the persons D and E who passed the gate 1 at 10:10 joined the person F and passed the gate 2 at 10:50. Further, the persons D and E in the group 2 pass through the gate 2 at 11:30. This means that the person F who passed the gate 2 together with the persons D and E at 10:50 is acting away from the persons D and E at 11:30. Therefore, according to the data table as shown in FIG. 15, it can be inferred that the person F only accidentally passed the gate 2 together with the persons D and E of the group 2.

  According to the group management as described above, it is possible to easily determine when each person in each group has passed through the gate together with whom, and it is possible to easily guess the behavior of each person as a group. For example, in the person monitoring system to which the passer-by identification system is applied, not only the action of each person (passage history of each gate, etc.) but also the action for each group can be monitored.

In the passer-by identification system according to the fourth embodiment, passer's face feature information detected from an image captured by the camera is collated with face information of each registrant's face stored in advance in the storage unit. Thus, the passerby is recognized, and the group of each passerby is determined. In the passer-by identification system, the person recognition result or group determination result as described above is stored in the data table as history information, and the data stored in the data table is output to an external device as appropriate. .
Thereby, according to 4th Embodiment, each passerby can be monitored for every group, and the action of each person of each group can be monitored easily.

Hereinafter, fifth to ninth embodiments will be described.
In the fifth to ninth embodiments described below, it is assumed that a person monitoring system for monitoring persons in a plurality of areas is assumed as an operation mode of the passer-by identification system. FIG. 16 is a diagram illustrating a setting example of a plurality of monitoring areas monitored by the monitoring system as the passer-by identification system according to the fifth to ninth embodiments. In the example shown in FIG. 16, a plurality of monitoring areas Sa1 to Sa6 are set in a certain security area SA. In the passer-by identification system according to fifth to ninth embodiments, which will be described later, cameras (cameras that capture images of each monitoring area) are set in each monitoring area Sa1 to Sa6 in one security area SA. It shall be.

Next, a fifth embodiment will be described.
FIG. 17 is a diagram illustrating a configuration example of a passer-by identification system 500 according to the fifth embodiment.
As shown in FIG. 17, the passer-by identification system 500 includes a plurality of cameras 501, a plurality of passer-by identification devices 502, a monitoring device 504, a server device 505, and the like. Each passer-by identification device 502 includes an input unit 511 (511A, 511B) and a control unit 512 (512A, 512B). Furthermore, each control unit 512 (512A, 512B) includes a feature detection unit 521 (521A, 521B), a recognition unit 522 (522A, 522B), and the like. The server device 505 includes a storage unit 531, a history database 532, a group determination unit 533, an output unit 534, and the like.
Note that the configuration of the passer-by identification system 500 as described above can be modified as appropriate according to the operation mode and the like. For example, each configuration of the server device 505 may be provided in any passer-by identification device 502.

  In the passer-by identification system 500 shown in FIG. 17, the camera 501 (501A, 501B), the input unit 511 (511A, 511B), the output unit 513 (513A, 513B), the feature detection unit 521 (521A, 521B), and The recognition unit 522 (522A, 522B) has the same configuration as that of the camera 101, the input unit 111, the output unit 113, the feature detection unit 121, and the recognition unit 122 in the passer-by identification system 100 shown in FIG. Therefore, detailed description is omitted.

Hereinafter, each part of the passer-by identification system 500 will be described in detail.
The storage unit 531 stores registration information (identification information, facial feature information, face image, group information, etc.) related to each registrant similar to the storage unit 114 described in the first embodiment. To do. The registration information stored in the storage unit 531 is information used in a person recognition process in the recognition units 522A and 522B or a group determination process by the group determination unit 533.

  The history database 532 stores information such as the recognition result by the passer-by identification device 502A or 502B and the determination result by the group determination unit 533 as history information. For example, the history information stored in the history database 532 includes a person recognition result (information indicating a recognized person, information indicating a shooting location (monitoring area), date and time of recognition (shooting)), and each Information such as a person group determination result is stored.

  The group determination unit 533 includes a process for determining each person's group (group determination process) and a process for determining a group action for each group (group action process) based on the person recognition result by each passer-by identification device 502. Determination process). The group determination process for each person in the group determination unit 533 can be realized by the same method as the group determination process by the group determination unit 123 described in the first embodiment. Further, the group determination process for each person may be executed by each passer-by identification device 502 as in the first embodiment.

  The group behavior determination processing by the group determination unit 533 is processing for determining (detecting) the state of group behavior by a plurality of persons for each group. For example, in the fifth embodiment, the group determination unit 533 determines whether or not a plurality of persons belonging to the same group are acting in a group as the group action determination process. The group determination unit 533 determines the degree of approach between the persons. The degree of approach between the persons is a value indicating the degree of approach between the persons in the security area SA. The degree of approach is determined based on a predetermined condition. For example, the degree of approach is “existing in the same place (monitoring area) at the same time” (first condition) or “entered or entered the security area at the same time” (second condition) ) And the like are satisfied each time a condition such as) is satisfied. In the fifth embodiment, the group determination unit 533 determines the degree of approach between each person for each group in order to determine the actions of a plurality of persons (groups).

First, the collective action determination process based on the degree of approach based on the first condition will be described.
The degree of approach based on the first condition is counted up (a predetermined value is accumulated) each time a plurality of persons belonging to the same group are detected (recognized) from one input image (image of one monitoring area). Value. However, the approach of a plurality of persons does not necessarily indicate that they are acting together. In other words, even if people detected at the same time from one input image are in the same group, it may happen that they were only present at that location by chance (actually, they act together) And may be acting apart).

  In other words, it is possible to determine that “a plurality of people are together in the same place at the same time” based on the recognition result of the person in a short time, but whether or not “they are acting together”. It is difficult to judge. For this reason, in the group behavior determination process based on the degree of approach based on the first condition, the “length of time spent together” is equal to or greater than a predetermined time (or “the number of times determined to have been together” is equal to or greater than the predetermined number of times. ), It is determined that “they are acting together”. In order to determine the length of time spent together as described above, it is necessary to continuously determine whether or not a plurality of persons are in close proximity. Here, it is assumed that the group determination unit 533 determines whether or not a plurality of persons are acting together based on a data table (access map) 533a that stores the degree of approach between the persons.

FIG. 18 is a diagram illustrating an example of the approach map 533a created in the passer-by identification system according to the fifth embodiment. The example illustrated in FIG. 18 is an example of an approach map 533a indicating the degree of approach between persons in a certain monitoring area.
In the approach map 533a, a value indicating the length of time each person has been together is stored as an approach degree. For this reason, when a plurality of persons in the same group are continuously detected (recognized) from the same input image (an image obtained by photographing a certain monitoring area), the degree of approach between the persons is predetermined. It is updated (counted up) every hour. For example, if the approach degrees of a plurality of persons belonging to the same group that are continuously recognized from the same input image are updated (counted up) every minute, the approach degrees stored in the approach map 533a are: This is a value indicating how many people have been together. Specifically, if the degree of approach is counted up every minute, the person A and the person B whose degree of approach is “12” have been together in the monitoring area for 12 minutes. Is shown. According to such an approach map 533a, it is possible to easily determine who and who have been, where and for how long.

Next, the flow of the collective action determination process based on the degree of approach based on the first condition will be described.
FIG. 19 is a flowchart for explaining an example of the group behavior determination process based on the degree of approach based on the first condition.
First, the group determination unit 533 sequentially acquires recognition results from each passer-by identification device 502 or the history database 532 (step S501). Here, it is assumed that the recognition result for one person or a plurality of persons obtained from one input screen is one recognition result. When the recognition result is acquired, the group determination unit 533 refers to the registration information stored in the storage unit 531 and determines a group for all persons recognized in the recognition result (step S502). Each person's group may be determined by each passer-by identification device 502. In this case, information indicating the group of each person is included in the recognition result. In this case, it is assumed that the group determination unit 533 determines a group of all persons from such a recognition result.

  When the groups of all recognized persons are determined, the group determination unit 533 determines whether or not a plurality of persons belonging to the same group have been detected (recognized) from the same input image (step S503). When it is determined that a plurality of persons belonging to the same group have not been recognized from one input image (step S503, NO), the group determination unit 533 returns to step S501 and repeatedly executes the above-described processing.

When it is determined that a plurality of persons belonging to the same group are recognized from one input image based on the above determination (step S503, YES), the group determination unit 533 displays the approach map 533a of the monitoring area where the input image is captured. In contrast, the degree of approach between each person is counted up (step S504). In addition, when continuously updating (counting up) the degree of approach between a plurality of persons detected simultaneously in each input image, the degree of approach between these persons is updated every predetermined time. When the degree of approach between a plurality of persons belonging to the same group recognized from one input image (a plurality of persons photographed at the same time) is counted up, the group determination unit 533 determines that the degree of approach between these persons is a predetermined degree. It is determined whether or not the value has reached or exceeded (step S505). If it is determined by this determination that the degree of approach is less than a predetermined value (step S505, NO), the group determination unit 533 returns to step S501 and repeatedly executes the above-described processing.

  If it is determined by the above determination that the degree of approach has reached or exceeded a predetermined value (step S505, YES), the group determination unit 533 determines that these persons are performing group behavior (acting together). (Step S506). When it is determined that a plurality of persons are performing a collective action, the group determination unit 533 determines each of the plurality of persons determined to be performing a collective action together with information indicating a monitoring area where the plurality of persons performing the collective action are captured. Information indicating a person is supplied to the output unit 534 (step S507).

  Accordingly, the output unit 534 outputs information indicating each person determined to be performing a collective action together with information indicating the monitoring area and information indicating the group to the monitoring device 504. As a result, the monitoring device 504 can display information such as “who and who are acting as a group in which monitoring area”. In addition, the output unit 534 transmits control information for generating an alarm or the like according to a group (or a warning level of each group) determined that a plurality of persons are acting together to the monitoring device 504. You may make it output.

  In the first condition, it is determined that a plurality of persons belonging to the same group are detected in one monitoring area. That is, in the collective action determination process based on the approach degree based on the first condition, it is determined whether or not a plurality of persons are performing collective actions for each monitoring area. Based on such a determination result, the output unit 534 may perform output processing according to the importance for each monitoring area in which a plurality of persons acting in a collective manner are detected. This is because the monitoring areas Sa1 to Sa6 may have different levels of importance (security levels) even within one security area SA. In the security area SA, an area with high importance (for example, a place where there is no problem if there are several persons in a specific group) and an area with low importance (for example, if there are several persons in a specific group are wary). It is possible that there is a mixture of In such a case, by setting the importance level for each monitoring area, the passer-by identification device can perform an output process for notifying an external device of an alarm corresponding to the importance level for each monitoring area. It becomes.

  For example, if there is a room (most important area) that manages an important property as one of the monitoring areas in a building (security area) where a plurality of monitoring cameras are installed, each monitoring in the security area What is necessary is just to set the importance of an area so that it may become so high that it approaches the said most important area. In such a case, an operation can be performed in which the alarm level increases as a group of a group approaches the most important area as described above.

FIG. 20 is a diagram illustrating an example of setting importance levels for a plurality of monitoring areas.
In the example shown in FIG. 20, the importance levels of the monitoring areas Sa1, Sa2, and Sa3 are set to “3”, “1”, and “2”, respectively, and the importance level of the monitoring area San is set to “5”. Yes. For this reason, when a plurality of persons in the same group are recognized in the monitoring area San having the highest importance, a warning to that effect is given the highest priority. In the example shown in FIG. 20, a comprehensive judgment value for judging a priority order for outputting an alarm to an external device is shown. For example, as shown in FIG. 20, the comprehensive judgment value is obtained by integrating a value obtained by multiplying the importance of each monitoring area by the detected number of people for each group. In the example shown in FIG. 20, the comprehensive judgment value of group 2 is the highest value. For this reason, when the determination result as shown in FIG. 20 is obtained, the output unit 534 gives the warning that there are two persons in the group 2 in the monitoring area San to the monitoring device 504 with the highest priority. Notice.

  As described above, in the passer-by identification system, by setting the importance level in the monitoring area, output processing is performed according to the importance level of each monitoring area in which a group acting in a group is detected. This makes it possible to efficiently monitor a group acting in a group in an important monitoring area or a monitoring area close to the important area.

Next, the group behavior determination process based on the degree of approach based on the second condition will be described.
The degree of approach based on the second condition is determined based on whether or not a plurality of persons in the same group existed in the security area in the same time zone. For example, the degree of approach based on the second condition is determined based on the time when a plurality of persons belonging to the same group enter or enter the security area. In this case, the degree of approach is counted up between “a plurality of persons who entered the security area at approximately the same time” or between “a plurality of persons who entered the security area at approximately the same time” ) In other words, the approach degree based on the second condition is a value for determining that a plurality of persons belonging to the same group have acted in the same time zone within the security area.

  For example, multiple people who are acting in a short period of time, such as handing over objects or waiting for them, can enter from different entrances or enter from different entrances with a short approach. It is possible. In other words, it is possible that a plurality of persons aiming at delivery of objects or meeting are acting as a group, but hardly acting together.

  In such a case, it is difficult to determine that the persons are acting together with the degree of approach based on the first condition. On the other hand, in the degree of approach based on the second condition, a plurality of persons belonging to the same group (entered into or out of the security area) existed in the security area at least in the same time zone. Can be identified. In the following description, it is assumed that the degree of approach based on the second condition is determined by the entry time or entry time of each person in the security area.

  In addition, it is assumed that the degree of approach based on the second condition is managed using the approach map 533a as described above. For example, it is assumed that the security area includes a plurality of monitoring areas provided with doorways. In this case, for a plurality of persons belonging to the same group who entered the security area (or entered from the security area) at the same time zone in a plurality of monitoring areas with entrances / exits, the access degree of the approach map 533a is increased (counting up) To do). According to such an approach map 533a, it is possible to easily determine a plurality of persons in the same group who entered and exited in the same time zone.

  Here, as shown in FIG. 16, in the security area SA, it is assumed that entrances and exits are provided in the monitoring areas Sa1, Sa2, and Sa3. For example, if the time when the person A is detected in the monitoring area Sa1 and the time when the person B is detected in the monitoring area Sa2 are within a predetermined time, and the person A and the person B are in the same group, the group The determination unit 533 integrates the degree of approach between the person A and the person B in the approach map 533a. According to such processing, the approach map 533a has a high degree of approach between persons entering and leaving the security area SA during the same time period. When these results are integrated and become a predetermined value or more, group determination unit 533 determines that there are a plurality of persons in the same group.

  Further, in each of the monitoring areas Sa1, Sa2, and Sa3 that are the entrances and exits of the security area SA, a camera that shoots a person who enters and a camera that shoots a person who enters the area may be installed. In this case, the person detected from the image taken by the camera that shoots the person to be entered is the person entering the security area SA, and the person detected from the image taken by the camera that shoots the person to be entered is the security area SA. Is a contestant. With such a configuration, it is possible to easily distinguish a person who enters the security area from a person who enters the security area.

Next, the flow of the collective action determination process based on the approach degree based on the second condition will be described.
FIG. 21 is a flowchart for explaining an example of a collective action determination process based on an approach degree based on the second condition.
First, the group determination unit 533 sequentially acquires recognition results from each passer-by identification device 502 or the history database 532 (step S521). In the case where only the entry time or entry time to the security area is to be determined, the group determination unit 533 applies the image taken in any of the monitoring areas Sa1, Sa2, and Sa3 that have an entrance to the security area SA. Only the recognition result may be acquired.

When the recognition result is acquired, the group determination unit 533 refers to the registration information stored in the storage unit 531 and determines the group of each person recognized in the recognition result (step S522). Each person's group may be determined by each passer-by identification device 502.
When the group of each person is determined, the group determination unit 533 determines whether a person who belongs to the same group as the person within a predetermined time from the time when each person in the recognition result is detected is any monitoring area in the security area SA. It is determined whether or not it has been detected (recognized) (step S523). For example, when the entry time or the entry time to the security area is to be determined, the group determination unit 533 determines whether or not the same group of persons are detected within a predetermined time in the monitoring areas Sa1, Sa2, and Sa3 having the entrances and exits. Determine whether.

  When it is determined by the above determination that no person belonging to the same group has been detected within a predetermined time (step S523, NO), the group determination unit 533 returns to step S521 and repeatedly executes the above-described processing. When it is determined that a person belonging to the same group is detected within a predetermined time by the above determination (step S523, YES), the group determination unit 533 determines the person recognized in the recognition result and the person within the predetermined time. The degree of approach with a person belonging to the same group detected in step S524 is counted up (step S524).

  When the degree of approach between a plurality of persons belonging to the same group detected within a predetermined time is counted up, the group determination unit 533 determines whether the degree of approach between those persons has become a predetermined value or more. (Step S525). When it is determined by this determination that the degree of approach is less than a predetermined value (step S525, NO), the group determination unit 533 returns to step S521 and repeatedly executes the above-described processing.

  If it is determined by the above determination that the degree of approach has reached a predetermined value or more (step S525, YES), the group determination unit 533 may cause those persons to act as a group in the security area SA. It is determined that there is (step S526). When it is determined that there is a possibility that a plurality of persons in the same group are acting together, the group determination unit 533 includes information indicating those persons who may be acting as a group and information indicating the group Are supplied to the output unit 534 (step S527).

  Accordingly, the output unit 534 outputs information indicating each person in the same group detected in any monitoring area within the security area to the monitoring device 504 within a predetermined time. As a result, for example, the monitoring device 504 can display information such as “who is in the same group and who is in the security area”. In addition, the output unit 534 outputs control information indicating that an alarm or the like is generated according to a group (or a warning level of each group) determined that there is a possibility that a plurality of persons are acting together. You may make it output to the monitoring apparatus 504. FIG.

  According to the degree of approach based on the second condition, even if the time to act together is short, a plurality of persons belonging to the same group that tend to exist in the security area at the same time zone act as a group It can be determined that there is a possibility. Furthermore, it is also possible to detect and notify a plurality of persons who may be performing a collective action in the security area based on each person's entry / exit pattern (entry time or entry time).

  The determination of the degree of approach according to the second condition can be applied only to a specific group. For example, the approach degree may be determined based on the second condition only for a group with a high alertness level. Thereby, it can be detected that a plurality of persons belonging to a group with a high degree of vigilance are acting in the same time zone within the security area. If a plurality of persons belonging to a group with a high degree of vigilance are present in the security area at the same time, it is predicted that there is a high possibility that these persons will take actions with high risk. As a result, for each person who belongs to a specific group with a high degree of vigilance, it is possible to support sufficient vigilance by determining the degree of approach under the second condition as described above.

  Note that it is determined whether a plurality of persons belonging to the same group are collectively acting in the security area by combining the approach degree based on the first condition and the approach degree based on the second condition. You may do it. In this case, the approach degree based on the first condition and the approach degree based on the second condition may be counted up in one approach map.

  When the passer-by identification system 500 as described above detects that a plurality of persons belonging to the same group are continuously present together in a given area for a predetermined period (time or the number of times recognized), It is determined that they are acting together, and the determination result is output to an external device. In addition, when a plurality of persons belonging to the same group are detected in an area such as a security area at the same time zone, a plurality of persons belonging to the same group exist in the security area at the same time zone (or the same group A plurality of persons belonging to the same behavior pattern), and a message to that effect is output to the external device.

  As a result, the monitor can easily recognize the group of each group acting together, and can accurately respond to the behavior of each group. Watchers can easily recognize that a group with a high degree of vigilance is acting as a group or that there is a high possibility that they are acting as a group, and quickly prepare an accurate security system. Is possible. It is also possible to output an alarm according to the group and the number of people acting together and the importance of each monitoring area. For this reason, the supervisor can recognize the warning level intuitively and easily by the warning according to the warning level.

Next, a sixth embodiment will be described.
FIG. 22 is a block diagram illustrating a configuration example of a passer-by identification system 600 according to the sixth embodiment.
As shown in FIG. 22, the passer-by identification system 600 includes a plurality of cameras 601 (601A, 601B), a plurality of passer-by identification devices 602 (602A, 602B), a monitoring device 604, a server device 605, and the like. ing. Each passer-by identification device 602 includes an input unit 611 (611A, 611B) and a control unit 612 (612A, 612B). Furthermore, each control unit 612 (612A, 612B) includes a feature detection unit 621 (621A, 621B), a recognition unit 622 (622A, 622B), and the like. The server device 605 includes a storage unit 631, a history database 632, a group determination unit 633, an output unit 634, a relevance determination unit 635, and the like. Note that the configuration of the passer-by identification system 600 as described above can be modified as appropriate according to the operation mode and the like. For example, each configuration of the server device 605 may be provided in any passer-by identification device 602.

  In addition, the camera 601, the input unit 611, the control unit 612, the storage unit 631, the history database 632, the group determination unit 633, and the output unit 634 in the passer-by identification system 600 shown in FIG. Since the configuration is the same as that of the camera 501, the input unit 511, the control unit 512, the storage unit 531, the history database 532, the group determination unit 533, and the output unit 534 in the person recognition system 500, detailed description thereof is omitted.

  The passer-by identification system 600 shown in FIG. 22 includes a relevance determination unit 635 that determines the relevance between persons from the history information of the recognition result in the server device of the passer-by identification system 500 shown in FIG. It has become. The relevance determination unit 635 performs relevance determination processing similar to the relevance determination unit 333 illustrated in FIG. 10 described in the third embodiment. The relevance determination unit 635 votes (accumulates) an evaluation value indicating the relevance between each person in the correlation map 635a by the same method as the relevance determination unit 333 described in the third embodiment. Process.

FIG. 23 is a flowchart for explaining an example of the flow of the relevancy determination process performed by the relevance determination unit 635 according to the sixth embodiment. Note that the processing in steps S601 to S609 in the flowchart shown in FIG. 23 is the same as the processing in steps S301 to S309 in the flowchart shown in FIG. 13 described in the third embodiment. Omitted.
That is, in the relevance determination unit 635, a group to which one person belongs is assigned to a plurality of persons determined to have high relevance (a plurality of persons whose relevance evaluation value is equal to or greater than a predetermined value). It is determined that the other person should be added to (step S610). In other words, the relevance determination unit 635 is characterized in that the relevance determination unit 635 performs grouping on persons whose relevance evaluation value is equal to or greater than a predetermined value based on the current group information of the persons. It is different from 333.

  For example, when the evaluation value of the relevance between the person A and the person B is equal to or greater than a predetermined value, the relevance determination unit 635 determines that the person A and the person B are likely to be in the same group. When it is determined that the person A and the person B seem to be in the same group, the relevance determination unit 635 adds the group 1 to the group information of the person B or the person B if the person A belongs to the group 1. Is notified to the monitoring apparatus 604.

FIG. 24 is a diagram illustrating an example of history information of recognition results by each passer-by identification device 602. FIG. 25 is a diagram illustrating an evaluation example of relevance based on the history information of the recognition result illustrated in FIG.
The history information of recognition results as shown in FIG. 24 is obtained by using the method described in the third embodiment (comparison between each recognition result and the immediately preceding recognition result). Relevance among registrants) is evaluated. In the example shown in FIG. 24, it is determined that the relevance between the registrant A and the registrant B is level 2, and the relevance between the registrant B and the registrant C, and between the registrant B and the unregistered person X is level. 1, the relevance between the unregistered person X and the registrant C is determined to be level 3, and the relevance between the registrant D and registrant E is determined to be level 3. When the evaluation of the relationship between each person is arranged, a correlation map 635a as shown in FIG. 25 is obtained.

  In the example shown in FIG. 24, the relevance of a person who has been determined as an unregistered person X is also evaluated with a person who has been recognized as another registrant. In the example shown in FIG. 24, the unregistered person X evaluates that the relevance with the registrant C recognized simultaneously (recognized from one input image) is high. Registration information such as facial feature information is not stored in the storage unit 631 for such unregistered persons. For this reason, group information stored in the storage unit 631 cannot be automatically updated for unregistered persons. That is, in order to group an unregistered person with another person (registrant), a monitor or system administrator determines whether or not the registration information (face feature information, etc.) of the unregistered person should be stored in the storage unit 631. Need to be judged. Therefore, for unregistered persons who are evaluated as having high relevance to the registrant, information indicating the relevance to each registrant and the face image of the person (unregistered person) detected in the recognition process are also included. The information is displayed on the monitoring device 604. Thereby, for the unregistered person, the registered information can be registered in the storage unit 631 and group information based on the relevance with other registrants can be set based on the judgment by the monitor or the system administrator.

  As for the unregistered person, an ID number may be assigned to the facial feature information detected from the input image and stored in the storage unit 631 as unregistered person information (temporary registration information). In this case, the unregistered person can be collated with facial feature information detected from the input image as an unregistered person with the ID number in the subsequent recognition process in the system. Thereby, even if it is an unregistered person, it becomes possible to continuously evaluate the relevance with other registrants or other unregistered persons to which ID numbers are assigned.

  Also, the group information of registrants determined to be highly relevant may be updated based on the judgment by the supervisor or system administrator. In this case, the relevance determination unit 635 causes the monitoring device 604 to display information regarding a plurality of persons evaluated as having high relevance. When a supervisor or system administrator who has looked at such a display registers each person evaluated as highly relevant as a group, he or she instructs the operation unit 604a. When the monitor or the system administrator gives an instruction to update the group information of the displayed person, the monitoring device 604 requests the relevance determination unit 635 to update the group information of the person. When receiving an update request for the group information of each person from the monitoring device 604, the relevance determination unit 635 updates the group information of the person stored in the storage unit 631. As a result, it is possible to update the group information of each person after obtaining visual confirmation by the supervisor or the system administrator. In this case, since the monitoring person or the system administrator can visually confirm the updated contents of the group information, reliable group management is possible.

Next, a seventh embodiment will be described.
The seventh embodiment is a first modification of the sixth embodiment. For this reason, the passer-by identification system 700 according to the seventh embodiment is configured as shown in FIG. 22 described in the sixth embodiment. However, in the passer-by identification system 700 according to the seventh embodiment, as registration information of an authenticated registrant, group information indicating a special group (authenticated group) different from other groups is included. It shall be remembered. In the seventh embodiment, when the relationship between the authenticated group person and the other group person becomes high, the evaluation value of the relationship is reset according to a special rule. Or updating group information.

FIG. 26 is a diagram illustrating an example of registration information stored in the storage unit 631 of the passer-by identification system 700 according to the seventh embodiment.
In the example shown in FIG. 26, in addition to the registration information shown in FIG. 3 described in the first embodiment, information indicating a group that is “authenticated” is added. It is assumed that a person who belongs to the “authenticated” group is a person who can be reliably trusted in managing the security area, such as a system administrator or a security guard. For example, a person who is acting with an authenticated person (that is, a person who has been evaluated as highly relevant to an authenticated person) is automatically registered as an authenticated group, a suspicious person, etc. It is possible to operate so as to avoid registering with a negative group (a group with high alertness).

FIG. 27 is a flowchart for explaining an example of the flow of relevance determination processing by the relevance determination unit 635 according to the seventh embodiment. Note that the processing in steps S701 to S709 in the flowchart shown in FIG. 27 is the same as the processing in steps S301 to S309 in the flowchart shown in FIG. 13 described in the third embodiment. Omitted. In addition, the process in step S712 in FIG. 27 is the same as the process in step S610 in FIG. 23 described in the sixth embodiment, and thus detailed description thereof is omitted.
That is, in the relevance determination unit 635 according to the seventh embodiment, “authenticated” for a plurality of persons determined to have high relevance (a plurality of persons whose relevance evaluation value is equal to or greater than a predetermined value). It is determined whether or not a person belonging to the group (an authenticated person) is included (step S710). If it is determined by this determination that an authenticated person is included (step S710, YES), the relevance determination unit 635 determines the evaluation values of the plurality of persons (that is, the persons belonging to the “authenticated” group). (Evaluation value) is reset (step S711).

  As a result, even if the person who belongs to the “authenticated” group is evaluated to be highly relevant to a specific group (for example, the group of the group requiring attention in the example shown in FIG. 26), Information is reset. That is, the passer-by identification system 700 does not evaluate the relevance of other persons belonging to the “authenticated” group. Note that it is also possible not to evaluate the relevance of a person who is determined to be highly relevant to a person belonging to the “authenticated” group.

  The authenticated group is a special group different from other groups. For this reason, in order to prevent the passer-by identification system from automatically registering people in other groups in the authenticated group, the evaluation value indicating the relationship with the person belonging to the authenticated group is correlated. You may avoid voting on the map. However, the operation mode of the passer-by identification system is set according to the security policy. For this reason, if it is permitted in the security policy to automatically add a person in another group to the authenticated group, the person who belongs to the authenticated group also shows relevance to another person. The evaluation value may be voted on the correlation map.

  As described above, the passer-by identification system according to the seventh embodiment belongs to a special group such as an authenticated group in addition to the processing of the passer-by identification system described in the sixth embodiment. Do not evaluate the relationship between each person and other persons. As a result, in the seventh embodiment, a person (authenticated person) who is clearly trusted can be mistakenly registered in an inappropriate group, or another person can be registered as an authenticated group. It becomes possible to prevent. As a result, group management for each person in the security area can be performed with high accuracy.

Next, an eighth embodiment will be described.
The eighth embodiment is a second modification of the sixth embodiment. For this reason, the passer-by identification system 800 according to the eighth embodiment is configured as shown in FIG. 22 described in the sixth embodiment. However, in the passer-by identification system 800 according to the eighth embodiment, the method for determining the relationship between the persons is different from that described in the sixth embodiment. That is, in the eighth embodiment, the relevance between detected (recognized) persons is increased in the same time zone even in separate monitoring areas in the security area.

  For example, even if persons who have entered or participated at different entrances and exits have the same time zone for entering or participating in the security area, the relevance of those persons is increased. Such a relevance determination method is similar to the approach determination processing based on the second condition described in the fifth embodiment. That is, in the passer-by identification system according to the eighth embodiment, the system was present in the security area during the same time period, entered the security area during the same time period, or entered the security area during the same time period. The relevance between each person is evaluated based on the similarity of each person's action pattern.

  Thereby, in the passer-by identification system according to the eighth embodiment, it is possible to evaluate the relevance even between persons who are approaching for only a short time within the security area, such as by waiting or handing over objects. In other words, in the eighth embodiment, even people acting separately in the security area (people who are not simultaneously detected in the same monitoring area) are present in the security area at the same time zone. Relevance is increased between persons who have a tendency (persons with similar behavior patterns).

  Next, an example of processing performed by the relevance determination unit 635 of the passer-by identification system according to the eighth embodiment will be described.

FIG. 28 is a flowchart for explaining relevance determination processing by the relevance determination unit 635 of the passer-by identification system according to the eighth embodiment.
First, the relevance determination unit 635 sequentially acquires recognition results from each passer-by identification device 602 or the history database 632 (step S801). Here, the recognition result stored in the history database 632 includes at least information such as date and time (shooting date of input image), monitoring area (shooting location of input image), and information indicating a recognized person. It shall be included. In the case where only a person who enters or participates in the security area is a target for determining the relevance, the relevance determination unit 635 has any of the monitoring areas Sa1, Sa2, and Sa3 that have an entrance or exit to the security area SA. Only the recognition result for the image photographed in step 1 may be acquired.

  When the recognition result is acquired, the relevance determination unit 635 detects (recognizes) another person in any monitoring area in the security area SA within a predetermined time based on the date and time included in the recognition result. It is determined whether or not (step S802). When only the entry time or the entry time to the security area is to be determined, the relevancy determination unit 635 may change another person within a predetermined time in any of the monitoring areas Sa1, Sa2, and Sa3 having an entrance or an exit. Is detected (recognized).

  If it is determined by the above determination that another person has not been detected within a predetermined time (step S802, NO), the relevance determination unit 635 returns to step S801 and repeatedly executes the above-described processing. Further, when it is determined by the above determination that another person is detected within a predetermined time (step S802, YES), the relevance determination unit 635 determines the relevance between the persons detected within the predetermined time. The evaluation value is increased (that is, the value between the persons in the correlation map 635a is counted up) (step S803). Even when a plurality of persons are detected within a predetermined time in the same monitoring area, the evaluation value of the relevance between these persons may be counted up. In this case, the correlation map 635a may be integrated with a value larger than the evaluation value for persons detected in different monitoring areas. As a result, a correlation map 635a in which a weighted value is voted according to the detected monitoring area is created.

  When the evaluation value between a plurality of persons detected within a predetermined time is counted up, the relevance determination unit 635 determines whether or not the evaluation value between those persons has become a predetermined value or more (step S804). ). If it is determined by this determination that the evaluation value is less than the predetermined value (step S804, NO), the relevance determination unit 635 returns to step S801 and repeatedly executes the above-described processing.

  If it is determined by the above determination that the evaluation value is equal to or greater than a predetermined value (step S804, YES), the relevance determination unit 635 determines that there is a high possibility that those persons are in the same group (step S4). S805). When it is determined that the groups are likely to be the same group, the relevance determination unit 635 performs a process of updating the group information of those persons stored in the storage unit 631 (step S806). As described in the sixth embodiment, the group information update process is performed for the person stored in the storage unit 631 so that the other person is added to the group to which the one person belongs. Group information shall be updated.

  In step S806, instead of the process of updating the group information, the determination result as described above may be output to the monitoring apparatus 604 as an external apparatus by the output unit 634. In this case, the relevance determination unit 635 may update the group information in response to a request from the monitoring device 604.

  In addition, by repeating the above process for a predetermined period, the correlation map statistically includes a plurality of persons who tend to exist in the security area in the same time zone, or entrances and exits in the same time zone. Information indicating a plurality of persons who tend to go in and out many times is accumulated. According to such a correlation map, it is determined that a plurality of persons existing in the security area in the same time zone, or a plurality of persons entering and exiting the entrance many times in the same time zone are likely to be in the same group. Is possible.

  As in the fifth embodiment, each monitoring area provided with an entrance / exit is provided with a camera for photographing a person entering the security area and a camera for photographing a person entering the security area. Anyway. In this case, it is possible to distinguish between a person who enters and a person who participates.

  According to the eighth embodiment, even people who have a short time to act together, such as meeting or handing over objects, tend to enter and exit the security area within the same time zone. Can be determined. This is effective in detecting a plurality of persons who seem to be performing collective actions in the security area based on the entry / exit patterns of each person.

Next, a ninth embodiment will be described.
The ninth embodiment is, for example, a modification of the fifth embodiment. For this reason, the passer-by identification system 900 according to the ninth embodiment is configured as shown in FIG. 17 described in the fifth embodiment. The passer-by identification system 900 according to the ninth embodiment can detect a group including a plurality of persons including a person (or a specific person) belonging to a specific group, even if the plurality of persons do not belong to the same group. In the case of an alarm, an alarm is notified to the monitoring device.

  In the fifth embodiment, a plurality of persons belonging to the same group are detected. For this reason, in the said 5th Embodiment, when the some person who belongs to a specific group is detected, the operation | movement which notifies a warning to a monitoring apparatus is possible. In contrast, in the ninth embodiment, a plurality of persons including persons belonging to a specific group are detected. Thereby, in the ninth embodiment, when at least one person belonging to a specific group is detected, an operation of notifying the monitoring device of an alarm is possible.

  That is, in the passer-by identification system 900 according to the ninth embodiment, the group determination unit 533 determines the group of each person based on the person recognition result by each of the passer-by identification devices 502 (group determination). Processing) and processing for determining group behavior for each group (group behavior determination processing). The group determination process for each person in the group determination unit 533 can be realized by the same method as the group determination process by the group determination unit 123 described in the first embodiment. Further, the group determination process for each person may be executed by each passer-by identification device 502 as in the first embodiment.

  The group behavior determination process by the group determination unit 533 is a process for determining whether or not a plurality of persons including persons belonging to a specific group are performing a group action. For this reason, the group determination unit 533 approaches a person (or a specific person) belonging to a specific group and another person (an unregistered person, a person belonging to another group, or a person who does not belong to the group). The degree shall be determined. As described in the fifth embodiment, the approach degree is determined based on a predetermined condition. The conditions for determining the degree of approach are, for example, “I was in the same place (monitoring area) at the same time” (first condition), or “I entered or entered the security area at the same time zone” ( The second condition). Here, a process of determining whether or not a plurality of persons including persons belonging to a specific group are performing a collective action based on the degree of approach based on the first condition will be described.

FIG. 29 is a diagram illustrating an example of the approach map 533a created in the passer-by identification system 900 according to the ninth embodiment. The example illustrated in FIG. 29 is an example of an approach map 533a indicating the degree of approach between persons in a certain monitoring area.
In the example of the approach map 533a shown in FIG. 29, as in FIG. 18, a value indicating the length of time each person has been together is stored as the approach degree. That is, when a plurality of persons in the same group are detected (recognized) continuously (successively) from the same input image (an image of a certain monitoring area), the degree of approach between these persons is determined for a predetermined time. Every time it is updated (counted up). In the example of the approach map 533a shown in FIG. 29, each person's group and the alarm level associated with each group are set. Furthermore, in the example of the approach map 533a shown in FIG. 29, the degree of approach is also voted for unregistered persons.

  For example, in the example shown in FIG. 12, the degree of approach between the person E belonging to the group requiring attention and the unregistered person A is high. If the proximity between the person E belonging to the group requiring attention (specific group) and the unregistered person A exceeds a predetermined threshold, the group determination unit 533 of the passer-by identification system 900 uses the person E and It is determined that the unregistered person A is performing a collective action. In this case, the output unit 534 notifies the monitoring device (external device) 504 of an alarm of alarm level 5 associated with the group requiring attention (a specific group).

  As for an unregistered person, when it is first detected, the face image of the unregistered person, the facial feature information, and the ID number (identification information) that is automatically issued are the information on the unregistered person ( Suppose that it is memorize | stored in the memory | storage part 531 as provisional registration information. As a result, when each unregistered person is detected for the second time or later, it can be recognized by the information of the unregistered person as described above.

  Next, the flow of collective action determination processing as the ninth embodiment will be described.

FIG. 30 is a flowchart for explaining an example of the group behavior determination process based on the degree of approach based on the first condition.
First, the group determination unit 533 sequentially acquires recognition results from each passer-by identification device 502 or the history database 532 (step S901). When the recognition result is acquired, the group determination unit 533 refers to the registration information stored in the storage unit 531 and determines a group for all persons recognized in the recognition result (step S902). When the groups of all recognized persons are determined, the group determination unit 533 determines whether or not a plurality of persons including persons belonging to a specific group are recognized from one input image (step S903). When it is determined that a plurality of persons including persons belonging to a specific group have not been recognized from one input image (NO in step S903), the group determination unit 533 returns to step S901 and repeatedly executes the above-described processing. .

  When it is determined that a plurality of persons including a person belonging to a specific group have been recognized from one input image based on the above determination (step S903, YES), the group determination unit 533 detects the monitoring area where the input image is captured. In the approach map 533a, the degree of approach of another person to a person belonging to a specific group is counted up (step S904). In addition, when the degree of approach between a plurality of persons recognized simultaneously in one input image is continuously updated (counted up), the degree of approach between these persons is updated every predetermined time. .

Further, when the degree of approach between a plurality of persons including persons belonging to a specific group recognized from one input image is counted, the group determination unit 533 determines that the degree of approach between these persons is equal to or greater than a predetermined value. It is determined whether or not (step S905).
If it is determined that the degree of approach is less than the predetermined value (NO in step S905), the group determination unit 533 returns to step S901 and repeatedly executes the above-described processing.

  Further, when it is determined by the above determination that the degree of approach has become a predetermined value or more (step S905, YES), the group determination unit 533 determines the level of alarm for collective action by those persons (step S906). . Note that the alarm level corresponding to each group is set in association with the group. If the group 3 is set to the alarm level “5”, the group determination unit 533 determines that the alarm level “ Information for transmitting the alarm “5” to the monitoring device 504 is supplied to the output unit 534. As a result, the output unit 534 transmits an alarm of the alarm level “5” to the external device.

  As described above, in the ninth embodiment, it is determined whether or not a plurality of persons including persons belonging to a specific group are acting as a group, and a plurality of persons including persons belonging to a specific group are grouped. When it is detected that the user is acting, an alarm is notified to an external device at an alarm level associated with the specific group.

  As a result, it is possible to detect that a plurality of persons including a specific person or a person belonging to a specific group are acting together, and further, it is possible to issue an alarm at an alarm level associated with a specific group. Become. In particular, it can be detected that an unregistered person and a person belonging to a specific group are acting together. As a result, for example, a group of suspicious persons can be effectively guarded, and high security can be realized.

  Note that the passer-by identification system according to each of the above-described embodiments may operate in cooperation with a system installed in another security area in the vicinity. For example, a passer-by identification system that monitors a specific area as shown in FIG. 16 shares information such as registrant information or history information in cooperation with systems installed in other security areas in the vicinity. It is also possible. In such a form, for example, the passer-by identification device or server device of each embodiment described above is provided with a function for communicating with other systems, and the information stored in the storage unit in the passer-by recognition device or server device. Etc. can be realized by sharing with other systems.

1 is a block diagram schematically showing a configuration example of a passer-by identification system according to a first embodiment. The figure which shows the example of the face area | region detected from the input image. The figure which shows the example of registration information. The figure which shows the example of the result of a group determination process. The flowchart for demonstrating the flow of a process in the passer-by identification system which concerns on 1st Embodiment. The block diagram which shows roughly the structural example of the passer-by identification apparatus which concerns on 2nd Embodiment. The figure which shows the example of registration information. The figure which shows the example of the result of a group determination process. The flowchart for demonstrating the flow of a process in the passer-by identification system which concerns on 2nd Embodiment. The block diagram which shows roughly the structural example of the passer-by identification system which concerns on 3rd Embodiment. The figure which shows the process example of a relevance determination part. The figure which shows the example of the correlation map produced by the relevance determination part. The block diagram which shows roughly the structural example of the passer-by identification system which concerns on 4th Embodiment. The figure which shows the process example of a group management part. The figure which shows the example of group management information. The figure which shows the example of a setting of a security area and a some monitoring area. The figure which shows the structural example of the passer-by identification system which concerns on 5th Embodiment. The figure which shows the example of the approach degree memorize | stored in an approach map. The flowchart for demonstrating the example of the determination process of collective action. The figure which shows the example of a setting of the importance with respect to each monitoring area. The flowchart for demonstrating the example of the determination process of collective action. The block diagram which shows the structural example of the passer-by identification system which concerns on 6th Embodiment. The flowchart for demonstrating the example of the flow of the determination process of the relationship by the relationship determination part of 6th Embodiment. The figure which shows the example of the log | history information of the recognition result by each passer-by identification apparatus. The figure which shows the evaluation example of the relationship based on the historical information of a recognition result. The figure which shows the example of the registration information in the passer-by identification system which concerns on 7th Embodiment. The flowchart for demonstrating the example of the flow of the determination process of the relationship by the relationship determination part of 7th Embodiment. The flowchart for demonstrating the determination process of the relevance in the passer-by identification system which concerns on 8th Embodiment. The figure which shows the example of the approach degree memorize | stored in an approach map. The flowchart for demonstrating the example of the determination process of the collective action which concerns on 9th Embodiment.

Explanation of symbols

  100, 200, 300, 400, 500, 600, 700, 800, 900 ... Passer recognition system, 101, 201, 301, 401, 501, 601 ... Camera, 102, 202, 302, 402, 502, 602 ... Passage Person recognition device 103, 203, 303, 403 ... Gate device, 104, 204, 304, 404, 504, 604 ... Monitoring device, 111, 211, 311, 411, 511, 611 ... Input unit, 112, 212, 312 412, 512, 612 ... control unit 113, 213, 313, 413, 534, 634 ... output unit, 114, 214, 314, 431, 531, 631 ... storage unit, 121, 221, 321, 421, 521, 621 ... Feature detection unit, 122, 222, 322, 422, 522, 622 ... Recognition unit, 123, 22 323, 423, 533, 633 ... group determination unit, 333, 635 ... relevance determination unit, 333a, 635a ... correlation map (data table), 405, 505, 605 ... server device, 432, 532, 632 ... history database 433 ... Group management unit, 433a ... Data table

Claims (30)

A person recognition device for recognizing a person,
Storage means for storing biometric information of at least one registrant and group information indicating a group of each registrant,
Information acquisition means for acquiring information including biological information of a person;
Detecting means for detecting biological information for at least one person from the information acquired by the information acquiring means;
Recognizing means for recognizing a person of the biological information detected by the detecting means by comparing each biological information detected by the detecting means with each biological information stored in the storage means;
A plurality of persons sequentially recognized by the recognition means are classified into group candidates based on a predetermined condition, and each person of the group candidates is grouped based on group information of each person stored in the storage means. A group determination means;
Output means for outputting the result of grouping by the group determination means to an external device;
A person recognition apparatus characterized by comprising: The output means performs a process of outputting display data for displaying a recognition result by the recognition means and a determination result by the group determination means to a display device.
The person recognition apparatus according to claim 1, wherein: The storage means stores attribute information composed of various information associated with the biometric information of the registrant as group information,
The group determination means groups each person of the group candidate based on attribute information of each person stored in the storage means;
The person recognition apparatus according to claim 1 or 2, characterized in that Further, the relevance for evaluating the relevance between a plurality of persons sequentially recognized by the recognizing means and updating the group information of each person stored in the storage means based on the relevance evaluation between the respective persons. Having a judging means,
The person recognition apparatus according to claim 1 or 2, characterized in that Furthermore, based on the determination result by the group determination unit, it has a group management unit for managing information indicating each person recognized by the recognition unit for each group,
The output means outputs information indicating each person recognized by the recognition means managed for each group by the group management means;
The person recognition apparatus according to claim 1 or 2, characterized in that A person recognition device for recognizing a person,
Storage means for storing biometric information of at least one registrant and group information indicating a group of each registrant,
Information acquisition means for sequentially acquiring information including biological information of persons existing in each monitoring area for each of a plurality of monitoring areas in one area;
Detection means for detecting biological information for at least one person from each piece of information acquired by the information acquisition means;
Recognizing means for recognizing a person of the biological information detected by the detecting means by comparing each biological information detected by the detecting means with each biological information stored in the storage means;
Based on the information of each monitoring area acquired by the information acquisition means and the recognition result by the recognition means on the biological information detected by the detection means from the information, the collective behavior of a plurality of persons in the area is determined. Group determination means to perform,
Output means for outputting the determination result by the group determination means to an external device;
A person recognition apparatus characterized by comprising: The group determination means determines that the plurality of persons are in an approached state when biometric information of a plurality of persons is detected from one information acquired by the information acquisition means.
The person recognition apparatus according to claim 6, wherein: The group determination unit determines that the persons are acting in a state of approach when biometric information of a plurality of persons is continuously detected for a predetermined period or more from each information sequentially acquired by the information acquisition unit. To
The person recognition apparatus according to claim 6, wherein: The group determination means acts in a state in which those persons are approached when biometric information of a plurality of persons belonging to the same group is continuously detected from each information sequentially acquired by the information acquisition means for a predetermined period or more. To determine that
The output means notifies an external device of an alarm of a level associated with a monitoring area in which a plurality of persons determined to be acting in a state approached by the group determination means are detected;
The person recognition apparatus according to claim 6, wherein: The group determination unit is in a state in which, when biometric information of a plurality of persons including persons belonging to a specific group is continuously detected from each information sequentially acquired by the information acquisition unit for a predetermined period or more, the persons are approaching To determine that you are acting
If the output means determines that a plurality of persons including persons belonging to a specific group are acting in close proximity by the group determination means, the output means outputs an alarm of a level associated with the specific group to the outside Notify the device,
The person recognition apparatus according to claim 6, wherein: In the case where biometric information of a plurality of persons belonging to the same group is detected in the same time period from the information of each monitoring area acquired sequentially by the information acquisition means, the group determination means is in the same time in the area. Determine that it was in the belt,
The person recognition apparatus according to claim 6, wherein: The group determination means includes biometric information of a plurality of persons belonging to the same group in the same time period from information of a plurality of monitoring areas corresponding to an entrance / exit of the area among information of each monitoring area sequentially acquired by the information acquisition means. If detected, determine that those persons were in and out of the area at the same time,
The person recognition apparatus according to claim 6, wherein: Furthermore, the relevance between a plurality of persons is evaluated based on the determination result by the group determining means, a plurality of persons that are likely to be the same group are determined based on the evaluation between the persons, and the plurality of persons that are determined to be likely to be the same group Relevance determining means for updating the group information of another person based on the group information of any one of
The person recognition apparatus according to claim 6, wherein: The relevance determining means resets the evaluation of relevance with other persons for the person determined to be in collective action with a person belonging to a special group by the group determining means.
The person recognition apparatus according to claim 13, wherein: The relevance determination means is a relevance relationship between a plurality of persons detected in the same time period from information of a plurality of monitoring areas corresponding to the entrances / exits of the area among information of each monitoring area sequentially acquired by the information acquisition means. Highly appreciated,
The person recognition apparatus according to claim 6, wherein: A person recognition method used in a person recognition device for recognizing a person,
Get information including the biological information of the person,
Detecting at least one biometric information from the acquired information,
By recognizing each detected biometric information and biometric information of at least one registrant stored in the storage unit in association with the group information, the person of the biometric information detected from the acquired information is recognized. ,
A plurality of people recognized sequentially are classified into group candidates based on a predetermined condition,
Grouping each person of the group candidates based on group information of each person stored in the storage means,
Output the result of this grouping to an external device.
A person recognition method characterized by the above. The output is a process of outputting display data for displaying the recognition result and the grouping result to a display device.
The person recognition method according to claim 16, wherein: In the storage means, attribute information consisting of various information associated with biometric information of the registrant is stored as group information,
In the grouping, each person of the group candidate is grouped based on attribute information of each person stored in the storage means.
The person recognition method according to claim 16, wherein: Furthermore, the relevance between a plurality of persons recognized sequentially is evaluated, and the group information of each person stored in the storage means is updated based on the evaluation of the relevance between the persons.
The person recognition method according to claim 16, wherein: Further, based on the result of the grouping, information indicating each recognized person is managed for each group,
The output outputs information indicating each recognized person managed for each group.
The person recognition method according to claim 16, wherein: A person recognition method used in a person recognition device for recognizing a person,
For each of a plurality of monitoring areas in one area, information including biological information of a person existing in each monitoring area is sequentially acquired,
Detecting biometric information for at least one person from the acquired information,
By comparing each detected biometric information with the biometric information of at least one registrant stored in the storage unit in association with the group information, the person of each biometric information detected from the acquired information is identified. Recognized,
Based on the information of each acquired monitoring area and the recognition result for the biological information detected from the information, determine collective behavior in a plurality of persons in the area,
Outputting the determination result of the collective behavior to an external device;
A person recognition method characterized by the above. In the determination of the collective action, when biological information of a plurality of persons is detected from the acquired one information, it is determined that the plurality of persons are in an approaching state.
The person recognition method according to claim 21, wherein: In the determination of the collective action, when biometric information of a plurality of persons is continuously detected for a predetermined period or more from each sequentially acquired information, it is determined that the persons are acting in an approaching state,
The person recognition method according to claim 21, wherein: In the determination of the collective action, when biometric information of a plurality of persons belonging to the same group is continuously detected for a predetermined period or more from each sequentially acquired information, it is determined that the persons are acting in an approaching state. And
The output notifies an external device of a level corresponding to a monitoring area in which a plurality of persons determined to behave in an approaching state are detected.
The person recognition method according to claim 21, wherein: The determination of the collective action is based on the fact that, when biometric information of a plurality of persons including persons belonging to a specific group is continuously detected from a sequentially acquired information for a predetermined period or longer, the persons are in the state of approaching each other. To determine that
When the output determines that a plurality of persons including persons belonging to a specific group are in an approaching state, the output notifies the external device of a level associated with the specific group.
The person recognition method according to claim 21, wherein: In the determination of the collective action, when biometric information of a plurality of persons belonging to the same group is detected in the same time period from the sequentially acquired information of each monitoring area, those persons are present in the area at the same time period. To determine what it was,
The person recognition method according to claim 21, wherein: The determination of the collective action is performed when biometric information of a plurality of persons belonging to the same group is detected in the same time period from information of a plurality of monitoring areas corresponding to the entrance / exit of the area among information of each monitoring area acquired sequentially. , Determine that they were in and out of the area at the same time,
The person recognition method according to claim 21, wherein: Further, the relevance between a plurality of persons is evaluated based on the determination result of the collective action, a plurality of persons that are likely to be the same group are determined based on the evaluation between the persons, and the plurality of persons that are determined to be likely to be the same group are determined. Update the group information of another person based on the group information of one of them,
The person recognition method according to claim 21, wherein: Furthermore, for the person determined to be in a collective action with a person belonging to a special group based on the determination result of the collective action, reset the evaluation of relevance with other persons.
The person recognition method according to claim 28, wherein: The relevance between a plurality of persons detected in the same time period from the information of a plurality of monitoring areas corresponding to the entrances and exits of the area among the information of each monitoring area acquired sequentially is highly evaluated.
The person recognition method according to claim 28, wherein:

Images