JP2011039959A - Monitoring system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a monitoring system for facilitating effective monitoring. <P>SOLUTION: The monitoring system is equipped with: a plurality of imaging means 1-1, 1-2, ... imaging persons who pass through a plurality places at which their incoming/outgoing are monitored, respectively; a face feature information generation means 3 generating information (face feature information) about the face features of the images of persons; an incoming/outgoing information storage means 12 storing the information (information of incoming/outgoing) including in a manner in which the information corresponding to the face feature information and the places are associated with the information of time; and an incoming/outgoing information correct/incorrect determination means 12 referring to the information (information of a device to be managed for incoming/outgoing) about the incoming/outgoing of the devices to be managed for incoming/outgoing to be stored by a system managing the devices to be managed for incoming/outgoing (e.c., IC card) carried by the persons to determine the consistency of the information about the face feature information to be a target and determine that it is correct when the consistency is obtained. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a monitoring system that realizes effective monitoring.

  In the present application, TASPO, SUICA, and NDR are registered trademarks.

Conventionally, various techniques for monitoring have been developed.
For example, a system that identifies a criminal by matching a face image using a face authentication technique in a monitoring system is considered.
In addition, a system for identifying the age and sex of a person from face image information and specifying whether or not the user is a TASPO user is considered.
In addition, in a monitoring system, there has been considered a system in which a video at the time when the supervisor himself leaves is confirmed later by time search reproduction from recorded video.

JP 2003-187352 A JP 2006-338479 A JP-A-2005-301861 JP 2003-151016 A JP 2005-196516 A JP-A-10-91378 JP 2006-154883 A

  However, in the matching processing by face authentication in the conventional monitoring system, there is a possibility that the identification is wrong due to the erroneous detection in the image processing when the criminal is identified or the person is identified. In particular, with regard to the use of TASPO, it is indispensable to prevent impersonation because it is possible to use TASPO as long as it has a card even if it is not the person himself / herself.

  In addition, it is very difficult for a monitor in the monitoring system to determine by himself / herself based on his / her own action records and memories whether the video is already monitored or unmonitored. In particular, it can be said that it is particularly difficult to specify an unmonitored video in a case where a large number of camera videos are monitored or in a system that sequentially switches and monitors camera videos.

  The present invention has been made in view of such conventional circumstances, and an object thereof is to provide a monitoring system capable of realizing effective monitoring.

In order to achieve the above object, the present invention has the following configuration in a monitoring system for monitoring the entrance and exit of a person.
That is, an imaging unit is provided at each of a plurality of locations where entry / exit is monitored, and each of the plurality of imaging units captures an image of a person passing through each location. The face feature information generating means generates face feature information that is information relating to the feature of the face image in the person image captured by the image capturing means. The entry / exit information storage means corresponds to the place where the imaging means for capturing the facial feature information generated by the facial feature information generation means and the image of the person from which the facial feature information is generated is provided. Information including information and information on the time at which the image of the person from which the facial feature information is generated is associated is stored as entry / exit information. The entry / exit information correctness determination means refers to the entry / exit managed device information that is information related to entry / exit of the entry / exit managed device stored by the system that manages the entry / exit managed device carried by the person, It is determined whether or not the entry / exit information related to the facial feature information to be stored in the entry / exit information storage means is consistent with the entry / exit managed device information. Judge that it is correct.

  Accordingly, since the entry / exit information acquired by the monitoring system is determined to be consistent with the entry / exit information acquired by the system managing the entry / exit managed device, the input / output information acquired by the monitoring system is determined. The accuracy of exit information can be improved, and effective monitoring can be realized.

Here, various modes may be used as the positions and the number of the plurality of places for monitoring entry / exit.
Further, the number of image pickup means provided at each place for monitoring entry / exit may be one or two or more. For example, two image pickup means are provided. A mode in which the imaging direction or the like is switched by time division or the like so as to image the above different places may be used.
Moreover, as an imaging means, various cameras can be used, for example.

Various types of face feature information may be used. For example, face authentication processing is performed based on a face image, and facial feature point information is obtained from the balance of the eyes, nose, and mouth of the face. The face ID that has been extracted and digitized can be used.
Further, the face feature information generating means may be provided for each imaging means, for example, or one face feature information generating means may be provided in common for two or more imaging means.

Various information may be used as the information corresponding to the location where the imaging unit that captured the image of the person from which the facial feature information is generated is provided. For example, the information is set in the imaging unit. The identification information and address information set can be used, or the identification information set for the imaging means and the location identification information set corresponding to the address information can be used.
Further, the time information when the image of the person from which the facial feature information is generated is captured is provided (for example, timed) by an imaging unit.

In addition, information other than (facial feature information, information corresponding to a place, time information) may be further included as information about the person's entry / exit stored in the entry / exit information storage means of the monitoring system. .
The entry / exit information storage means can be configured using, for example, a memory or a database.

  In addition, various devices may be used as an entry / exit managed device and a system for managing the same, for example, an external system different from the monitoring system is used, specifically, an entry / exit managed device IC cards such as SUICA are used, and systems for railways (systems that detect SUICA, etc.) are used as systems for managing them. The entry / exit information correct / incorrect determination means accesses a storage unit for storing entry / exit managed device information provided in such a system, and refers to the entry / exit managed device information.

  Further, as the face feature information that is a target for determining consistency, for example, any face feature information included in any entry / exit information stored in the entry / exit information storage unit may be used. As a specific example, in the entry / exit information stored in the entry / exit information storage means, when the entry / exit information including the same face feature information is collected, the total of persons (the same person) corresponding to the face feature information is integrated. Therefore, it is possible to use a mode of determining consistency between a group of entry / exit information about the same face feature information.

When determining the consistency between the entry / exit information in the monitoring system and the entry / exit managed device information in the system of the entry / exit managed device, the following (determination example 1) to (determination example 3) Etc. can be used.
(Judgment example 1) No person is specified for entry / exit managed device information in the system of the entry / exit managed device. In this case, if the location and time of the entry / exit information in the monitoring system and the entry / exit managed device information in the system of the entry / exit managed device match, even if the information is related to different persons Judge that.

  (Determination mode example 2) With regard to the entry / exit managed device information in the system of the entry / exit managed device, the person is specified, and the entry / exit information about the same person is put together, and this group is used for the consistency judgment. Use. In this case, if there is an entry / exit information in the monitoring system and an entry / exit managed device information in the system of the entry / exit managed device (in this example, the above-mentioned unity), the location and time unit match, Even if the information is related to a different person, it is determined that the information matches. The person in the system of the entry / exit managed device can be specified by, for example, reading and using identification information stored in the memory of the entry / exit managed device.

  (Judgment mode example 3) For the entry / exit managed device information in the system of the entry / exit managed device, the person is specified, the entry / exit information about the same person is summarized, the person is specified, and this Group and person information is used for consistency determination. Also, with regard to entry / exit information in the monitoring system, a person corresponding to each face feature information is specified, and this person information is used for consistency determination. In this case, the entry / exit information in the monitoring system (in this example, the person is specified) and the entry / exit managed device information in the entry / exit managed device system (in this example, the above-mentioned group is used to identify the person) If there is a match that includes not only the location and time, but also whether or not the person is the same, it is determined to match. That is, even if the location and time of entry / exit match, if it is a different person, it is determined that they do not match. The person corresponding to the facial feature information in the monitoring system may be specified by, for example, storing the facial feature information and information identifying the person in advance in association with each other, or if a person is an IC card or the like. When using the camera, the person captures an image of the person and obtains facial feature information based on the image, reads information identifying the person from a memory such as the IC card, and associates the information with each other. This can be done by storing it.

  As described above, according to the monitoring system of the present invention, effective monitoring can be realized.

It is a figure which shows the structural example of the monitoring system which concerns on one Example of this invention. It is a figure which shows the structural example of the surveillance camera which concerns on one Example of this invention. It is a figure which shows an example of the display screen of the to-be-monitored person information displayed by the system operation terminal which concerns on one Example of this invention. It is a figure which shows an example of a mode that the surveillance camera which concerns on one Example of this invention was installed in the ticket gate of the station. It is a figure which shows the structural example at the time of applying the monitoring system which concerns on one Example of this invention to immigration. It is a figure which shows the structural example at the time of applying the monitoring system which concerns on one Example of this invention to the reader of TASPO. It is a figure which shows the structural example of the system which discriminate | determines the monitoring image | video and the unmonitored image | video in the monitoring center which concerns on one Example of this invention. It is a figure which shows an example of the video frame output to the monitoring monitor which concerns on one Example of this invention.

Embodiments according to the present invention will be described with reference to the drawings.
FIG. 1 shows an example of a system system of a monitoring system according to an embodiment of the present invention.
The monitoring system of this example includes, for example, monitored units installed in a plurality of different locations, one monitoring unit common to the plurality of monitored units, and an IP ( (Internet Protocol) network 21.

  One monitored unit includes, for example, a plurality of cameras (surveillance cameras) 1-1, 1-2,..., A network switch 2, a face authentication server 3, a network type digital video recorder (hereinafter also referred to as NDR). 4 is provided. Here, each of the monitoring cameras 1-1, 1-2,..., The face authentication server 3, and the NDR 4 are connected to the network switch 2. The network switch 2 is connected to the IP network 21.

  The monitoring unit includes a network switch 11, a system management server 12, and a system operation terminal 13. Here, the system management server 12 and the system operation terminal 13 are connected to the network switch 11. The network switch 11 is connected to the IP network 21.

FIG. 2 shows a configuration example (example of functional blocks) of a surveillance camera according to an embodiment of the present invention.
The surveillance camera (for example, surveillance camera 1-1) of this example includes a lens 31, a CCD (Charge Coupled Device) 32, an A / D (Analog to Digital) unit 33, a DSP (Digital Signal Processor) signal processing unit 34, A primary memory 35, an image processing unit 36, a secondary memory 37, a video encoding unit 38, and an I / F (Interface) unit 39 are provided.

An example of the process performed in the surveillance camera of this example is shown.
In the surveillance camera of this example, the CCD 32 captures (captures) an image (image) through the lens 31, and the analog video signal obtained thereby is converted into a digital video signal.
The DSP signal processing unit 34 stores the digitized video signal in the primary memory 35 for each video frame. The image processing unit 36 reads a video frame from the primary memory 35 as needed, performs image processing, and detects a face image. Then, the image processing unit 36 extracts only the face image portion and stores it in the secondary memory 37.

Also, normal (for example, 30 frames / second) digitized video data is sent from the DSP signal processing unit 34 to the video encoding unit 38 without passing through the image processing unit 36, and the I / F unit 39. To the IP network 21.
In the monitoring camera of this example, a face image and acquired time data are distributed to a terminal device (for example, a monitoring unit or another communication terminal device) connected to the IP network 21.

  Further, in the monitoring camera of this example, for example, only face images and time data can be distributed according to the system, and image data distributed (transmitted) to the network as compared with a normal network type monitoring system. Can be reduced.

An example of the process performed in the monitoring system of this example is shown.
A monitoring camera (such as the monitoring camera 1-1) has a function of detecting (detecting) an image portion of a person (existing in a video frame) that appears in a video (based on a video signal that has been captured and digitized) and a time of the video ( (Acquisition time) is specified, and when a person is detected, the face image and the acquisition time information are distributed (transmitted) to the face authentication server 3.

When the face authentication server 3 acquires the face image sent from the monitoring camera and the acquisition time information, the face authentication server 3 identifies the destination monitoring camera (the monitoring camera that sent the face image etc.) from the destination IP address. Information is managed as acquisition location information.
Further, the face authentication server 3 performs face authentication processing based on the face image data, extracts facial feature point information from the balance of the eyes, nose, and mouth of the face, and further converts the feature point information to digital To generate a face ID (in this example, digitized feature point information). The face authentication server 3 records the face image data, face ID, time information (for example, acquisition time information), and acquisition location information as monitored person information in the database. At the same time, the face authentication server 3 distributes monitored person information to the system management server 12 as needed.

The system management server 12 generates entry / exit information from the received acquisition time information and acquisition location information, and records and manages it in the database together with the face ID. As an example, when applied to a railway monitoring system, if the acquisition location is the ticket gate entrance, the acquisition time is managed as the entrance time and the acquisition location is managed as the entrance station.
In the monitoring system of this example, all camera videos are recorded in NDR4.
In addition, on the monitored portion side terminal (not shown) and the system operation terminal 13, the entrance / exit information, the recorded image, and the like can be browsed.

  For example, when an incident occurs, a recorded image of a surveillance camera that captures the location of the incident is played by a terminal (or system operation terminal 13 or the like) on the monitored unit side, and when a criminal is identified, the face image And the face image data (monitored person information in this example) is transferred (transmitted) to the system management server 12. The system management server 12 extracts the face ID from the transferred monitored person information, and performs matching with many face IDs recorded in the database of the own device. This matching result and monitored person information can be sent from the system management server 12 to the system operation terminal 13 to confirm the entry / exit location and entry / exit time of the criminal.

  Here, when only the entrance information exists and the exit information does not exist, it can be determined that the criminal is still in the facility. At this time, the system management server 12 performs a real-time matching process between the face ID sent from the face authentication server 3 and the face ID of the criminal in real time, and if a criminal is detected, the system operation terminal 13 can be alerted.

  In this example, the face authentication server 3 of the monitored unit generates a face ID based on the face image data and transmits the face ID to the system management server 12 of the monitoring unit. However, as another configuration example, A configuration in which face image data is transmitted from the monitored unit to the system management server 12 of the monitoring unit and the face ID is generated based on the face image data received by the system management server 12 may be used.

FIG. 3 shows an example of a display screen of monitored person information displayed by the system operation terminal 13. In this example, an example in which the present invention is applied to a railway monitoring system is shown.
Specifically, as the monitored person information (or information based on it) 41, face ID, information on the entrance station, information on the ticket gate of the entrance station, information on the entrance time, information on the exit station, information on the ticket gate of the exit station The exit time information is displayed in association with each other, and the face image 42 corresponding to the information is displayed.
Here, the monitored person information and the like are displayed in a list in the new order. Since this list is expected to be enormous, it is possible to search for each item. For example, when a user (person) designates a place where an entry station is entered with the mouse and right-clicks, a list of station names is displayed. When a station to be searched is selected, only the entry station that is the station is displayed in the list. Is done.

FIG. 4 shows an example of a state in which the monitoring camera of this example is installed at the ticket gate of a station when the monitoring system of this example is applied to a railway monitoring system.
Specifically, a ticket gate 51, an IC card (in this example, SUICA) reader 52, a suspension type surveillance camera 53 (corresponding to the surveillance camera 1-1 in FIG. 1), a ticket gate embedded type. The monitoring camera 54 (corresponding to the monitoring camera 1-1 in FIG. 1) and the monitored person (passenger) 55 are shown.

  In the monitoring system of this example, the monitoring cameras 53 and 54 capture the customer's face when the monitored person 55 picks up the IC card when passing the ticket gate. The monitored person 55 is grasped from both upward and downward directions by the monitoring cameras 53 and 54 installed at two places. Each of the monitoring cameras 53 and 54 transfers the face image and the acquisition time information to the face authentication server 3. When the face authentication server 3 acquires two face images from the monitoring cameras 53 and 54, the face authentication server 3 performs face authentication processing to determine the face direction, generates a face ID from an image close to the front direction, and creates a database. Register with.

  When confirming the entrance / exit information, the system management server 12 makes an inquiry to the SUICA center server (not shown), and the entrance station information, entrance time information, entry station information, and entry time information are the face IDs. Check whether there is consistency with the entry / exit information based on the authentication result. When this consistency is obtained (confirmed), it is possible to provide accurate entry / exit information by determining the entry / exit information.

  Furthermore, by incorporating the disaster prevention cameras, security cameras, and vehicle entrance / exit monitoring cameras installed in the station premises as surveillance cameras for the surveillance system of this example, face authentication processing is also performed on the images of these cameras. It becomes possible to grasp the action trajectory in more detail.

FIG. 5 shows a configuration example when the monitoring system of this example is applied (applied) to immigration.
Specifically, the auditor 61, the system operation terminal 62 (corresponding to the system operation terminal 13 in FIG. 1), the traveler 63, the IC tag reader 64, the monitoring camera 65 (corresponding to the monitoring camera 1-1 in FIG. 1, etc.) ), A face authentication server 66 (corresponding to the face authentication server 3 in FIG. 1), and a system management server 67 (corresponding to the system management server 12 in FIG. 1).

In this example, when the passport is updated, a face ID is generated from the face image of the owner (traveler 63) and recorded in an IC tag built in the passport together with personal information (eg, personal ID). Keep it. The personal information and face ID are also registered in the system management server 67.
At the time of immigration, the face authentication server 66 generates a face ID from the face image of the traveler 63 captured by the monitoring camera 65 and transfers it to the system management server 67. At the same time, when the traveler 63 passes the passport to the IC tag reader 64, the personal information and face ID recorded in the passport are read and transferred to the system management server 67. The system management server 67 determines whether the two face IDs of the face ID from the face authentication server 66 and the face ID corresponding to the personal information from the IC tag reader 64 (or the face ID from the IC tag reader 64) match. In accordance with whether or not they match, it is determined whether or not the user is the person himself / herself. The determination result is transferred from the system management server 67 to the system operation terminal 62, and the system operation terminal 62 allows the inspector 61 to quickly confirm the identity.

Here, since the passport is periodically updated, it is possible to update the face ID each time, and it can be said that this example functions effectively even for changes in the face due to age and other factors.
In addition to a passport, a driver's license and the like are periodically updated with a face photo, and a system similar to this example can be constructed.

Furthermore, when an electronic money function is added to a passport, a driver's license, etc., as an example, every time electronic money is used for a passport, a driver's license, etc. for shopping, etc., the IC tag reader must The personal information and face ID are read out from the information, and the information, the place of use and the time of use are transferred to the system management server of this system. Also, for example, a face ID is generated by a face authentication server from a face image captured by a monitoring camera installed at a cash register and transferred to the system management server together with the face image. Thereby, the system management server can manage the action trajectory of the person (user such as a passport or a driver's license) together with the face image.
In addition, for example, a system in which a camera is installed in a shopping mall, image processing is performed on an unspecified number of human face images, information such as sex and age is extracted, and a purchase tendency is provided. In the configuration of this example, since personal data is directly read and managed, it is possible to reliably grasp the purchase tendency.

FIG. 6 shows a configuration example when the monitoring system of this example is applied (applied) to a TASPO (non-contact type IC card) reader.
Specifically, a customer 71, a TASPO reader 72, a monitoring camera 73 (corresponding to the monitoring camera 1-1 in FIG. 1), a face authentication server 74 (corresponding to the face authentication server 3 in FIG. 1), A system management server 75 (corresponding to the system management server 12 in FIG. 1), a system operation terminal 76 (corresponding to the system operation terminal 13 in FIG. 1), and a supervisor 77 are shown.

  In this example, when the purchaser 71 presents TASPO to the TASPO reader 72 provided in the vending machine, the monitoring camera 73 installed in the vending machine captures the face image of the purchaser 71 and performs a face authentication server. The face authentication server 74 generates a face ID from the face image and transfers the face ID to the system management server 75. Further, the TASPO reader 72 transfers information read from the TASPO (for example, personal information recorded in the TASPO) to the system management server 75. The system management server 75 compares the face ID received from the face authentication server 74 with the face ID registered at the time of updating the passport, driver's license, etc., for example, a TASPO user (in this example, a customer It is possible to determine whether or not 71) is the owner and whether or not it is an adult. In this determination, for example, personal information received from the TASPO reader 72 may be referred to as necessary.

However, the configuration of this example is effective only when the TASPO user is an owner of a passport, a driver's license, or the like.
Here, the configuration of this example can be applied to an ATM such as a bank, for example, and at the time of using an ATM, it is identified (automatically by the apparatus) whether or not the user of the ATM is the person himself / herself. It becomes possible.

FIG. 7 shows a configuration example of a system for discriminating between a monitored video and an unmonitored video (automatically by the apparatus) in the monitoring center.
Specifically, the monitor 81, the monitor camera 82 of the monitor (for example, the same as that shown in FIG. 2), the face authentication server 83, the system management server 84 (corresponding to the system management server 12 in FIG. 1). 1), a control unit 85 of the monitoring system, a monitoring camera 86 (corresponding to the monitoring camera 1-1 in FIG. 1), an image recording device 87 made of NDR (corresponding to NDR 4 in FIG. 1), a video decoder, etc. 88, a monitor 89, and a system operation terminal 90 (corresponding to the system operation terminal 13 in FIG. 1).

  In this example, the monitoring camera 82 of the monitoring person always images the monitoring person 81 from the front direction. The monitoring camera 82 transfers the face image of the monitoring person 81 to the face authentication server 83 at regular intervals. The face authentication server 83 determines the face direction of the monitor 81 based on the received face image, and transfers the face direction information to the system management server 84. If the system management server 84 determines that the face orientation is the front direction based on the information received from the face authentication server 83, the image currently displayed on the monitor monitor 89 is the one that has been monitored. If the face orientation is determined to be other than the front direction, it is determined that the image currently displayed on the monitor monitor 89 is unmonitored.

  If the system management server 84 determines that the image currently displayed on the monitoring monitor 89 is unmonitored, the system management server 84 notifies the control unit 85 of the monitoring system that the image is not being monitored. During normal times, the control unit 85 of the monitoring system always controls the video decoder 88 to switch the camera video for decoding (camera video displayed on the monitoring monitor 89). The displayed camera video (for example, identification information of the corresponding camera) is grasped, and the camera information and unmonitored time when unmonitored can be managed.

  In addition, when it is desired to provide unmonitored video in response to a request from the supervisor 81, the control unit 85 of the monitoring system selects an unmonitored camera and an unmonitored time from the video recorded by the image recording device 87. Based on the information, the video at the time (for example, the time zone) in the monitoring camera (for example, the monitoring camera 86) is extracted by time search and output to the video decoder 88. The video decoder 88 displays the video at this time on the monitor monitor 89.

FIG. 8 shows an example of a video frame output to the monitoring monitor 89.
Specifically, regarding the images taken by the three monitoring cameras (cameras 1, 2, and 3), the state of a plurality of continuous frames constituting the images is shown, and the monitored image 101 and the non-monitored image are shown. An unmonitored video 102 for display and an unmonitored video 103 because the monitor 81 is facing in a direction other than the front direction are shown.

  In normal times, for example, images from three cameras are sequentially displayed on one monitor monitor 89, and a video frame (shown as a monitored video) indicated by hatching in FIG. 2, camera 3, camera 1, camera 2, camera 3, camera 1,... Here, since all the videos are recorded in the image recording device 87, as the unmonitored video, the video that has not been displayed on the monitor monitor 89 (unmonitored video for non-display of the monitor) and the supervisor 81 in the front direction It is possible to provide both images that are considered not to be monitored because they faced other than the above (unmonitored images that are determined by face authentication and that the supervisor 81 is facing away from the front).

Here, FIG. 8 shows an example of the provided unmonitored video.
Normally, the larger the number of cameras and the smaller the number of monitoring monitors, the smaller the number of images being monitored, and the more unmonitored images for non-monitor display. It is also possible to provide the video frame by thinning it out. In the example shown in FIG. 8, the unmonitored video for non-display of the monitor is thinned out to ½, and the unmonitored video for the monitor 81 facing away from the front direction is provided at the normal frame rate. .

(Summary of this example)
As described above, in the monitoring camera of this example shown in FIG. 2 (such as the monitoring camera 1-1 shown in FIG. 1), the captured image is stored in the primary memory 35 inside the camera for a certain period of time, Processing is performed to extract a face image, and the extracted face image (for example, only the face image may be cut out) is recorded in the secondary memory 37. Further, in the monitoring camera of this example, when a request from the face authentication server 3 is received spontaneously, the face image (for example, only the face image may be recorded) recorded in the secondary memory 37 is sent to the face authentication server 3. To do.
For example, if only the face image (or only the face image and other small amount of data) is distributed and it is not necessary to distribute the entire video frame, this is given to the transmission system (in this example, the IP network 21). The load can be reduced.

The face authentication server 3 of this example shown in FIG. 1 acquires a face image from a surveillance camera, and extracts feature point information of the face image (the balance of eyes, nose and mouth in the face) by a face authentication technique based on image processing. Then, the feature point information is digitized to generate a face ID, and the monitored person information (face image, face ID, acquisition time information, acquisition destination monitoring camera location information) is made into a database and managed.
In this example, a monitoring camera is arranged at an entry / exit location of the monitoring target facility, and the system management server 12 of this example shown in FIG. 1 is monitored based on the monitored person information generated by the face authentication server 3. The location and time of entry / exit of a person is identified and managed in a database.

  In the monitoring system of this example shown in FIG. 1, for example, when an incident occurs, the video at the time of the incident is identified from the recorded images of various monitoring cameras (such as the monitoring camera 1-1) arranged for crime prevention purposes, and the criminal The face image is captured as a still image, and a similar face image is searched from the face images stored in the face authentication server 3 based on the face image of the criminal captured in the still image. Location information can be provided. In the monitoring system of this example, for example, the face image of the criminal captured as a still image is sent to the face authentication server 3, and the face authentication server 3 generates a face ID (face feature information), and the system management server 12 However, it is possible to search the database for a face ID that matches or is similar to the face ID of the criminal, and provide information on the entry / exit time of the criminal.

In the monitoring system of this example shown in FIG. 1, instead of comparing the face image data itself, the system management server 12 compares the criminal's face ID (aggregated face feature information) with the face ID existing in the database. By doing so, matching can be performed in a short time.
In the monitoring system of this example shown in FIG. 1, when there is no exit record and the criminal is still in the facility, a warning is given with an alarm sound or the like when the criminal is caught by a surveillance camera installed at the exit location.

  As shown in FIG. 4, in the monitoring system of this example applied to railway facilities and the like, it is recorded in a center server such as SUICA by cooperating with an IC card such as SUICA and an entrance / exit system using the IC card (SUICA or the like). By comparing the information on the entry station and the entry station (obtained through the use of) with the information on the entry station and the entry station obtained by face authentication in the monitoring system of this example, more accurate entry / exit information Can be provided. Here, the entry / exit information is, for example, information on a face ID, a face image, an entry / exit location, and an entry / exit time (note that either the face ID or the face image may be either), and a center server such as SUICA Check whether there is a match with the entry / exit location and entry / exit time record information obtained from the above, and if there is a match, it will be confirmed as entry / exit information, and if there is no match, it will be uncertain. .

  In this way, in this example, by linking a monitoring system based on face authentication and an IC card reading system such as SUICA used at the time of entry / exit, entry / exit management (for example, identification of entry / exit location and time) User authentication can be performed more accurately and quickly. Specifically, entry / exit management can be performed by matching processing using face IDs, and the entry / exit information obtained when the target person passes through the gate (when entering / leaving) and the entry / exit obtained by face authentication. By comparing the exit information, more accurate entrance / exit information can be provided. Thereby, for example, a criminal's action (entrance / exit information) can be managed together with information by a face image or an IC card, and accurate information on the criminal's entry / exit can be provided.

  In the monitoring system of this example as shown in FIG. 5 (example applied to the identity verification system), a facial image is used in a system for updating a passport or a driver's license that is periodically updated with a facial photograph. Register with personal ID and face ID in the database of management server 67, put IC tag with personal ID recorded on passport, driver's license, etc., and take picture with surveillance camera 65 when posting passport, driver's license, etc. The face ID is generated from the face image thus generated, and the personal ID and the face ID are transmitted to the system management server 67. Further, the system management server 67 uses the face ID registered in the system management server 67 in advance from the personal ID. Locating (or receiving the face ID from the IC tag reader 64) and comparing the face ID (for example, comparing with the face ID from the face authentication server 66) , The captured face image by the monitoring camera 65 can be performed quickly identity verification based.

  In the monitoring system of this example as shown in FIG. 5 (example applied to a personal tracking system), when an electronic money function is added to a passport or a driver's license, the passport or driver's license is used for shopping or other purposes. Each time the IC tag reader (IC tag reader) reads out the personal ID and face ID, the information is used together with information on the place of use and the time of use (use time) to the system management server of the monitoring system of this example (for example, further , Which may include a face image), whereby the system management server 67 can manage the action trajectory of the person (for example, together with the monitoring image).

  In the monitoring system (spoofing prevention system) of this example as shown in FIG. 6, when TASPO or the like is posted, the face ID generated from the face image captured by the monitoring camera 73 installed in the vending machine is used. The system management server 75 compares the face ID with the face ID generated from the current face image registered at the time of updating the passport, driver's license, etc. It is possible to confirm whether or not the user is the person himself / herself.

  Thus, in this example, when a passport, a driver's license, etc. are periodically updated, the face image of the owner is extracted and the current face ID (in this case, the update) is acquired. It is possible to perform identity verification (automatically by the device) when posting a passport, a driver's license, or the like or when posting a TASPO. For example, the person can be identified by comparing the information about the face image captured on the spot with the information about the current face image of the person registered in the database, and is regularly updated as an example. By managing information related to facial images taken when updating passports and driver's licenses as current personal data, the relevant personal data is read out as needed and compared with information related to video captured by the surveillance camera. Thus, it can be determined whether or not the user is the person himself / herself.

  In the monitoring system of this example as shown in FIG. 7, the monitoring camera 82 of the monitoring person 81 is arranged in the monitoring center (monitoring monitoring equipment), and the face angle of the monitoring person 81 is identified by face authentication, It is determined whether or not the video of the monitoring monitor 89 currently displayed by the monitor 81 is being monitored, and the monitor video at the time (time) when the monitor 81 is facing the front is set as the monitored video. The unmonitored video is used as the unmonitored video, and the unmonitored video can be collectively displayed on the monitor later.

  In this way, in this example, the presence / absence of monitoring by the supervisor in the monitoring system is determined based on the supervisor's face direction information (automatically by the apparatus) by face authentication technology, for example, only unmonitored video Can be intensively played back and monitored later. Specifically, whether the currently displayed video is a monitored video or an unmonitored video by identifying a supervisor who performs centralized monitoring with a surveillance camera and identifying the face direction of the supervisor. (Automatically by the apparatus), for example, it is possible to display (reproduce) only the unmonitored video on the monitor monitor later in response to the request of the monitor.

(One structural example of the invention grasped from the present embodiment)
In a monitoring system that monitors the entry and exit of a person (for example, the system shown in FIG. 4 to which the configuration shown in FIG. 1 is applied),
A plurality of imaging means provided at each of a plurality of locations (for example, a station in FIG. 4) that monitor entry / exit, and that captures an image of a person (for example, the monitored person 55 in FIG. 4) passing through each location. (For example, surveillance camera 1-1 in FIG. 1, surveillance cameras 53 and 54 in FIG. 4),
Face feature information generating means (for example, face authentication server 3 in FIG. 1) that generates face feature information that is information related to the characteristics of the face image in the person image captured by the image capturing means;
The face feature information generated by the face feature information generating means, the information corresponding to the place where the imaging means for picking up the image of the person from which the face feature information is generated, and the face feature information Entry / exit information storage means (for example, the system management server 12 in FIG. 1) that stores information including information on the time at which the image of the person from which the image is generated is associated with each other as entry / exit information;
The entry / exit managed device stored by a system (for example, the SUICA center server in the railway system in FIG. 4) that manages the entry / exit managed device carried by a person (for example, the SUICA IC card in FIG. 4). Referring to entry / exit managed device information which is information about entry / exit of the user, the entry / exit information related to the facial feature information stored in the entry / exit information storage means matches the entry / exit managed device information An entry / exit information correct / incorrect determination means (for example, the system management server 12 in FIG. 1) that determines that the entry / exit information is correct when matching is determined;
A monitoring system characterized by comprising:

Here, the configuration of the system and apparatus according to the present invention is not necessarily limited to the configuration described above, and various configurations may be used. The present invention can also be provided as, for example, a method or method for executing the processing according to the present invention, a program for realizing such a method or method, or a recording medium for recording the program. It is also possible to provide various systems and devices.
The application field of the present invention is not necessarily limited to the above-described fields, and the present invention can be applied to various fields.
In addition, as various processes performed in the system and apparatus according to the present invention, for example, the processor executes a control program stored in a ROM (Read Only Memory) in hardware resources including a processor and a memory. A controlled configuration may be used, and for example, each functional unit for executing the processing may be configured as an independent hardware circuit.
The present invention can also be understood as a computer-readable recording medium such as a floppy (registered trademark) disk or a CD (Compact Disc) -ROM storing the control program, and the program (itself). The processing according to the present invention can be performed by inputting the program from the recording medium to the computer and causing the processor to execute the program.

  1-1, 1-2, 65, 73, 86 .. surveillance camera, 2, 11 .. network switch, 3, 66, 74, 83 .. face authentication server, 4 .. network type digital video recorder, 12, 67, 75, 84 ... System management server 13, 62, 76, 90 ... System operation terminal, 21 ... IP network, 31 ... Lens, 32 ... CCD, 33 ... A / D section, 34 ... -DSP signal processing unit, 35 ... Primary memory, 36 ... Image processing unit, 37 ... Secondary memory, 38 ... Video encoding unit, 39 ... I / F unit, 41 ... Monitored person information, 42 ... Face image, 51 ... Ticket gate, 52 ... SUICA reader, 53 ... Suspended surveillance camera, 54 ... Embedded ticket surveillance camera, 55 ... Monitored passengers (passengers) ), 6 1 ... Auditor, 63 ... Travel traveler, 64 ... IC tag reader, 71 ... Purchaser, 72 ... TASPO reader, 77, 81 ... Surveillant, 82 ... Surveillance camera, 85 ... Control unit of the monitoring system, 87 .. Image recording device, 88 .. Video decoder, 89 .. Monitoring monitor,

Claims (1)

In a monitoring system that monitors the entry and exit of people,
A plurality of imaging means provided in each of a plurality of locations for monitoring entry and exit, and for capturing an image of a person passing through each location;
Face feature information generating means for generating face feature information, which is information relating to features of a face image in a person image captured by the image capturing means;
The face feature information generated by the face feature information generating means, the information corresponding to the place where the imaging means for picking up the image of the person from which the face feature information is generated, and the face feature information Entry / exit information storage means for storing information including the information on the time at which the image of the person from which the image is generated is associated with each other as entry / exit information;
Referring to the entry / exit managed device information, which is information related to the entry / exit of the entry / exit managed device stored by the system for managing the entry / exit managed device carried by a person, stores it in the entry / exit information storage means It is determined whether the entry / exit information related to the facial feature information to be applied matches the entry / exit managed device information, and if the entry / exit information matches, the entry / exit information is determined to be correct. Exit information correctness determination means;
A monitoring system characterized by comprising:

Images