JP5955056B2 - Face image authentication device - Google Patents

Description

  The present invention relates to a face image authentication device, and more particularly to a face image authentication device that authenticates a user based on a pre-registered face image.

  An authentication device for determining whether or not a user is a registrant is used to restrict entry to a building or room other than a registrant registered in advance. In general, the authentication device determines whether or not the user is a registrant by collating biometric information such as a password, an ID card, or a user's fingerprint, vein pattern, iris, and face. When it is determined that the user is a registrant, the door provided at the entrance is controlled so that the entrance of the building, room, etc. can be passed. In such an authentication apparatus, when it is determined that the user is a registrant and the entrance is allowed to pass, a person who is not determined as a registrant enters with a user who is determined as a registrant. There was a problem of taking in.

  Therefore, Patent Document 1 proposes a monitoring device that detects a companion act. This monitoring apparatus includes an IC card reader that reads information recorded on an IC card, an operation panel that receives an input of a personal identification number, an authentication apparatus that includes a device that reads biometric information of the person to be authenticated, and a monitoring image that includes the person to be authenticated. And an imaging device for photographing. Then, the monitoring apparatus determines whether or not a plurality of persons are included in the image captured by the imaging apparatus at the timing when the authentication by the authentication apparatus is successful. An abnormal signal is output as there is.

JP 2009-230284 A

  The monitoring apparatus described in Patent Literature 1 can detect a co-action by determining whether or not a person other than the person to be authenticated is shown in the captured monitoring image. However, the monitoring device described in Patent Document 1 does not appear in the monitoring image when the authentication by the authentication device is successful, for example, and detects the accompanying action by a person who tries to enter the room while the door is open. It is not possible.

On the other hand, in order to authenticate a person walking in the monitoring area, the monitoring image is sequentially photographed, the face area image including the face of the person extracted from each of the captured monitoring images is tracked, and the registered face image registered in advance Conventionally, a so-called walk-through type face image authentication apparatus that calculates the similarity to the above and authenticates whether or not the registered person is a registered person is used. When a plurality of persons are imaged, in the conventional walk-through type face image authentication device, the timing at which the authentication result is output differs for each of the plurality of persons depending on the state of the face image to be captured. For example, even if two people are walking in parallel, a person who has acquired a face image facing the front direction has an authentication result earlier than a person who has acquired a face image facing the horizontal direction. The timing of the authentication result varies depending on the face image acquisition environment.
For this reason, in the conventional walk-through type face image authentication apparatus, if there is a person other than the person to be authenticated at the time when the authentication is successful as in Patent Document 1, if it is determined that the person is a companion act, then registration When there is another person determined to be a person, there is a problem that it is erroneously determined to be a companion act.

  An object of the present invention is to provide a walk-through type face image authentication device capable of appropriately determining the presence or absence of accompanying entry.

  The present invention for solving this problem provides a face image authentication device for determining co-entry approach. Such a face image authentication apparatus includes an imaging unit that sequentially acquires a monitoring image obtained by capturing a monitoring region, a storage unit that stores a registered face image of a registrant, and a face that extracts a face region image including a person's face from the monitoring image. A detection means, a face tracking means for tracking and grouping face area images of the same person, and generating series data; for each series data, comparing the face area image included in the series data with a registered face image; Authentication determination means for determining whether the person of the affiliated data is a registrant or an unregistered person, and a predetermined period before and after the authentication time when the person of the particular affiliated data is determined to be a registrant. A companion determination means for determining a companion approach if the determined other series data exists.

  Further, in the face image authenticating device according to the present invention, the authentication determining unit is configured to store the series data that is not determined to be a registrant until the determination suspension period elapses after the tracking of the face area image is started. It is preferable that the person is determined to be an unregistered person, and the accompanying data determination unit determines that the accompanying data is not registered as a registered person after the determination suspension period has elapsed.

  Further, in the face image authentication apparatus according to the present invention, when it is determined that a person of specific series data is a registrant, a door provided at the entrance so as to allow passage from the monitoring area to the restricted area. And a door control means for controlling the door, and the predetermined period is a first period during which the entrance after the authentication time is allowed to pass and a second period shorter than the first period before the authentication time. Preferably there is.

  In the face image authentication device according to the present invention, the door control means acquires a signal indicating the unlocked state and the locked state from the electric lock provided on the door, and the first period is the locked state after the authentication time. It is preferable to set the period to end when the signal indicating is acquired.

  In the face image authentication device according to the present invention, the face detection means extracts a person area representing a person from the monitoring image, further extracts a face area image from the person area, and the face tracking means extracts the face area image. Instead of tracking and categorizing, it is preferable to track and categorize the person area of the same person.

  The walk-through type face image authentication device according to the present invention has an effect that it is possible to appropriately determine the presence or absence of accompanying entry.

It is a schematic diagram showing the example of installation of an imaging part in case the face image authentication apparatus to which this invention is applied is installed in the entrance of a room. It is a schematic block diagram of a face image authentication apparatus. It is a schematic diagram of a history table before being determined to be accompanying entry by the accompanying determination means. It is a schematic diagram of a history table before being determined to be accompanying entry by the accompanying determination means. It is a schematic diagram of a history table before being determined to be accompanying entry by the accompanying determination means. It is a schematic diagram of a history table after it is determined that accompanying entry is performed by the accompanying determination unit. It is a flowchart which shows operation | movement of the authentication process of the face image authentication apparatus by this invention. It is a flowchart which shows the operation | movement of the determination process of the accompanying approach by a accompanying determination means.

Hereinafter, an embodiment in which a face image authentication apparatus to which the present invention is applied is installed at an entrance to a restricted area where entry into a room or the like is restricted will be described with reference to the drawings.
The face image authentication apparatus to which the present invention is applied determines whether a person shown in a monitoring image obtained by photographing a monitoring area is a registered person who is registered in advance or an unregistered person who is not a registered person. In such a case, the entrance to a restricted area where entry into a building, room, or the like is restricted is allowed to pass. In addition, when the face image authentication device determines that another person who has entered the monitoring area in a predetermined period before and after the time when the specific person is determined as a registrant is determined as an unregistered person, the unregistered person follows the registrant. It is determined that there is a so-called accompanying entry that has entered the restricted area. Thereby, the face image authentication apparatus appropriately determines the presence / absence of accompanying entry.

  FIG. 1 schematically shows an installation example of an imaging unit when the face image authentication device is installed at the entrance of a room. In the example shown in FIG. 1, a room is set as a restriction area 256 that restricts entry by persons other than registered persons registered in advance, and a monitoring area 255 is set outside the room entrance 251. As shown in FIG. 1, for example, when the face image authentication device is installed at the entrance 251 of the room, the imaging unit 100 is provided on the wall where the entrance 251 is installed so that the monitoring area 255 includes a passage leading to the entrance 251. The camera is attached to the upper side or the ceiling in a state where the shooting direction is slightly downward and toward the passage. Thereby, the imaging unit 100 can image the persons 260 and 261 heading toward the entrance 251 along the traveling direction 252 at predetermined time intervals. It is assumed that the person 262 exists outside the monitoring area 255 and is not shown in the monitoring image. In FIG. 1, the person 260 is a registered person who is registered as a person who can enter the restricted area 256, and the persons 261 and 262 are unregistered persons who are not registered persons.

  The entrance 251 is provided with a door 253, and the door 253 is provided with an electric lock 254. The electric lock 254 is locked in a normal time. The electric lock 254 is connected to the face image authentication device, and locking and unlocking can be controlled by the face image authentication device. The electric lock 254 is automatically locked when a certain period of time (for example, 0.5 seconds) elapses after the door is closed after being unlocked and the door is opened.

With reference to FIG. 1, a description will be given of the accompanying approach to be determined by the face image authentication apparatus. When it is determined that the person 260 shown in FIG. 1 is a registered person and the electric lock 254 is unlocked, the person 260 can open the door 253 and pass through the entrance 251 for a certain time (for example, 10 seconds). The monitoring area 255 is set to an area that can move from end to end in about a few seconds. Therefore, many unregistered persons who plan to enter together will wait outside the monitoring area 255 until the person 260 is determined to be a registrant, such as the person 262, and the person 260 is determined to be a registrant. After the 254 is unlocked, the person enters the monitoring area 255 and tries to pass through the entrance 251 while the person 260 opens the door 253. This action of the person 262 is hereinafter referred to as “running”.
Alternatively, when many unregistered persons who plan to enter together with a person walk along the immediate vicinity (particularly immediately behind) of the registered person 260 like the person 261, the person 260 opens the door 253. Trying to pass the entrance 251. In the walk-through type face image authentication device, for example, when a person is depressed until moving near the imaging unit 100, or when only a part of the face is reflected in the monitoring image behind another person, the determination as a registrant is made. It may take time to be done. Therefore, the person 260 is not necessarily determined to be a registered person immediately after entering the monitoring area 255, and the person 261 may be determined to be an unregistered person first. In this way, it is possible for a person 261 who has been determined to be an unregistered person before the person 260 is determined to be a registrant to enter together. Hereinafter, this is referred to as “replacement”.

  FIG. 2 is a diagram showing a schematic configuration of the face image authentication apparatus 10 to which the present invention is applied. As illustrated in FIG. 2, the face image authentication device 10 includes an imaging unit 100, an output unit 200, and an image processing unit 300. Hereinafter, each part of the face image authentication device 10 will be described in detail.

The imaging unit 100 is a monitoring camera that captures a predetermined monitoring area as shown in FIG. 1. For example, the imaging unit 100 is a two-dimensional array of photoelectric conversion elements (for example, a CCD) that outputs an electrical signal corresponding to the amount of received light. A sensor, a C-MOS, and the like) and an imaging optical system for forming an image of the monitoring region on the photoelectric conversion element.
The imaging unit 100 is installed substantially in front of the person's passing direction so that the face of the person passing through the monitoring area can be sequentially photographed. Then, the imaging unit 100 acquires a monitoring image obtained by capturing the monitoring area at predetermined time intervals (for example, 200 msec). The imaging unit 100 is connected to the image processing unit 300 and passes the acquired monitoring image to the image processing unit 300.

  The monitoring image can be a grayscale or color multi-tone image. In the present embodiment, the monitoring image is a color image having 1280 pixels by 960 pixels and having 8-bit luminance resolution for each of the RGB colors. However, a monitor image having a resolution and gradation other than this embodiment may be used.

  The output unit 200 includes, for example, a communication interface connected to an electric lock 254 or an external device that controls the electric lock 254 and a control circuit thereof. When the output unit 200 receives a signal indicating successful authentication of a person from the image processing unit 300, the output unit 200 outputs a signal requesting, for example, unlocking of the electric lock 254 to the connected device. In addition, when the output unit 200 receives a signal indicating that there has been a companion approach from the image processing unit 300, the output unit 200 turns on an alarm lamp (not shown) connected to the output unit 200 or sounds a buzzer (not shown). Let Alternatively, a signal indicating that there is a joint entry is output to a monitoring center device (not shown).

  The image processing unit 300 is configured by, for example, a so-called computer. Then, the image processing unit 300 extracts an image of an area including a person's face from the monitoring image received from the imaging unit 100 as a face area image, and performs face matching processing using each face area image. Then, when it is determined that the person shown in the monitoring image is a registered person, the electric lock 254 is unlocked. Further, it is determined whether a person who is not a registrant can enter the room together with a person who is determined to be a registrant. For this purpose, the image processing unit 300 includes a storage unit 310 and a collation unit 320. Furthermore, the collation unit 320 includes a time measuring unit 321, a face detecting unit 322, a face tracking unit 323, an authentication determining unit 324, a door control unit 325, and a accompanying determination unit 326.

  The storage unit 310 includes a semiconductor memory such as a ROM and a RAM, or a magnetic recording medium and its access device or an optical recording medium and its access device. The storage unit 310 stores in advance a computer program and various parameters for controlling the face image authentication apparatus 10. In addition, for each of one or more registrants, the storage unit 310 stores in advance a registered face image showing the registrant's face in association with the registrant ID of the registrant. In addition, the storage unit 310 stores a history table for managing information about a person generated by image processing. Details of the history table will be described later.

  Each unit of the verification unit 320 is a functional module implemented by a microprocessor, a memory, a peripheral circuit thereof, and software operating on the microprocessor. Alternatively, these means may be integrated by firmware. Moreover, you may comprise some or all of these means with an independent electronic circuit, firmware, a microprocessor, etc. Hereinafter, each means of the collation part 320 is demonstrated in detail.

  Every time the monitoring unit 321 acquires a monitoring image from the imaging unit 100, the time measuring unit 321 stores the time when the monitoring image is acquired in the storage unit 310.

Each time the face detection unit 322 acquires a monitoring image from the imaging unit 100, the face detection unit 322 detects a face area that is an area in which a person's face is reflected from the acquired monitoring image, and extracts a face area image.
In order to detect the face area, the face detection unit 322 uses, for example, an inter-frame difference process or a background difference process, and changes with time in luminance values in a plurality of monitoring images generated by the imaging unit 100 are performed. Extract regions. Then, the face detection unit 322 extracts, from the extracted change areas, a change area that seems to be a person from a feature amount such as the size of the change area as a person area representing a person. Then, the face detection unit 322 performs edge pixel extraction on the extracted person region using a Sobel filter or the like so that the inclination direction of the luminance change can be known. Then, the face detection unit 322 detects an elliptical edge distribution having a predetermined size and approximating the contour shape of the head from the extracted edge pixels, and uses the area surrounded by the edge distribution as a face area. Extract. In this case, the face detection unit 322 can detect an elliptical edge distribution using, for example, a generalized Hough transform.

Alternatively, the face detection unit 322 may detect a face area using an Adaboost discriminator. This method is described in a paper by P. Viola and M. Jones “Rapid Object Detection Using a Boosted Cascade of Simple Features” (Proc. The IEEE International Conference on Computer Vision and Pattern Recognition, vol.1, pp.511-518, 2001). ) Can be referred to.
The face detection unit 322 cuts out the detected face region from the monitoring image to create a face region image, and outputs the face region image and the coordinate information of the face region in the monitoring image to the face tracking unit 323.

The face tracking unit 323 performs a tracking process using a known tracking technique on the face area detected by the face detection unit 322 from each of a plurality of monitoring images continuously acquired at a predetermined time interval, Continuously extracted face area images showing the faces of people who are considered to be the same are used as face image data of the same series.
For example, the face tracking unit 323 determines the position of the center of gravity of the face area (hereinafter referred to as the face area of the current frame) detected from the latest monitoring image and the face area (hereinafter referred to as the previous area) detected from the monitoring image one frame before. The distance of the center of gravity position (referred to as a face area of the frame) is obtained, and when the distance is equal to or less than a predetermined threshold, the face area is associated with the same person. It should be noted that when the person moves a certain distance when moving away from the imaging unit 100 and when the person moves the same distance when moving closer to the imaging unit 100, The difference in position is different. Therefore, for example, by setting a predetermined threshold as the size of the face area, it is appropriately evaluated whether the face area of the current frame and the face area of the previous frame are from the same person regardless of the position of the person in the monitoring area. can do. When a plurality of face areas are extracted, it is checked whether or not face areas with the closest distance between the centroid positions correspond to each other.

  Alternatively, the face tracking unit 323 may perform face area tracking processing using a method such as an optical flow or a particle filter.

  The face tracking unit 323 updates the history table stored in the storage unit 310 when the face area tracking process is performed. 3A to 3C show examples of history tables. As shown in FIGS. 3A to 3C, the history table 350 manages the series data for each person. The series data is a set of data represented in each row of the history table 350, and the face tracking unit 323 updates the trial number 351, the tracking flag 352, the face image data 353, and the tracking position information 354 among the series data.

Here, the trial number 351 in the history table 350 is an identification number for discriminating the series data of the person being tracked from the series data of another person, and while the person exists in the monitoring area 255, that is, While being tracked by the face tracking means 323, the same identification number continues to be assigned.
The tracking flag 352 is a flag indicating whether or not the tracking of the person is continued. The tracking flag 352 is turned on when the face tracking unit 323 starts tracking the person, and turned off when the tracking is finished.
The face image data 353 is a face area image extracted by the face detection unit 322, and is stored in series data associated with the same person in the tracking process by the face tracking unit 323.
The tracking position information 354 includes the coordinate information in the monitoring image of the face area associated with the person in the tracking process by the face tracking unit 323 and the time when the monitoring image recorded by the timing unit 321 is acquired. The time information shown is stored in association with the face area image.

  When the face tracking means 323 cannot associate the face area of the current frame of interest with the face area of the previous frame, the face area of the current frame shows a person who has newly entered the monitoring area 255. As a result, series data for the person is newly added to the history table 350. That is, the face tracking unit 323 assigns a new trial number 351 to the series data and sets the tracking flag 352 to ON. Further, the face tracking unit 323 records a face area image created from the face area of the current frame in the face image data 353, and coordinates information of the face area and the monitoring image including the face area in the tracking position information 354. Record time information. Further, the face tracking unit 323 records blanks as initial values for the authentication candidate person 355, the similarity 356, the authentication result 357, and the authentication time 358, which will be described later, and sets the companion flag 359 and the notification flag 360 to OFF as initial values. To do. Then, the face tracking unit 323 outputs the newly added series data to the authentication determination unit 324.

  On the other hand, if the face tracking unit 323 can associate the face area of the current frame of interest with the face area of the previous frame, the series data for that person has already been created in the history table 350. Therefore, the face tracking unit 323 adds the face area image created from the face area of the current frame to the face image data 353 of the corresponding series data in the history table 350, and the face area information of the face area is added to the tracking position information 354. An update process for adding the coordinate information and the time information of the monitoring image including the face area is performed. Then, the face tracking unit 323 outputs the series data to the authentication determination unit 324.

  Further, when there is a face area of the previous frame that is not associated with all face areas of the current frame, the face tracking unit 323 turns off the tracking flag 352 of the corresponding series data in the history table 350. The tracking process for the person is terminated.

The authentication determination unit 324 stores each face area image and each of the face data stored in advance in the storage unit 310 for each series data grouped by the face tracking unit 323, that is, each series data in the history table 350 shown in FIGS. 3A to 3C. The similarity with the registered face image is calculated. The authentication determination unit 324 calculates the similarity using a known face matching technique. For example, the authentication determination unit 324 calculates the sum of squares of the luminance values of the pixels included in the face area image and the corresponding pixels of the registered face image while shifting the positions of the face area image and the registered face image, and calculates the calculated square. The reciprocal of the normalized value obtained by dividing the smallest sum of the sum by the number of pixels included in the face area image is obtained as the similarity.
If the highest similarity among the calculated similarities exceeds the authentication threshold, the person shown in each face area image of the series data is determined as a registrant. This authentication threshold is a reference for authenticating whether or not the face included in the face area image is a registrant's face. It should be noted that, when the degree of similarity of a predetermined number (for example, three) or more among the similarities calculated for each face area image with a specific registered face image exceeds the authentication threshold, the person is determined as a registrant. Also good.

On the other hand, if the face area image of the specific series data is extracted over a predetermined determination suspension period, and the similarity for the extracted face area image is less than or equal to the authentication threshold, the authentication determination unit 324 A person shown in each face area image of the series data is determined as an unregistered person. That is, this unregistered person includes not only visitors who are not registered in advance but also suspicious persons who plan crimes. Note that the determination suspension period is used so that it is not erroneously determined to be accompanying entry when another person who has not been authenticated yet when it is determined that a person is a registrant. It is determined based on an average time from when a person enters the monitoring area 255 and starts to appear in the monitoring image until he or she walks and does not appear in the monitoring image. The determination suspension period can be empirically determined in advance depending on the performance of the imaging unit 100, the installation location, and the like. For example, when the length of the monitoring area 255 is 10 m, it can be set to 5 seconds.
As described above, the authentication determination unit 324 maintains an indeterminate state that is neither a registrant nor an unregistered person for a person who is not determined to be a registrant until the determination hold period elapses, and the determination hold period has elapsed. Sometimes decide to be unregistered.

  Further, the authentication determination unit 324 also determines that a person who has moved out of the monitoring area 255 without being determined as a registered person is an unregistered person even if the determination suspension period has not elapsed.

Returning to FIGS. 3A to 3C, the authentication candidate person 355 is an identification ID of the registrant when the authentication determination unit 324 determines that the person is a registrant.
The similarity 356 is a collation score for specifying the authentication candidate person 355.
The authentication result 357 indicates whether the person is a registered person or an unregistered person. If it is not determined whether the person is a registered person or an unregistered person by the authentication determining means 324 as in the state where the person has just started tracking, a blank is set, and the person is determined to be a registered person. If registered, “Registered” is recorded, and if determined to be an unregistered, “Unregistered” is recorded.
The authentication time 358 represents the time when the authentication determination unit 324 determines that the person is a registered person or an unregistered person.

  The authentication determination unit 324 includes the latest face area image and the storage unit of the series data output from the face tracking unit 323, and the tracking flag 352 of the history table 350 is ON and the authentication result 357 is not a registrant. The similarity with each registered face image stored in advance in 310 is calculated. If the highest similarity among the calculated similarities exceeds the authentication threshold, the person of the series data is determined as a registrant, the tracking flag 352 is set to OFF, and the similarity 356 is the highest. The similarity is recorded, the registrant is recorded in the authentication result 357, and the current time is recorded in the authentication time 358. Furthermore, the registrant of the registered face image corresponding to the highest similarity is specified, and the information (registrant identification ID) is stored in the authentication candidate person 355.

  On the other hand, the authentication determination unit 324 determines whether the authentication result 357 remains unconfirmed until the determination suspension period elapses, or the person in the series data whose authentication flag 352 is unconfirmed and the tracking flag 352 is OFF. Is determined to be an unregistered person, the unregistered person is recorded in the authentication result 357, and the current time is recorded in the authentication time 358. In addition, in order to improve the detection accuracy of the registrant, the authentication determination unit 324 keeps the tracking flag 352 ON for the series data determined to be an unregistered person, and whether or not it is a registrant in subsequent monitoring images. Continue judgment. Alternatively, the tracking flag 352 may be set to OFF for the series data determined to be an unregistered person, and it may not be determined whether or not the user is a registered person in subsequent monitoring images.

  When the authentication determining unit 324 determines that the person who passes the monitoring area 255 is a registrant, the door control unit 325 determines that the authentication is successful in order to control the door 253 so that the entrance 251 can pass. The output signal is output to the output unit 200.

  Based on the history table 350, the accompanying determination unit 326 determines whether or not an unregistered person has entered the restricted area 256 together. The companion determination unit 326 determines that another series data exists in a predetermined period before and after the authentication time when the person of the specific series data is determined to be a registrant, and the person of the other series data is determined to be an unregistered person. In the case, it is determined that there is a joint entry.

  The accompanying determination unit 326 is an unregistered one in which tracking of the face area image is started in a predetermined period after the authentication time when the person of the specific series data is determined to be a registrant (hereinafter referred to as a period after successful authentication). If there is other series data determined to be a person, it is determined that there was a joint entry due to rushing. The period after successful authentication is a period in which the entrance 251 after the person 260 shown in FIG. 1 is determined to be a registrant is allowed to pass, that is, the person 260 opens the door 253 after the electric lock 254 is unlocked. It can be determined in consideration of the time required for opening and entering the room and closing and locking the door 253 (for example, 12 seconds).

  On the other hand, the accompanying determination unit 326 starts tracking the face area image during a predetermined period before the authentication time when the person of the specific series data is determined to be a registrant (hereinafter referred to as a pre-authentication period). When there is other series data determined to be an unregistered person, it is determined that there is a companion approach due to replacement. This pre-authentication period is shorter than the post-authentication period. The reason for setting the pre-authentication period shorter than the post-authentication period is to enter the monitoring area before the person who is determined to be a registrant. This is because it should not be determined. In addition, when a change occurs, the person who is determined to be a registrant and the person who is determined to be an unregistered person are walking in close proximity, and the determination of the registrant and the unregistered person is considered to be made continuously. Because. Therefore, the period before the authentication success is a time during which a registered person and an unregistered person can be detected continuously without misjudging that the person who is trying to return without succeeding in authentication is determined to be accompanied by entry. Seconds.

  As described above, in both cases of rushing and replacement, the accompanying determination means 326 determines that the tracking start time of the face area image of the person determined to be an unregistered person is a person of other series data as a registered person. If it is within a predetermined period before and after the authentication time 358, it is determined that the accompanying entry has occurred. As a result, the fact that there has been a joint entry can be output to the outside and an alarm can be notified, and the administrator or the like can take an appropriate response.

Returning to FIG. 3A to FIG. 3C, the accompanying flag 359 is a flag indicating whether or not it is determined that the accompanying person has entered the affiliated data. If it is not determined that there has been a companion approach, it is set to OFF. If it is determined that there has been a companion approach, it is set to ON.
The notification flag 360 is a flag that indicates whether or not the fact has been reported to the outside regarding the series data that has been determined to have been accompanied by a companion. The initial value of the notification flag 360 is OFF, and the accompanying flag 359 is set to ON by the accompanying determination unit 326, and is set to ON when notification processing is performed. That is, that the notification flag 360 is ON indicates that the notification processing has already been performed.

An example of the accompanying approach determined by the accompanying determination unit 326 will be described with reference to FIGS. 3A to 3C and FIG. FIGS. 3A to 3C show the history table 350 before the accompanying determination unit 326 determines that the accompanying entry is performed, and FIG. 4 shows the history table 350 after the accompanying determination unit 326 determines that the accompanying approach is performed. Indicates.
In the series data with trial numbers 351 “12” and “13”, the person of the series data with trial number 351 “12” is determined as a registrant in FIG. 3A, and the series with trial number 351 “13” in FIG. An example is shown in which it is determined that the person of the data is an unregistered person and that there is a combined entry by rushing. In the series data with the trial numbers 351 “24” and “25”, the person of the series data with the trial number 351 “24” is determined as the registrant in FIG. 3B, and the series with the trial number 351 “25” in FIG. An example is shown in which it is determined that the person of the data is an unregistered person and that there is a combined entry by rushing. In the series data of trial numbers 351 “36” and “37”, the person of the series data of trial number 351 “36” is determined as a registrant in FIG. 3C, and the series of trial numbers 351 “37” in FIG. An example is shown in which it is determined that the person of the data is an unregistered person, and that there is a companion approach due to replacement.

  The operation of the authentication process by the face image authentication apparatus 10 to which the present invention is applied will be described below with reference to the flowchart shown in FIG. The flow of operations described below is controlled by a control unit (not shown) that operates on the microprocessor constituting the image processing unit 300 and controls the face image authentication apparatus 10 as a whole. Note that the history table 350 is initialized to be blank before the processing described below is performed. The operation described below is performed every time one monitoring image is acquired.

  First, the imaging unit 100 of the face image authentication apparatus 10 acquires a monitoring image obtained by capturing a monitoring area, and sends it to the verification unit 320 of the image processing unit 300 (step S501). When the collation unit 320 acquires the monitoring image, the time measuring unit 321 stores the time when the monitoring image is acquired in the storage unit 310 in association with the monitoring image. Next, the face detection unit 322 detects a face area in which a person's face is shown from the acquired monitoring image, extracts the face area from the monitoring image, and extracts a face area image (step S502). Next, the face detection unit 322 determines whether one or more face regions have been detected (step S503). If no face area is detected, the process proceeds to step S501, and the processes in steps S501 to S503 are repeated until the face area is detected.

  On the other hand, when one or more face areas are detected and a face area image is extracted in step S503, the face tracking unit 323 sets the tracking flag 352 in the history table 350 to ON for all detected face areas. Correspondence with the current series data is performed (step S504). Then, the face tracking unit 323 determines whether there is any data that is not associated with any of the face areas of the current frame in the series data for which the tracking flag 352 is ON (step S505). If there is no face area associated with any face area of the current frame, the process proceeds to step S507. On the other hand, when there is series data that is not associated with any of the face areas of the current frame and the tracking flag 352 is ON, the person of the series data has entered the room with authentication success or is authenticated. It is thought that it was away from the monitoring area. Therefore, the face tracking unit 323 sets the tracking flag 352 of the series data that is not associated with any of the face areas of the current frame to OFF so that the tracking process for the series data is not performed thereafter (step S506). ).

  Then, the face tracking unit 323 determines whether there is any face area in the current frame that could not be associated with the series data for which the tracking flag 352 is ON (step S507). If there is any data that could not be associated with the series data for which the tracking flag 352 is ON, the face tracking unit 323 newly adds the series data for the person in the face area that could not be associated to the history table 350. to add. In other words, the face tracking unit 323 assigns a new trial number 351 to the series data for the person in the face area that could not be associated. Then, the tracking flag 352 is set to ON, the face area image created from the face area of the current frame is recorded in the face image data 353, and the coordinate information of the face area and the time information of the monitoring image are recorded in the tracking position information 354. Record a blank in the authentication result 357. Further, the accompanying flag 359 is set to OFF, and the notification flag 360 is set to OFF (step S508).

  When the face area of the current frame can be associated with the series data for which the tracking flag 352 is ON, the face tracking unit 323 corresponds the coordinate information of the face area and the time information of the monitoring image. Update processing to be added to the tracking position information 354 of the series data is performed. Then, the face tracking unit 323 outputs the updated series data to the authentication determination unit 324.

  The following processing in steps S509 to S514 is performed for each series data in which the tracking flag 352 is ON and the authentication result 357 is not a registrant. The reason why the series data whose authentication result 357 is the registrant among the series data whose tracking flag 352 is ON is not to be processed is that authentication has already been performed.

  The authentication determination unit 324 calculates the similarity between all the registered face images stored in the storage unit 310 with respect to the latest face area image of the processing target series data (step S509).

  Next, the authentication determination unit 324 determines whether the person is a registrant based on whether the highest similarity among the calculated similarities exceeds the authentication threshold (step S510). If it is determined that the person is not a registered person, the process proceeds to step S513. On the other hand, if it is determined that the person is a registrant, the authentication determination unit 324 sets the tracking flag 352 of the series data to OFF, records the highest similarity in the similarity 356, and the authentication result 357. And the current time is recorded at the authentication time 358. Further, the information of the registered person of the registered face image corresponding to the highest similarity is stored in the authentication candidate person 355 (step S511).

  Then, the door control unit 325 outputs a signal indicating a successful authentication to the output unit 200 (step S512). When a signal indicating successful authentication is output to the output unit 200, the electric lock is unlocked via the output unit 200. Thus, a person who has succeeded in authentication can enter the room. The electric lock is automatically locked as soon as the person's entry is confirmed or after a predetermined time has elapsed.

Next, a case where it is determined in step S510 that the similarity does not exceed the authentication threshold will be described.
The authentication determination unit 324 determines whether or not the determination suspension period has elapsed since the tracking of the person was started (step S513). If the determination suspension period has elapsed, the authentication determination unit 324 determines that the person is an unregistered person, records the person as an unregistered person in the authentication result 357 of the series data, and sets the current time at the authentication time 358. Is recorded (step S514). On the other hand, if the determination suspension period has not yet elapsed, the authentication result 357 for the person remains unconfirmed and no particular processing is performed.
Then, the authentication determination unit 324 shifts the process to step S509 if there is still series data in which the tracking flag 352 is ON and the authentication result 357 is not a registrant, and shifts the process to step S515 if not. .

  The authentication determination unit 324 refers to the history table 350 and performs the process of step S515 on the series data in which the tracking flag 352 is OFF and the authentication result 357 is unconfirmed. In step S515, the authentication determination unit 324 determines that the person is an unregistered person, records the unregistered person in the authentication result 357, and records the current time in the authentication time 358 in the same manner as in step S514.

  Next, the accompanying determination unit 326 refers to the history table 350 and determines whether there is an accompanying entry (step S516). Details of this processing will be described later using the flowchart shown in FIG.

The accompanying determination unit 326 determines that there is an accompanying entry in the determining process of accompanying entry depending on whether there is series data in which the accompanying flag 359 is ON and the notification flag 360 is OFF, and the outside It is determined whether or not there is series data that has not been reported yet (step S517). If there is such series data, the accompanying determination unit 326 outputs a signal indicating that the accompanying entry has occurred to the output unit 200 and sets the notification flag 360 of the series data to ON (step S518). ), The process returns to step S501. On the other hand, if there is no such series data, no process is performed and the process returns to step S501.
When a signal indicating that the accompanying entry has occurred is output to the output unit 200, a warning lamp (not shown) is turned on, a buzzer (not shown) is sounded, and the like via the output unit 200. Alternatively, a signal indicating that the accompanying entry has occurred is output to the monitoring center device (not shown) via the output unit 200. As a result, the monitoring center device can notify the administrator that there has been a shared entry and record the history.

Next, the accompanying entry determination process performed by the accompanying determination unit 326 in step S516 of the flowchart in FIG. 5 will be described with reference to the flowchart in FIG.
First, the companion determination unit 326 determines from the history table 350 the series data that has been tracked within the latest predetermined period (for example, 10 minutes), that is, the series in which the latest time of the tracking position information 354 is within the latest predetermined period. Of the data, series data in which the sharing flag 359 is OFF and the notification flag 360 is OFF is acquired (step S601).

  The accompanying determination unit 326 performs the processing of steps S602 to S605 for all acquired series data. The accompanying determination unit 326 extracts series data in which the authentication result 357 is a registrant from the acquired series data (step S602).

The accompanying determination unit 326 performs the processing of steps S603 to S605 for all other series data other than the series data extracted in step S602 among the acquired series data. The accompanying determination unit 326 extracts series data whose authentication result 357 is an unregistered person from other series data (step S603). Next, the companion determination unit 326 extracts whether the tracking start time of the tracking position information 354 of the extracted sequence data is within a predetermined period before and after the authentication time 358 of the sequence data extracted in step S602. It is determined whether or not the series data is series data within a predetermined period before and after successful authentication of the registrant (step S604). Note that the predetermined periods before and after the successful authentication are the above-described pre-authentication period and post-authentication period. If the extracted series data is series data within a predetermined period before and after successful authentication of the registrant, the accompanying determination unit 326 determines that the accompanying entry has occurred, and the registrant extracted in step S602 is determined. The sequence data accompanying flag 359 of the sequence data and the unregistered person extracted in step S603 is set to ON (step S605). On the other hand, when the extracted series data is not series data within a predetermined period before and after successful authentication of the registrant, no particular processing is performed.
Then, the companion determination means 326 still has series data other than the series data extracted in step S602, which is not extracted in step S603 and whose authentication result 357 is an unregistered person, not extracted in step S603. If so, the process proceeds to step S603. On the other hand, if there is still series data that is not extracted in step S602 and the authentication result 357 is a registrant among all series data acquired in step S601, the process proceeds to step S602, and there is no If so, the process of the flowchart of FIG. 6 is terminated, and the process proceeds to step S517.

  As described above, the accompanying determination unit 326 determines the accompanying entry by comparing the series data in which the authentication result 357 is a registered person and the series data in which the unregistered person is an unregistered person. In other words, for series data that is not determined to be a registrant, the determination pending period elapses as long as the person of the series data is tracked without being subject to determination of accompanying entry until it is determined to be an unregistered person. After that, determine whether you will be accompanied by a companion. As a result, a certain period of time can be secured until it is determined that the person who is not determined as a registered person or an unregistered person in the monitoring area is accompanied by entry, and erroneous determination of accompanying entry can be suppressed. .

  Hereinafter, an example of how the history table 350 illustrated in FIGS. 3A to 3C and 4 is processed in the flowcharts of FIGS. 5 and 6 will be described. In the following description, it is assumed that the period after successful authentication is 12 seconds and the period before successful authentication is 6 seconds. For ease of explanation, it is assumed that monitoring images are acquired at 1-second intervals, and the history table 350 is updated at 1-second intervals.

First, series data with trial numbers 351 of “12” and “13” in FIGS. 3A and 4 will be described. The person of the series data whose trial number 351 is “12” is a registrant and corresponds to the person 260 of FIG. On the other hand, the person of the series data whose trial number 351 is “13” is an unregistered person who intends to make a joint entry by rushing and corresponds to the person 262 of FIG. This person enters the monitoring area 255 after the person of the series data whose trial number 351 is “12” is determined to be a registrant, and the door 253 is closed after the person determined to be the registrant opens the door 253. By the time of entering the restricted area 256.
In step S601 of the flowchart of FIG. 6, the accompanying determination unit 326 uses two series data with trial numbers 351 of “12” and “13” in FIG. 3A as series data that has been tracked within a recent predetermined period. Is obtained. In this case, the sharing determination unit 326 extracts the series data with the trial number 351 of “12” as the series data in which the authentication result 357 is the registrant in step S602, but the authentication result 357 is the unregistered person in step S603. Certain series data cannot be extracted, and the process ends.
On the other hand, as shown in FIG. 4, when the person of the series data whose trial number 351 is “13” goes out of the monitoring area 255, the tracking flag 352 for the series data is set to OFF in step S506 of FIG. In step S515, the unregistered person is recorded in the authentication result 357. Thereafter, when the processing of the flowchart of FIG. 6 is performed, the accompanying determination unit 326 extracts the sequence data with the trial number 351 being “12” as the sequence data in which the authentication result 357 is the registrant in step S602, In S603, the sequence data with the trial number 351 “13” is extracted as the sequence data in which the authentication result 357 is an unregistered person. The first time (hereinafter referred to as tracking start time) “08:50:15” recorded in the tracking position information 354 of the series data whose trial number 351 is “13” is the series whose trial number 351 is “12”. Since the data authentication time 358 is later than “08:50:12” and is included in the period after the authentication success, in step S605, the shared data 359 of the series data with the trial numbers 351 of “12” and “13” is turned ON. Is set. Then, in step S518, a signal indicating that there is a joint entry is output, and the notification flag 360 is set to ON.

Next, the series data in which the trial numbers 351 in FIGS. 3B and 4 are “24” and “25” will be described. The person of the series data whose trial number 351 is “24” is the registrant, and corresponds to the person 260 of FIG. 1, like the person of the series data whose trial number 351 is “12”. On the other hand, the person of the series data whose trial number 351 is “25” is an unregistered person who intends to carry out the joint entry by rushing in the same manner as the person of the series data whose trial number 351 is “13”. It corresponds to. However, unlike the person of the series data whose trial number 351 is “13”, this person has not been determined as a registrant but has been determined to be an unregistered person after the determination suspension period has elapsed.
In step S601 of the flowchart of FIG. 6, the companion determination unit 326 uses two series data with the trial numbers 351 of “24” and “25” in FIG. 3B as series data that has been tracked within the recent predetermined period. Is obtained. As in the case of FIG. 3A, in this case, the accompanying determination unit 326 cannot extract the series data in which the authentication result 357 is an unregistered person in step S603, and ends the processing.
On the other hand, as shown in FIG. 4, when the determination suspension period has elapsed for the series data whose trial number 351 is “25” thereafter, an unregistered person is recorded in the authentication result 357 in step S <b> 514 of FIG. 5. The accompanying determination unit 326 subsequently updates the series data with the trial numbers 351 of “24” and “25” in the same manner as the series data of the trial numbers 351 of “12” and “13”.

Next, the series data in which the trial numbers 351 in FIGS. 3C and 4 are “36” and “37” will be described. The person of the series data whose trial number 351 is “36” is an unregistered person who makes a joint entry by replacement, and corresponds to the person 261 in FIG. On the other hand, the person of the series data whose trial number 351 is “37” is a registrant and corresponds to the person 260 in FIG. That is, the person of the series data whose trial number 351 is “36” is walking immediately behind the person of the series data whose trial number 351 is “37”, but the person of the series data whose trial number 351 is “37” is registered. Before entering the monitoring area 255.
In step S601 in the flowchart of FIG. 6, the accompanying determination unit 326 uses two series data with trial numbers 351 of “36” and “37” in FIG. 3C as series data that has been tracked within a recent predetermined period. Is obtained. In this case, the accompanying determination unit 326 cannot extract the series data in which the authentication result 357 is the registrant in step S602, and ends the process.
On the other hand, as shown in FIG. 4, when the person of the series data whose trial number 351 is “37” is subsequently determined as a registrant, the registered person is recorded in the authentication result 357 of the series data in step S511 of FIG. Is done. Thereafter, when the processing of the flowchart of FIG. 6 is performed, the accompanying determination unit 326 extracts the sequence data with the trial number 351 of “37” as the sequence data in which the authentication result 357 is the registrant in step S602, In S603, the sequence data with the trial number 351 “36” is extracted as the sequence data in which the authentication result 357 is an unregistered person. The tracking start time “10:24:19” recorded in the tracking position information 354 of the series data whose trial number 351 is “36” is the authentication time 358 “10:24” of the series data whose trial number 351 is “37”. : 24 ”and included in the first half of the authentication success, the trial numbers 351 are“ 36 ”and“ 37 ”in the same manner as in the case of the series data with the trial numbers 351“ 12 ”and“ 13 ”. The series data is updated.

  As described above, the face image authentication apparatus to which the present invention is applied determines whether the person shown in the monitoring image obtained by photographing the monitoring area is a registrant or an unregistered person, and is a registrant. In some cases, the entrance to the restricted area is allowed. On the other hand, if it is determined that another person who has entered the monitoring area in a predetermined period before and after the time when the specific person is determined to be a registrant is determined to be an unregistered person, it is determined that there has been a joint entry. Thereby, the face image authentication device can appropriately determine the presence or absence of the accompanying entry.

The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments. For example, in this embodiment, the period after successful authentication is set to a certain period during which the entrance is allowed to pass. However, if the signal indicating that the face image authentication device is locked by the electric lock can be acquired, the electric lock is released. It is good also as a period from locking to locking. In that case, the door control means obtains a signal indicating the unlocked state and the locked state from the electric lock, and the accompanying determination means obtains a signal indicating the locked state from the electric lock in the period after the successful authentication. The period after successful authentication is terminated. Thereby, based on an actual locking / unlocking state, the presence / absence of accompanying entry can be determined more appropriately.
Further, in consideration of the time required for the person to move from end to end of the monitoring area, the period after successful authentication may be shortened according to the size of the monitoring area. The accompanying determination means determines whether or not there is a possibility of accompanying entry by determining whether or not the person can enter the room while the electric lock is unlocked based on the movement speed of the person. May be. In that case, for example, a list representing a correspondence relationship between the position of the face area in the monitoring image and the position of the person in the face area in the monitoring area is stored in advance in the storage unit. Then, the accompanying determination unit obtains a change in the position of the person in the monitoring area from the change in the position of the face area in each series data, and calculates the movement speed.

In the present embodiment, the face area extracted from the monitoring image is tracked to generate series data, and the accompanying approach is determined. However, the face area cannot necessarily be extracted for all persons.
Therefore, instead of tracking the face area, the face tracking means may track and group the person areas extracted by the face detection means. Since it is highly likely that a person whose face area cannot be extracted is a suspicious person, the determination suspension period for the series data from which the face area cannot be extracted may be set shorter (eg, 3 seconds) than the normal determination suspension period. Good.
Thereby, it is possible to appropriately detect a suspicious person who tries to enter the restricted area while hiding his face.

Further, in the present embodiment, an example is shown in which the tracking start time is within the time after successful authentication, and it is determined that there is a shared entry when it is determined as an unregistered person after the determination suspension period has elapsed. You may determine with the accompanying approach without waiting for progress of a holding | maintenance period. In other words, the accompanying determination unit determines whether or not there is an accompanying entry by comparing the time when each person is determined to be a registrant and the time when each person is determined as an unregistered person. However, in this case, there is a high possibility that a person who has not been determined despite being a registrant will be erroneously determined to enter together.
Also, instead of comparing the time, for example, a frame number is assigned to each monitoring image, and the frame number of the monitoring image that starts tracking the person determined to be an unregistered person and the person determined to be a registrant are The presence or absence of accompanying entry may be determined by comparing the frame number of the determined monitoring image. Alternatively, using a counter that counts up at a predetermined time interval, a counter value when tracking of a person determined to be a non-registered person and a counter value of a person determined to be a registrant are determined. The presence or absence of accompanying entry may be determined by comparison.

  In the present embodiment, the face image authentication device controls whether or not the entrance of a restricted area such as a room is permitted depending on whether or not the person who passes the monitoring area is a registrant. It is not limited to. For example, the face image authentication device may determine whether a person who passes through the monitoring area is a registered person or an unregistered person and simply notifies the outside. Even in such a case, the face image authentication device passes only the registrant, only the unregistered person, or the unregistered person follows the registrant. Therefore, it is possible to appropriately detect a suspicious person who attempts to pass through the monitoring area illegally.

  In addition, the face image authentication device not only issues an alarm when detecting the accompanying entry, but also locks the electric lock or maintains the lock of the electric lock so that the person in the monitoring area You may make it impossible to enter.

  In the present embodiment, an example of a face image authentication device has been described. However, the present invention can be applied to other walk-through authentication devices. For example, an authentication device that performs authentication using an RFID tag, an iris device, and the like. The present invention can also be applied to an authentication device that performs authentication.

  As described above, those skilled in the art can make various modifications in accordance with the embodiment to be implemented within the scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Face image authentication apparatus 100 Image pick-up part 200 Output part 300 Image processing part 310 Storage part 320 Collation part 321 Time measuring means 322 Face detection means 323 Face tracking means 324 Authentication determination means 325 Door control means 326 Joint determination means

Claims (4)

A face image authentication device for determining joint entry,
An imaging unit that sequentially acquires monitoring images obtained by imaging the monitoring area;
A storage unit for storing a registered face image of the registrant;
Face detection means for extracting a face area image including a human face from the monitoring image;
Face tracking means for tracking and grouping the face area images of the same person and generating series data;
Authentication determination means for comparing, for each series data, the face area image included in the series data and the registered face image to determine whether the person of the series data is a registered person or an unregistered person;
A predetermined time period before and after the authentication time of the person is determined to registered users of a particular series data, and judges tailgating determining means and unregistered person and if there is determined to be other series data tailgating entry, the Have
The authentication determination means determines that the person of the series data that has not been determined as a registrant from the start of tracking of the face area image until the determination suspension period has passed is an unregistered person,
For the series data that is not determined to be a registrant, the accompanying companion determination means determines the accompanying entry after the determination suspension period has elapsed.
A face image authentication apparatus characterized by the above. When it is determined that the person of the specific series data is a registrant, it further includes door control means for controlling a door provided at the entrance so as to allow passage from the monitoring area to the restricted area. ,
The predetermined period is the the first period inlet are passable, the first second period shorter than the period before the authentication time after the authentication time, to claim 1 The face image authentication device described. The door control means obtains a signal indicating an unlocked state and a locked state from an electric lock provided on the door,
The face image authentication device according to claim 2 , wherein the first period is a period that ends when a signal indicating the locked state is acquired after the authentication time. The face detection means extracts a person area representing a person from the monitoring image, further extracts a face area image from the person area,
The face image according to any one of claims 1 to 3 , wherein the face tracking means tracks and groups the person areas of the same person instead of tracking and grouping the face area images. Authentication device.

Images