JP2010033151A - Monitoring device and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce a processing load in the collation processing of a face image. <P>SOLUTION: An image acquisition part 161 continuously acquires the face images of collation candidates. A face image detection processing part 162 extracts the acquired face images. A simple featured value extraction part 152 extracts featured values from the extracted face images. A dictionary featured value table 156 stores only the prescribed number of featured values of the continuously acquired face images as dictionary data. After the prescribed number of featured values of the face images are stored as the dictionary data by a dictionary featured value table 156, a collation featured value table 157 stores only the prescribed number of feature values of the continuously acquired face images as the collation data. When the prescribed number of featured values of the face images in the dictionary data are not similar with the prescribed number of featured values in the collation data, the face images acquired in the timing are supplied to the face image recognition device. This invention may be applied to a monitoring system. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a monitoring apparatus, method, and program, and more particularly, to a monitoring apparatus, method, and program that can easily collect customer visit information.

  Amusement stores are intensifying competition to acquire players as competition with competing stores intensifies due to the decrease in gaming population and equipment replacement. In order to attract fixed players, the amusement arcade has developed a unique management strategy that differentiates it from neighboring amusement stores.

  Specifically, by grasping the trend of the players who visit the store, issuing a membership card and reading the membership card inserted in a dedicated device when visiting the store, in order to provide services that meet the needs of the player The game model, game time, and the like are accumulated as a database, and the trend of players visiting the store is understood by analyzing the database, and the needs of the players are investigated.

  However, in reality, it is not possible to sufficiently grasp the trend of the player who can be originally investigated due to the fact that the membership card is not brought in or is not inserted into the dedicated device even if it is brought. In order to solve this problem, a face authentication device has been proposed.

  In recent years, performance of this face authentication device has improved, and each player can be identified. In order to identify a player by face authentication, the camera installation position for detecting the player's face is important.

  In addition, in this face authentication device, it is required to quantify information about what kind of person is playing what kind of game by managing each player's face image and game information together. Yes.

  In order to know the trend of the player in the face authentication device, it is necessary to grasp the staying time of the player on the game machine.

  In view of this, a technique has been proposed in which customer face images during a game are acquired for each gaming machine and customer management is performed by collating face images (see Patent Document 1).

JP 2007-159883 A

  However, in face authentication processing, face images with different face orientations of the acquired face images are collated due to changes in the player's face movement (changes in face orientation), etc., and the person is authenticated as another person. As a result, the identity recognition rate may be reduced.

  In addition, a face image with a different face orientation may be determined to be replaced by another person, and the registered face image may be updated. In such a case, a face image that is not suitable for collation is registered as a collation target. Therefore, even if a face image suitable for collation is acquired, it may not be recognized as the user.

  The present invention has been made in view of such a situation, and in an authentication process using a face image, it is possible to accurately recognize occurrence of a person replacement by comparing face images supplied continuously. Is.

  The monitoring device according to one aspect of the present invention includes an accumulating unit that accumulates a face image as an accumulator's face image in an accumulator database, an acquisition unit that continuously acquires a face image of a person to be collated, and an acquisition unit that acquires the face image. A face image extracting unit that extracts the face image, a feature amount extracting unit that extracts a feature amount from the face image extracted by the face image extracting unit, and a feature amount of the face image acquired continuously, A dictionary data storage unit that stores a predetermined number of dictionary data as dictionary data, and a feature amount of the predetermined number of face images stored as the dictionary data by the dictionary data storage unit, and then the facial image acquired continuously A collation data storage unit that stores a predetermined number of feature quantities as collation data, a feature quantity of a predetermined number of face images in the dictionary data, and a feature quantity of a predetermined number of face images in the collation data If not, the similarity between the face image of the person to be collated acquired at that timing and the face image of the accumulator accumulated in the accumulating means is calculated and collated, and the collation result of the collating means Similarity degree determination means for determining whether or not the face image of the person to be collated is the face image of the accumulator person by comparing a certain degree of similarity with a predetermined threshold value.

  A feature amount comparison / determination unit that determines whether or not the feature amount of the predetermined number of face images in the dictionary data is similar to the feature amount of the predetermined number of face images in the matching data may be further included. If it is determined by the feature amount comparison and determination unit that the similarity is not similar, the collation target person's face image acquired at that timing, and the accumulator's face image accumulated in the accumulation unit The similarity can be calculated and collated.

  An average feature amount calculation unit that calculates an average feature amount of a predetermined number of face images in the dictionary data as an average feature amount can be further included, and the feature amount comparison and determination unit includes: Each of the feature quantities of the predetermined number of face images is compared with the average feature quantity, and if both are not similar, the feature quantity of the predetermined number of face images in the dictionary data and the predetermined number of features in the matching data It can be determined that the feature amount of the face image is not similar.

  When the feature quantity of the predetermined number of face images in the dictionary data and the feature quantity of the predetermined number of face images in the collation data are not similar, at that timing, the dictionary data storage means stores the feature data as the dictionary data. The feature quantity of the predetermined number of face images is deleted, and the feature quantity of the predetermined number of face images stored as the collation data is stored as new dictionary data by the collation data storage means. can do.

  When the feature value of the facial image acquired continuously and the feature value stored in the dictionary data and having the lowest priority are not similar to the dictionary data storage means, the dictionary data The feature quantity with the lowest priority stored in the table can be updated with the feature quantity of the acquired face image.

  In the collation data storage means, when the feature quantity of the face image acquired continuously and the feature quantity stored as the collation data and having the lowest priority are not similar, the collation data The feature quantity with the lowest priority stored in the table can be updated with the feature quantity of the acquired face image.

  The monitoring method according to one aspect of the present invention includes an accumulating unit that accumulates a face image as an accumulator's face image in an accumulator database, an acquisition unit that continuously acquires a face image of a person to be collated, and an acquisition unit that acquires the face image. A face image extracting unit that extracts the face image, a feature amount extracting unit that extracts a feature amount from the face image extracted by the face image extracting unit, and a feature amount of the face image acquired continuously, A dictionary data storage unit that stores a predetermined number of dictionary data as dictionary data, and a feature amount of the predetermined number of face images stored as the dictionary data by the dictionary data storage unit, and then the facial image acquired continuously A collation data storage unit that stores a predetermined number of feature quantities as collation data, a feature quantity of a predetermined number of face images in the dictionary data, and a feature quantity of a predetermined number of face images in the collation data If not, the similarity between the face image of the person to be collated acquired at that timing and the face image of the accumulator accumulated in the accumulating means is calculated and collated, and the collation result of the collating means A monitoring method of a monitoring device including similarity determination means for determining whether or not the face image of the person to be collated is the face image of the accumulator by comparing a certain similarity with a predetermined threshold, An acquisition step of continuously acquiring the face image of the person to be collated by the acquisition means, a face image extraction step of extracting the face image acquired by the processing of the acquisition step by the previous face image extraction step, and the characteristics A feature amount extracting step for extracting a feature amount from the face image extracted by the processing of the face image extracting step by the amount extracting unit; and the face image continuously acquired by the dictionary data storing unit. The dictionary data storage step of storing a predetermined number of feature quantities as dictionary data and the dictionary data storage step by the collation data storage means store the feature quantities of the predetermined number of face images as the dictionary data. After that, a collation data storage step of storing a predetermined number of feature values of the face image continuously obtained as collation data, and a feature number of a predetermined number of face images in the dictionary data by the collation means, When the feature quantity of the predetermined number of face images in the matching data is not similar, the similarity between the face image of the person to be collated acquired at that timing and the face image of the person accumulated in the accumulation unit is calculated. Then, by comparing the collation step to be collated with the similarity that is the collation result of the collation step by the similarity determination unit and a predetermined threshold, A similarity determination step of determining whether or not the face image of the target person is the face image of the accumulator.

  A program according to an aspect of the present invention is acquired by an accumulation unit that accumulates a face image as an accumulator's face image in an accumulator database, an acquisition unit that continuously acquires a face image of a person to be collated, and the acquisition unit. A face image extracting unit for extracting a face image, a feature amount extracting unit for extracting a feature amount from the face image extracted by the face image extracting unit, and a feature amount of the face image acquired continuously The number of features of the face image acquired continuously after the feature amount of the predetermined number of face images is stored as the dictionary data by the dictionary data storing means for storing the number of dictionary data as the number of dictionary data. A collation data storage means for storing a predetermined amount of collation data as collation data, a feature amount of a predetermined number of face images in the dictionary data, and a feature amount of a predetermined number of face images in the collation data. If not similar, the collation means for calculating and collating the similarity between the face image of the person to be collated acquired at that timing and the face image of the accumulator accumulated in the accumulation means, and the collation result of the collation means A computer that controls a monitoring device including similarity determination means for determining whether or not the face image of the person to be collated is the face image of the accumulator by comparing the similarity with a predetermined threshold; An acquisition step of continuously acquiring the face image of the person to be collated by the acquisition means, a face image extraction step of extracting the face image acquired by the processing of the acquisition step by the previous face image extraction step, and the characteristics A feature amount extracting step for extracting a feature amount from the face image extracted in the face image extracting step by the amount extracting unit; and a continuous acquisition by the dictionary data storing unit. A dictionary data storage step of storing a predetermined number of feature values of the face image as dictionary data, and a process of the dictionary data storage step by the collation data storage means, whereby the feature amount of the predetermined number of face images as the dictionary data A collation data storing step of storing a predetermined number of feature quantities of the face image continuously acquired as collation data after the data is stored, and a feature of the predetermined number of face images in the dictionary data by the collation means If the amount and the feature quantity of the predetermined number of face images in the matching data are not similar, the similarity between the face image of the person to be collated acquired at that timing and the face image of the person accumulated in the accumulation unit A collation step for calculating and collating the degree of similarity, and a comparison between the similarity that is the collation result of the collation step and a predetermined threshold by the similarity determination unit And a similarity determination step for determining whether or not the face image of the person to be collated is the face image of the accumulator.

  In one aspect of the present invention, a face image is accumulated in the accumulator database as an accumulator face image, the face image of the person to be collated is continuously acquired, the acquired face image is extracted, and the extracted face After the feature amount is extracted from the image, the feature amount of the face image obtained continuously is stored as a predetermined number of dictionary data, and the feature amount of the predetermined number of face images is stored as the dictionary data, A predetermined number of feature values of the face image acquired continuously are stored as collation data, and a predetermined number of face image feature amounts in the dictionary data and a predetermined number of face image feature amounts in the collation data Are not similar, the similarity between the face image of the person to be collated acquired at that timing and the accumulated face image of the accumulator is calculated and collated, and the similarity that is the collation result and a predetermined threshold value By comparison The verification subject's face image whether the face image of the storage user is determined.

  In the monitoring apparatus according to one aspect of the present invention, the accumulation unit that accumulates the face image as the accumulator face image in the accumulator database is, for example, a face image database, and continuously acquires the face image of the person to be collated. The acquisition unit is, for example, an image acquisition unit, and the face image extraction unit that extracts the face image acquired by the acquisition unit is, for example, a face image detection processing unit, and is extracted by the face image extraction unit. The feature amount extraction means for extracting feature amounts from the face image is, for example, a simple feature amount extraction unit, and dictionary data that stores a predetermined number of feature amounts of the face image acquired continuously as dictionary data. The storage means is, for example, a dictionary feature quantity table, and after the feature quantities of the predetermined number of face images are stored as the dictionary data by the dictionary data storage means, before being continuously acquired. The collation data storage means for storing a predetermined number of face image feature amounts as collation data is, for example, a collation feature amount table, and a predetermined number of face image feature amounts in the dictionary data and a predetermined amount in the collation data. When the feature amounts of the number of face images are not similar, collation is performed by calculating the degree of similarity between the face image of the person to be collated acquired at that timing and the face image of the person accumulated in the accumulation unit The means is, for example, a similarity calculation unit, and whether or not the face image of the person to be collated is the face image of the accumulator by comparing the similarity that is the collation result of the collation means with a predetermined threshold value. The similarity determination unit that determines whether or not is, for example, a similarity determination unit.

  That is, a face image is extracted from images continuously captured by the camera, and feature quantities are further extracted from the face images, and a predetermined number of face images are stored in the dictionary feature quantity table as dictionary data. Further, when the dictionary feature quantity table is satisfied, the feature quantity is stored in the collation feature quantity table as collation data for a predetermined number of face images thereafter, and those having a high degree of similarity are stored for the amount exceeding the predetermined number. Selectively stored in the matching feature table. Then, the feature amount stored in the dictionary feature amount table and the feature amount stored in the collation feature amount table are compared, and if they do not match, that is, the supplied face image changes, When a face image of a new person has been supplied, collation processing is executed for the face image at that timing.

  As a result, as long as it is determined that face images of the same person have been supplied, the matching process is not executed, so that the processing load in the face image matching process can be reduced.

  According to the present invention, it is possible to reduce the processing load in face image matching processing.

  Embodiments of the present invention will be described below. Correspondences between the configuration requirements of the present invention and the embodiments described in the detailed description of the present invention are exemplified as follows. This description is to confirm that the embodiments supporting the present invention are described in the detailed description of the invention. Accordingly, although there are embodiments that are described in the detailed description of the invention but are not described here as embodiments corresponding to the constituent elements of the present invention, It does not mean that the embodiment does not correspond to the configuration requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

  That is, the monitoring device according to one aspect of the present invention includes a storage unit (for example, the face image database 22 in FIG. 9) that stores face images in the store database as face images of store users, and a face of the person to be collated continuously. An acquisition unit (for example, the image acquisition unit 161 in FIG. 8) for acquiring an image, a face image extraction unit (for example, the face image detection processing unit 162 in FIG. 8) for extracting the face image acquired by the acquisition unit, A feature amount extraction unit (for example, the simple feature amount extraction unit 152 in FIG. 8) that extracts a feature amount from the face image extracted by the face image extraction unit, and a feature amount of the face image that is continuously acquired, The dictionary data storage means (for example, the dictionary feature value table 156 in FIG. 8) that stores a predetermined number of dictionary data as the dictionary data and the dictionary data storage means store the feature values of the predetermined number of face images as the dictionary data. After that, collation data storage means (for example, a collation feature quantity table 157 in FIG. 8) that stores a predetermined number of feature quantities of the face image continuously acquired as collation data, and a predetermined number in the dictionary data If the feature amount of the face image is not similar to the feature amount of the predetermined number of face images in the collation data, the face image of the person to be collated acquired at that timing and the accumulator stored in the storage means The collation target (for example, the similarity calculation unit 232 in FIG. 9) for calculating and collating the similarity with the face image, and comparing the similarity as a collation result of the collation unit with a predetermined threshold value Similarity determination means (for example, similarity determination unit 233 in FIG. 9) for determining whether or not a person's face image is the accumulator's face image.

  Feature amount comparison / determination means for determining whether or not the feature amount of the predetermined number of face images in the dictionary data is similar to the feature amount of the predetermined number of face images in the collation data (for example, 3 to 3 in FIG. 8) A collation unit 154) can be further included, and if the collation unit determines that the feature amount comparison determination unit does not resemble the face image of the person to be collated acquired at that timing, The similarity with the accumulator face image accumulated in the accumulation means can be calculated and collated.

  An average feature amount calculating means (for example, the averaging unit 174 in FIG. 8) for calculating an average of feature amounts of a predetermined number of face images in the dictionary data as an average feature amount can be further included. The amount comparison / determination unit compares each of the feature amounts of the predetermined number of face images in the collation data with the average feature amount. It can be determined that the feature amount and the feature amount of a predetermined number of face images in the collation data are not similar.

  If the feature quantity of the predetermined number of face images in the dictionary data and the feature quantity of the predetermined number of face images in the collation data are not similar, the dictionary data storage means (for example, the dictionary feature quantity of FIG. In the table 156), the feature amounts of the predetermined number of face images stored as the dictionary data are deleted, and the characteristics of the predetermined number of face images stored as the verification data are stored by the verification data storage unit. The quantity can be stored as new dictionary data.

  The dictionary data storage means (for example, the dictionary feature value table 156 in FIG. 8) stores the feature values of the face image continuously obtained and the dictionary data, and has the lowest priority. When the feature quantity is not similar, the feature quantity with the lowest priority stored as dictionary data can be updated with the feature quantity of the acquired face image.

  In the matching data storage means (for example, the matching feature value table 157 in FIG. 8), the feature values of the face image acquired continuously and the matching data are stored and have the lowest priority. When the feature amount is not similar, the feature amount with the lowest priority stored as the collation data can be updated with the feature amount of the acquired face image.

  The monitoring method according to one aspect of the present invention includes an accumulating unit that accumulates a face image as an accumulator's face image in an accumulator database, an acquisition unit that continuously acquires a face image of a person to be collated, and an acquisition unit that acquires the face image. A face image extracting unit that extracts the face image, a feature amount extracting unit that extracts a feature amount from the face image extracted by the face image extracting unit, and a feature amount of the face image acquired continuously, A dictionary data storage unit that stores a predetermined number of dictionary data as dictionary data, and a feature amount of the predetermined number of face images stored as the dictionary data by the dictionary data storage unit, and then the facial image acquired continuously A collation data storage unit that stores a predetermined number of feature quantities as collation data, a feature quantity of a predetermined number of face images in the dictionary data, and a feature quantity of a predetermined number of face images in the collation data If not, the similarity between the face image of the person to be collated acquired at that timing and the face image of the accumulator accumulated in the accumulating means is calculated and collated, and the collation result of the collating means A monitoring method of a monitoring device including similarity determination means for determining whether or not the face image of the person to be collated is the face image of the accumulator by comparing a certain similarity with a predetermined threshold, The acquisition step (for example, step S1 in FIG. 11) for continuously acquiring the face image of the person to be collated by the acquisition unit, and the face image acquired by the processing of the acquisition step by the previous face image extraction step are extracted. A feature image extraction step (for example, step S2 in FIG. 11) and a feature value extraction step for extracting feature values from the face image extracted by the feature value extraction means in the process of the face image extraction step. (For example, step S5 in FIG. 11) and a dictionary data storage step (for example, in FIG. 12) that stores a predetermined number of feature values of the facial image acquired continuously by the dictionary data storage means as dictionary data. In step S52, S54, S60) and the dictionary data storage step by the collation data storage means, the feature quantity of the predetermined number of face images is stored as the dictionary data, and then continuously acquired. A collation data storing step (for example, steps S82, S84, S90 in FIG. 13) for storing a predetermined number of feature quantities of the face image as collation data, and a predetermined number of face images in the dictionary data by the collation means. If the feature amount and the feature amount of the predetermined number of face images in the matching data are not similar, the person to be collated acquired at that timing A collation step (for example, step S103 in FIG. 14) for calculating and collating the similarity between the face image of the accumulator and the accumulator face image accumulated in the accumulating means, and the similarity as a collation result by the collation step And a predetermined threshold value to determine whether or not the face image of the person to be collated is the face image of the accumulator (for example, step S25 in FIG. 11).

  FIG. 1 is a diagram showing a configuration of an embodiment of a game shop monitoring system according to the present invention.

  The game stores 1-1 to 1-n are so-called pachinko stores, pachislot stores, or casino stores. These amusement stores 1-1 to 1-n are affiliated stores or affiliated stores of the face image management center or the third party amusement store management center, and stores that need to manage a plurality of stores in an integrated manner. It is. Each of the game stores 1-1 to 1-n is connected by a face image management bus 6 and a third party game store management bus 7. The public communication line networks 8 and 9 represented by these buses and the Internet are used. Via each other, face image information and third-party amusement store management information are exchanged. In the following description, when there is no particular need to distinguish each of the game stores 1-1 to 1-n, the game store 1 is simply referred to, and the other configurations are also referred to in the same manner.

  The face image management bus 6 mainly functions as a transmission path for distributing face image information managed by the face image recognition device 21 of each game store 1. The third-party game store management bus 7 functions as a transmission path for distributing medium lending management information managed mainly by the medium lending management device 27 of each amusement store 1.

  The face image management center 2 is a server used by a business operator who manages and manages the face image management center. The face image management center 2 manages the registered player DB managed by the face image management database (hereinafter also referred to as DB) 3. In addition to updating based on the unregistered player DB generated by each gaming store 1, the updated updated registered player DB is distributed to the face image recognition device 21 of each gaming store 1.

  The third-party amusement store management center 4 is a server used by a business operator that manages and operates the third-party amusement store management center, and the medium lending management managed by the third-party amusement store management database (DB) 5 In addition to updating the information DB based on the information supplied from each game store 1, the updated latest medium lending management information is distributed to the medium lending management device 27 of each game store 1.

  The face image recognition device 21 is extracted by the image processing units 39-1 to 39-m from the images captured by the cameras 38-1 to 38-m and supplied via the face image information bus 31. Based on this information, the face image registered in the face image information database 22 is collated and if it matches, the registered player's visit to the portable terminal 20 is notified, or an organic EL (Electronic Luminescence) or LCD ( The image is displayed on the display unit 23 including a liquid crystal display. If the face images registered in the face image information database 22 are collated and do not match, the face image recognition device 21 accesses the face image management database 3 and registers as an unregistered player in the unregistered player DB. To do.

  The game store management device 24 is a so-called hall computer, and monitors the operations of the game machines 36-1 to 36-m via the game store management information bus 30. The game store management device 24 determines a predetermined amount according to the monitoring status such as information on the payout of the game table 36 or the payout of medals, the call information of the players on the game tables 36-1 to 36-m, or the occurrence of an error. The above process is executed, and the execution result is displayed on the display unit 25 including an organic EL or LCD. The game store management device 24 identifies information supplied from the counter 35, the game machines 36-1 to 36-m, and the game machine peripheral terminals 37-1 to 37-m, respectively. For example, it is managed by the gaming machine management database 26 in association with the gaming machine number.

  The medium lending management device 27 manages the medium lending management information of the rented game media using the medium lending management database 29 based on the information from the payment machine 33 and the lending machine 34, and also the medium lending management database. When the medium lending management information registered in 29 is updated, the updated information is sent to the third party amusement store management center 4 via the third party amusement store management bus 7 and the public communication line network 9. Further, the medium lending management device 27 acquires the medium lending management information supplied by the third party amusement store management center 4 via the third party amusement store management bus 7 and the public communication line network 9, and manages the medium lending management. Store in the database 29.

  The lending machine 34 lends a number of game media according to the amount when the player accepts a predetermined amount by cash or a prepaid card when playing on the gaming table 36. At this time, the lending machine 34 supplies information on the number of lent gaming media to the medium lending management device 27 together with information such as the accepted cash and the remaining number of prepaid cards. Thereby, the medium lending management device 27 registers information on the number of lent gaming media in the medium lending management information database 29 together with information such as the received cash and the remaining number of prepaid cards.

  The checkout machine 33 sells prepaid cards with a frequency for borrowing a ball. At this time, the checkout machine 33 supplies the frequency of the prepaid card sold and the amount paid to the medium lending management device 27. Further, the settlement vending machine 33 settles and pays out cash based on the remaining number of game media lent as a frequency such as a prepaid card. At this time, the checkout machine 33 supplies the remaining number of prepaid cards and the amount of cash refunded to the medium lending management device 27.

  The counter 35 counts the number of game media acquired by the player playing on the game table 36, and outputs the counting result as a magnetic card or a receipt.

  The game machines 36-1 to 36-m execute a game when a predetermined operation is performed by the player, and pay out a game ball or a medal according to a so-called small hit or big hit.

  The game console peripheral terminals 37-1 to 37-m are so-called pedestrian machines provided corresponding to the respective game machines 36-1 to 36-m. The same as the lending machine 34) is provided. Further, the gaming machine peripheral terminal 37 acquires facial image information such as a facial image of a player who plays the gaming machine 36 and transmits it to the facial image recognition device 21 together with the gaming machine identification information (gaming machine number). FIG. 1 shows an example in which cameras 38-1 to 38-m for acquiring a player's face image are provided as a function for acquiring the face image information.

  For example, as shown in FIG. 2, the cameras 38-1 to 38-m are arranged below the table display lamps 61-1 to 61-4 provided on the respective upper sides of the game tables 36-1 to 36-4. As shown in FIG. 3, a face image may be taken so that the player can take an image within the reading range δ. Can be used.

  Further, as shown in FIG. 4, the cameras 38-1 to 38-m are provided with convex portions 71-1 to 71-4 on the gaming machine peripheral terminals 37-1 to 37-4, as shown in FIG. As described above, it may be provided so that a player's face image can be captured within the reading range θ.

  Further, for example, as shown in FIG. 6, the cameras 38-1 to 38-m may be provided at the center of the game table 36 (on the board surface of the game table 36) to take an image. That is, by installing the camera 38 in the installation unit 81 of FIG. 6, the player is imaged within the reading range φ as shown in FIG.

  Next, a configuration example of the image processing unit 39 will be described with reference to FIG.

  The image processing unit 39 includes a face detection unit 151, a simple feature value extraction unit 152, a table management unit 153, a 3 to 3 matching unit 154, an output unit 155, a dictionary feature value table 156, and a matching feature value table 157. .

  The face detection unit 151 extracts a face image from the image captured by the camera 38 and supplies the face image to the simple feature amount extraction unit 152 and the output unit 155. The simple feature amount extraction unit 152 extracts a feature amount from the face image supplied from the face detection unit 151. Note that “simple” in the simple feature quantity extraction unit 152 is for distinguishing from the feature quantity extraction unit 231 of the matching unit 222 in the face image recognition device 21 described later. That is, the feature quantity extraction unit 231 of the matching unit 222 in the face image recognition device 21 is used for matching face images, and therefore uses a great number of types and patterns. Extracts feature values only for determining whether or not a face image is appropriate for the face image authentication process. Therefore, the types of extracted feature values are small and simple. It is.

  When a new feature amount is supplied, the table management unit 153 sequentially supplies a dictionary feature amount table 156 that stores the feature amounts of three reference face images based on the feature amount. A collation feature quantity table 157 that stores the feature quantities of the three face images of the person to be collated is managed.

  The three-to-three matching unit 154 includes the average face image feature value of the three face images serving as the reference stored in the dictionary feature value table 156 and the person to be matched stored in the matching feature value table 157. The three face images are collated, and the collation result is supplied to the output unit 155 and the table management unit 153.

  When a new feature amount is supplied, the table management unit 153 sequentially supplies a dictionary feature amount table 156 that stores the feature amounts of three reference face images based on the feature amount. A collation feature quantity table 157 that stores the feature quantities of the three face images of the person to be collated is managed.

  The three-to-three matching unit 154 includes the average face image feature value of the three face images serving as the reference stored in the dictionary feature value table 156 and the person to be matched stored in the matching feature value table 157. The three face images are collated, and the collation result is supplied to the output unit 155 and the table management unit 153.

  The output unit 155 causes the face image recognition device 21 to output the face image supplied from the face detection unit 151 when the collation results of the 3 to 3 collation unit 154 do not match.

  The dictionary feature quantity table 156 sequentially stores the feature quantities of the face image continuously supplied from the simple feature quantity extraction unit 152, the dictionary upper feature quantity buffer 171, the dictionary middle feature quantity buffer 172, and the dictionary lower feature quantity buffer 173. Let me remember. At this time, if the feature value is buffered in the dictionary upper feature amount buffer 171, if the feature amount is not buffered in the dictionary middle feature amount buffer 172, the supplied feature amount is the dictionary middle feature amount. Buffered in the buffer 172. Here, the feature amount buffered in the dictionary upper feature amount buffer 171 and the feature amount buffered in the dictionary middle feature amount buffer 172 are compared, and when there is a difference, that is, the degree of similarity between them. Is low, the feature value buffered in the dictionary middle feature amount buffer 172 is overwritten and buffered in the dictionary upper feature amount buffer 171 and is also buffered in the dictionary middle feature amount buffer 172. The amount is deleted.

  Similarly, when the feature value is buffered in the dictionary middle feature value buffer 172, if the feature value is not buffered in the dictionary lower feature value buffer 173, the supplied feature value is changed to the dictionary lower feature value buffer. Buffered at 173. Here, the feature quantity buffered in the dictionary upper feature quantity buffer 171 and the feature quantity buffered in the dictionary lower feature quantity buffer 173 are compared, and when there is a difference, that is, the degree of similarity between them. Is low, the feature quantity buffered in the dictionary low-order feature quantity buffer 173 is overwritten and buffered in the dictionary high-order feature quantity buffer 171 and is also stored in the dictionary middle-order feature quantity buffer 172 and the dictionary low-order feature quantity buffer 173. The buffered feature is deleted.

  The dictionary feature quantity table 156 is supplied with similar feature quantities three times in succession when information of three feature quantities is buffered in the dictionary upper feature quantity buffer 171 to the dictionary lower feature quantity buffer 173. This is stored as a feature value of the reference face image, that is, a feature value to be stored as a dictionary. Further, the dictionary feature quantity table 156 is stored in accordance with the feature quantities buffered in the collation upper feature quantity buffer 181 to the collation lower feature quantity buffer 183 of the collation feature quantity table 157 described later by the table management unit 153 as necessary. The dictionary upper feature buffer 171 to the dictionary lower feature buffer 173 are overwritten and updated.

  The averaging unit 174 obtains an average of the feature values stored in the upper dictionary feature value buffer 171 to the lower dictionary feature value buffer 173 to obtain an average face image feature value, and buffers it in the average face image feature value buffer 175. Let

  The collation feature value table 157 includes the dictionary upper feature value buffer 171 to the dictionary lower feature value of the dictionary feature value table 156 among the feature values of the face image continuously supplied from the simple feature value extracting unit 152. Only when the feature values are stored in all of the buffers 173, the matching upper feature value buffer 181, the matching middle feature value buffer 182, and the matching lower feature value buffer 183 are sequentially stored.

  At this time, if the feature amount is buffered in the matching upper feature amount buffer 181, if the feature amount is not buffered in the matching middle feature amount buffer 182, the feature amount buffer 182 has been supplied to the matching middle feature amount buffer 182. Features are buffered. Here, the feature quantity buffered in the matching upper feature quantity buffer 181 is compared with the feature quantity buffered in the matching middle feature quantity buffer 182, and when there is a difference, that is, mutual similarity. When the degree is low, the feature value buffered in the matching middle feature amount buffer 182 is overwritten and buffered in the matching upper feature amount buffer 181 and is also buffered in the matching middle feature amount buffer 182. The feature amount is deleted.

  Similarly, when the feature value is buffered in the matching middle feature value buffer 182, if the feature value is not buffered in the matching lower feature value buffer 183, the supplied feature value is the matching lower feature value buffer. Buffered at 183. Here, the feature quantity buffered in the matching upper feature quantity buffer 181 and the feature quantity buffered in the matching lower feature quantity buffer 183 are compared, and when there is a difference, that is, the degree of similarity between them. Is low, the feature value buffered in the matching lower feature value buffer 183 is overwritten and buffered in the matching upper feature value buffer 181, and is also stored in the matching middle feature value buffer 182 and the matching lower feature value buffer 183. The buffered feature is deleted.

  The matching feature value table 157 is supplied with similar feature values three times in succession when information on three feature values is buffered in the matching upper feature value buffer 171 to the matching lower feature value buffer 173. This is stored as a feature amount of the face image of the person to be collated.

  Next, a configuration example of the face image recognition device 21 will be described with reference to FIG.

  The face image acquisition unit 221 acquires the face image supplied from the image processing unit 39 and supplies it to the collation unit 222. The collation unit 222 collates the face image acquired by the face image acquisition unit 221 with the registered player's face image registered in advance in the face image information DB 22, and finds a face image that is a candidate with high similarity. For example, the face images up to the third candidate are displayed on the display unit 23 as a matching result. Further, the collation unit 222 supplies the supplied face image to the unregistered player database registration unit 223 when there is no face image that is a candidate with high similarity.

  More specifically, the feature amount extraction unit 231 of the matching unit 222 extracts a feature amount for identifying a face image, and supplies the feature amount together with the face image to the similarity calculation unit 232. The similarity calculation unit 232 extracts the feature amount of the registered player's face image registered in the face image information DB 22 and uses the feature amount supplied from the feature amount extraction unit 231 to obtain the face image. The degree of similarity with the face images of all registered players registered in the information DB 22 is obtained, and the face images supplied from the face image acquisition unit 221 and the face images having the highest degree of similarity are the similarity degrees. It supplies to the determination part 233. More specifically, the similarity calculation unit 232, for example, based on various facial feature amounts such as the ratio between the distance between the eyes, the distance from the chin to the forehead, and the length from the chin to the nose, Each difference sum, average ratio, or ratio sum is obtained as the similarity.

  The similarity determination unit 233 sequentially accumulates the similarities supplied from the similarity calculation unit 232 in the buffer 233a, and with respect to each of the similarities of the top three face images, When the similarity is compared with a predetermined threshold, and the registered face image that is ranked first is similar to the face image supplied from the face image acquisition unit 221 based on the comparison result (similarity When the similarity is higher than the predetermined threshold in the case of similarity indicating that the similarity is higher, or when the similarity is lower than the predetermined threshold in the case of similarity indicating that the similarity is lower as the similarity is lower. Information on the similarity between the face image and the top three is supplied to the display unit 23, and is supplied to the communication unit 224 when displayed. In addition, the similarity determination unit 233 compares the similarity between the face image ranked first and a predetermined threshold value, and the registered face image ranked first is compared with the face image acquisition unit 221 based on the comparison result. When the face image supplied is not similar, the face image supplied from the face image acquisition unit 221 is supplied to the unregistered player database registration unit 223.

  The unregistered player database registration unit 223 registers the face image that has been supplied as being unregistered by the matching unit 222 in the unregistered player DB of the face image management database 3.

  The operation unit 225 includes buttons, a mouse, a keyboard, and the like. The operation unit 225 is operated when any one of the face images displayed on the display unit 23 with the above-mentioned similarity ranked in the top three is selected. Is supplied to the communication unit 224. The communication unit 224 includes a modem or the like, and distributes the selected face image to the mobile terminal 20 based on an operation signal from the operation unit 225.

  Here, for example, the similarity indicates a higher value as it is closer to the face image registered as a registered player as indicated by the ratio sum, and the similarity is a value higher than a predetermined threshold value. In this case, an example will be described in which it is determined that the face image of the registered player corresponding to the similarity is. However, for example, when the similarity is expressed as a difference sum in each feature amount between the captured face image and the face image registered as the registered player, the similarity determination unit 233 determines that the similarity is a threshold value. If it is smaller than that, the captured face image is regarded as a registered player's face image, or in the case of an average ratio or the like, it is greater than or equal to a predetermined value in the range of 0 to 1 and close to 1. If it is a value, it can be considered that they are the same person.

  When a new registered player database is distributed from the face image management center 2, the database management unit 226 updates the face image information DB 22 based on the new registered player database.

  Next, a configuration example of the mobile terminal 20 will be described with reference to FIG.

  The communication unit 271 is configured by a modem or the like, and exchanges data with the face image recognition device 21 via a wireless communication network in the game store 1. In addition, the communication unit 271 acquires information indicating that a player with a facial image similar to the facial image supplied from the image processing unit 39 and distributed from the facial image recognition device 21 has visited the store, and the image processing unit 272.

  The image processing unit 272 is a display constituted by an LCD or the like based on information indicating that a player with a face image similar to the face image supplied from the image processing unit 39 supplied from the communication unit 271 has visited the store. In addition to generating an image to be displayed on the unit 273, the image is displayed on the display unit 273.

  Next, the registered player visit monitoring process will be described with reference to the flowchart of FIG.

  In step S <b> 1, the camera 38 captures an image within a predetermined range from the installed position and supplies the image to the image processing unit 39. The image acquisition unit 161 of the face detection unit 151 supplies the image supplied from the camera 38 to the face image detection processing unit 162.

  In step S <b> 2, the face image detection processing unit 162 extracts the player's face image from the supplied image, and supplies it to the face image acquisition unit 163. More specifically, the face image detection process 162 extracts a face image from, for example, the arrangement of characteristic parts such as eyes and nose, where the skin is exposed from the color of the captured image. And supplied to the face image acquisition unit 163.

  In step S <b> 3, the face image acquisition unit 163 determines whether the face image has been extracted based on whether the face image has been supplied from the face image detection processing unit 162. If no face image is supplied in step S3, it is considered that no face image has been extracted, and the process returns to step S1. That is, the processes in steps S1 to S3 are repeated until a face image is detected.

  On the other hand, when it is determined in step S3 that a face image has been extracted, in step S4, the face image acquisition unit 163 acquires the supplied face image and supplies it to the face image cutout unit 164. The face image cutout unit 164 supplies the face image supplied from the face image acquisition unit 163 to the simple feature amount extraction unit 152 and the output unit 155.

  In step S <b> 5, the simple feature amount extraction unit 152 extracts a simple feature amount from the supplied face image and supplies the extracted feature amount to the table management unit 153, the dictionary feature amount table 156, and the matching feature amount table 157.

  In step S6, a replacement determination process is executed by the table management unit 153 and the 3-to-3 verification unit 154, and a replacement has occurred in a person existing in the imaging range of the camera 38 based on the supplied face image. It is determined whether or not.

  Here, the replacement determination process will be described with reference to the flowchart of FIG.

  In step S51, the one-to-one matching unit 191 of the table management unit 153 has pre-buffered feature quantities in addition to the currently supplied feature quantities in the dictionary upper feature quantity buffer 171 of the dictionary feature quantity table 156. Determine whether or not. In step S51, for example, in the first process, when the feature quantity is not buffered in the dictionary upper feature quantity buffer 171, in step S52, the one-to-one matching unit 191 controls the registration unit 191a, The dictionary upper feature amount buffer 171 buffers the supplied feature amount, and the process proceeds to step S66. At this time, the other buffers are controlled so that the state before the new feature amount is supplied is maintained.

  On the other hand, if the feature amount is already buffered in the dictionary upper feature amount buffer 171 in step S51, the one-to-one matching unit 191 is currently supplied to the dictionary middle feature amount buffer 172 in step S53. It is determined whether or not feature quantities other than the received feature quantities are buffered in advance. In step S53, for example, when no feature value is buffered in the dictionary intermediate feature value buffer 172, in step S54, the one-to-one matching unit 191 controls the registration unit 191a to control the dictionary intermediate feature value buffer. In 172, the supplied feature value is buffered. At this time, the other buffers are controlled so that the state before the new feature amount is supplied is maintained.

  In step S <b> 55, the one-to-one matching unit 191 calculates the similarity between the feature value buffered in the dictionary middle feature value buffer 172 and the feature value buffered in the dictionary higher feature value buffer 171. .

  In step S56, the one-to-one matching unit 191 determines whether or not the calculated similarity is a value greater than a predetermined threshold. Here, it is assumed that the similarity is higher as the two feature quantities are similar. If it is determined in step S56 that the degree of similarity is not higher than the predetermined feature amount, that is, it is highly likely that the face image is a person having different feature amounts, the process proceeds to step S57.

  In step S57, the one-to-one matching unit 191 controls the registration unit 191a to use the feature value buffered in the dictionary middle feature amount buffer 172 and the feature buffered in the dictionary upper feature amount buffer 171. Update by overwriting the amount.

  In step S58, the one-to-one matching unit 191 deletes the feature amount stored in the dictionary middle feature amount buffer 172, and the process proceeds to step S66.

  If the similarity is higher than the predetermined value in step S56, the processes in steps S57 and S58 are skipped.

  On the other hand, if the feature amount is already buffered in the dictionary middle feature amount buffer 172 in step S53, the one-to-one matching unit 191 is currently supplied to the dictionary lower feature amount buffer 173 in step S59. It is determined whether or not feature quantities other than the received feature quantities are buffered in advance. In step S59, for example, when the feature quantity is not buffered in the dictionary lower feature quantity buffer 173, in step S60, the one-to-one matching unit 191 controls the registration unit 191a to store the feature quantity in the dictionary lower feature quantity buffer 173. The supplied feature value is buffered. At this time, the other buffers are controlled so that the state before the new feature amount is supplied is maintained.

  In step S <b> 61, the one-to-one matching unit 191 calculates the similarity between the feature quantity buffered in the dictionary lower feature quantity buffer 173 and the feature quantity buffered in the dictionary upper feature quantity buffer 171.

  In step S62, the one-to-one matching unit 191 determines whether the calculated similarity is a value greater than a predetermined threshold value. If it is determined in step S62 that the degree of similarity is not higher than the predetermined feature amount, that is, it is highly likely that the face images of the persons have different feature amounts, the process proceeds to step S63.

  In step S <b> 63, the one-to-one matching unit 191 controls the registration unit 191 a so that the feature value buffered in the dictionary upper feature amount buffer 171 is the feature amount buffered in the dictionary lower feature amount buffer 173. Overwrite and update.

  In step S64, the one-to-one matching unit 191 deletes the feature values stored in the dictionary middle feature value buffer 172 and the dictionary lower feature value buffer 173, and the process proceeds to step S66.

  If the similarity is higher than the predetermined value in step S62, the processes in steps S63 and S64 are skipped.

  That is, by the above processing, the dictionary upper feature amount buffer 171 to the dictionary lower feature amount buffer 173 of the dictionary feature amount table 156 have a feature amount similarity higher than a predetermined value, that is, face images considered to be the same person. Are supplied three times in succession, the feature amounts of the three types of face images are buffered as a dictionary.

  In step S59, if the feature amount is already buffered in the dictionary lower feature amount buffer 173, that is, the dictionary upper feature amount buffer 171 to the dictionary lower feature amount buffer 173 of the dictionary feature amount table 156 have 3 as a dictionary. If the type of feature quantity is buffered, the process proceeds to step S65, and the collation replacement determination process is executed.

  Here, the collation replacement determination process will be described with reference to the flowchart of FIG.

  In step S81, the one-to-one matching unit 191 of the table management unit 153 has pre-buffered feature quantities in addition to the currently supplied feature quantities in the matching upper feature quantity buffer 181 of the matching feature quantity table 157. Determine whether or not. In step S81, for example, if the feature quantity is not buffered in the collation upper feature amount buffer 181, in step S82, the one-to-one collation unit 191 controls the registration unit 191a to store in the collation upper feature amount buffer 181. The supplied feature value is buffered, and the process proceeds to step S66 (FIG. 12). At this time, the other buffers are controlled so that the state before the new feature amount is supplied is maintained.

  On the other hand, if the feature value is already buffered in the matching upper feature value buffer 181 in step S81, the one-to-one matching unit 191 is currently supplied to the matching middle feature value buffer 182 in step S83. It is determined whether or not feature quantities other than the received feature quantities are buffered in advance. In step S83, for example, if the feature quantity is not buffered in the matching intermediate feature quantity buffer 182, in step S84, the one-to-one matching section 191 controls the registration section 191a to check the matching middle feature quantity buffer. By controlling 182, the supplied feature value is buffered. At this time, the other buffers are controlled so that the state before the new feature amount is supplied is maintained.

  In step S <b> 85, the one-to-one matching unit 191 calculates the similarity between the feature value buffered in the matching middle feature value buffer 182 and the feature value buffered in the matching higher feature value buffer 181. .

  In step S86, the one-to-one matching unit 191 determines whether the calculated similarity is a value greater than a predetermined threshold. In this case as well, the similarity is assumed to take a higher value as the two feature quantities are similar. If it is determined in step S86 that the degree of similarity is not higher than the predetermined feature amount, that is, it is highly likely that the face images are of different persons, the process proceeds to step S87.

  In step S <b> 87, the one-to-one matching unit 191 controls the registration unit 191 a, and features that are buffered in the matching upper feature value buffer 181 with the feature values buffered in the matching middle feature value buffer 182. Update by overwriting the amount.

  In step S88, the one-to-one matching unit 191 deletes the feature amount stored in the matching middle feature amount buffer 182, and the process proceeds to step S66 (FIG. 12).

  If the similarity is higher than the predetermined value in step S86, the processes in steps S87 and S88 are skipped.

  On the other hand, if the feature value is already buffered in the matching middle feature value buffer 182 in step S83, the one-to-one matching unit 191 is currently supplied to the matching lower feature value buffer 183 in step S89. It is determined whether or not feature quantities other than the received feature quantities are buffered in advance. In step S89, for example, when the feature quantity is not buffered in the collation lower feature amount buffer 183, in step S90, the one-to-one collation unit 191 controls the registration unit 191a to store in the collation lower feature amount buffer 183. The supplied feature value is buffered. At this time, the other buffers are controlled so that the state before the new feature amount is supplied is maintained.

  In step S <b> 91, the one-to-one matching unit 191 calculates the similarity between the feature value buffered in the matching middle feature value buffer 182 and the feature value buffered in the matching middle feature value buffer 182. To do.

  In step S92, the one-to-one matching unit 191 determines whether the calculated similarity is a value greater than a predetermined threshold value. If it is determined in step S92 that the degree of similarity is not higher than the predetermined feature amount, that is, it is highly likely that the face image is a person having a different feature amount, the process proceeds to step S93.

  In step S <b> 93, the one-to-one matching unit 191 controls the registration unit 191 a to use the feature values buffered in the matching lower feature value buffer 183 and the feature values buffered in the matching higher feature value buffer 181. Overwrite and update.

  In step S94, the one-to-one matching unit 191 deletes the feature values stored in the matching middle feature value buffer 182 and the matching lower feature value buffer 183, and the process proceeds to step S95.

  If the similarity is higher than the predetermined value in step S92, the processes in steps S93 and S94 are skipped.

  That is, by the above processing, the matching upper feature amount buffer 181 to the matching lower feature amount buffer 183 of the matching feature value table 157 have a feature value similarity higher than a predetermined value, that is, face images considered to be the same person. Are supplied three times in succession, the feature values of the three types of face images of the person to be collated are buffered.

  In step S89, if the feature value is already buffered in the collation lower feature amount buffer 183, that is, 3 as a dictionary in the collation upper feature amount buffer 181 to the collation lower feature amount buffer 183 of the collation feature amount table 157. If the type of feature quantity is buffered, the process proceeds to step S66 (FIG. 12).

  Now, the description returns to the flowchart of FIG.

  In step S66, the table updating unit 192 accesses the dictionary feature quantity table 156 and the collation feature quantity table 157, and the dictionary upper feature quantity buffer 171 to the dictionary lower feature quantity buffer 173 and the collation upper feature quantity buffer 181 to the collation lower order. It is determined whether or not feature quantities are buffered in all feature quantity buffers 183. In step S66, for example, when it is determined that the feature quantity is buffered in all of the dictionary upper feature quantity buffer 171 to the dictionary lower feature quantity buffer 173 and the collation upper feature quantity buffer 181 to the collation lower feature quantity buffer 183. The table updating unit 192 notifies the 3-to-3 matching unit 154 and the averaging unit 174 to that effect, and the process proceeds to step S67.

  In step S67, the averaging unit 174 averages the feature values buffered in the dictionary upper feature value buffer 171 to the dictionary lower feature value buffer 173, and causes the average face image feature value buffer 175 to buffer the feature values.

  In step S68, the 3-to-3 matching unit 154 reads the average face image feature value from the average face image feature value buffer 175, and the feature values buffered in the matching upper feature value buffer 181 to the matching lower feature value buffer 183 Are collated respectively. That is, the 3-to-3 matching unit 154 calculates three types of similarity for the average face image feature value and the feature values buffered in the matching upper feature value buffer 181 to the matching lower feature value buffer 183, respectively.

  In step S69, the 3 to 3 matching unit 154 determines whether or not all of the three types of feature amounts are larger than a predetermined value, that is, the three types of feature amounts of the face image buffered in the dictionary feature amount table 156. Are similar to the feature quantities buffered in the matching upper feature quantity buffer 181 to the matching lower feature quantity buffer 183 of the matching feature quantity table 157, and the person of the feature quantity registered as a dictionary Then, it is determined whether or not the matching target person of the feature quantity buffered in the matching feature quantity table 157 is the same.

  In step S69, for example, when it is determined that they are the same person, in step S70, the 3-to-3 collation unit 154 has not switched to a person existing within the imaging range captured by the camera 38. This is notified to the output unit 155 and the table update unit 192.

  On the other hand, in step S69, for example, when it is determined that they are not the same person, in step S71, the 3-to-3 collation unit 154 switches to a person existing within the imaging range captured by the camera 38. To the output unit 155 and the table update unit 192.

  Now, the description returns to the flowchart of FIG.

  In step S <b> 7, the output unit 154 and the table update unit 192 determine whether or not a replacement has occurred according to the notification content from the 3-to-3 verification unit 154.

  In step S7, for example, if it is determined by the processing in step S70 that no replacement has occurred, in step S8, the table update unit 192 causes the collation upper feature amount buffer 181 to the collation lower order in the collation feature amount table 157. The feature quantity buffered in the feature quantity buffer 183 is deleted. That is, it is in a state of newly accepting three types of feature amounts of the person to be verified. At this time, since there is no change in the person of the face image captured by the camera 38, the output unit 154 discards the face image supplied from the face image cutout unit 164 of the face detection unit 151 and recognizes the face image. It is not supplied to the device 21.

  On the other hand, if it is determined in step S7 that, for example, a change has occurred by the processing in step S71, the output unit 155 outputs the face supplied from the face image cutout unit 164 of the face detection unit 151 in step S9. The image is supplied to the face image recognition device 21. That is, the occurrence of replacement means that the face image of the same person has not been subjected to the face image authentication processing so far, and is supplied to the face image recognition device 21 as a new face image. Note that the first face image is unconditionally supplied to the face image recognition device 21.

  In step S10, the table updating unit 192 uses the feature values buffered in the collation upper feature amount buffer 181 to the collation lower feature amount buffer 183 of the collation feature amount table 157, and the dictionary upper feature amount buffer of the dictionary feature amount table 156. 171 thru | or the dictionary low-order feature-value buffer 173, and also deletes the feature-value buffered by the collation high-order feature-value buffer 181 thru | or the collation low-order feature-value buffer 183 of the collation feature-value table 157 after that, Return to step S1. As a result of this processing, the feature quantities buffered in the collation upper feature quantity buffer 181 to the collation lower feature quantity buffer 183 of the collation feature quantity table 157 as the feature quantity of the new person face image are used as the reference person face. The image features are buffered in the dictionary upper feature buffer 171 through the dictionary lower feature buffer 173 of the dictionary feature table 156, and the subsequent processing is repeated.

  In other words, the face image is supplied from the image processing unit 39 to the face image recognition device 21 only when it is determined that the face image is newly replaced by the above processing. For this reason, it is possible to reduce the number of times of authentication processing using a face image, which will be described later, for the face image recognition device 21.

  In step S21, the face image acquisition unit 221 of the face image recognition device 21 acquires the face image supplied by the process in step S10. In step S <b> 22, the face image acquisition unit 221 extracts any unprocessed one of the supplied face images and supplies the extracted unprocessed one to the feature amount extraction unit 231.

  In step S23, the feature amount extraction unit 231 of the collation unit 222 extracts the feature amount from the supplied face image, and supplies the feature amount together with the face image to the similarity calculation unit 232.

  In step S24, the similarity calculation unit 232 executes similarity calculation processing.

  Here, the similarity calculation processing will be described with reference to the flowchart of FIG.

  In step S101, the similarity calculation unit 232 extracts a face image of one unprocessed registered player from the face images registered in the registered player DB of the face image information DB 22, and sets it as a processing target. .

  In step S102, the similarity calculation unit 232 extracts a feature amount similar to the feature amount supplied from the feature amount extraction unit 231 from the face image registered in the registered player DB set as the processing target. .

  In step S <b> 103, the similarity calculation unit 232, for the face image supplied from the feature amount extraction unit 231, the ratio of the eye-to-eye spacing, the length from the chin to the forehead, and the length from the chin to the nose, etc. Using the various feature values of the various faces and the similar feature values in the face image registered in the registered player DB registered in the face image information DB 22, the difference sum, the average ratio, or the ratio sum of each. In step S104, the similarity with the face image registered in the registered player DB, which is the calculation result, is supplied to the similarity determination unit 233 and registered in the buffer 233a.

  In step S105, the similarity calculation unit 232 determines whether or not an unprocessed registered player's face image exists in the registered player DB of the face image information DB 22, and the unprocessed registered player's face image is determined. If it exists, the process returns to step S101. That is, the processes of steps S101 to S105 are repeated until the similarity with the face images of all registered players in the registered player DB is calculated. In step S105, when it is determined that there is no unprocessed registered player face image in the registered player DB, the similarity calculation processing ends.

  Now, the description returns to the flowchart of FIG.

  In step S25, the similarity determination unit 233 obtains a rank based on the calculation result of the similarity registered in the buffer 233a, and faces up to the top three of the face images registered in the registered player DB. Extract information about image and similarity. Then, the similarity determination unit 233 determines whether or not the highest similarity is greater than a predetermined threshold based on the top three face images and similarity information. That is, the similarity determination unit 233 registers the most similar registered player (of the face images registered in the face image information DB 22 and most similar to the face image acquired by the face image acquisition unit 221). Player: Here, the similarity of the registered player having the highest similarity is compared with a predetermined threshold.

  Note that, as described above, the similarity between the face image of the registered player who is most similar to the captured face image is not necessarily the highest because of the definition of the similarity. And the threshold value may differ from those in this example.

  If it is determined in step S25 that the highest similarity is greater than a predetermined threshold, in step S26, the similarity determination unit 233 determines that the top third facial image supplied from the similarity calculation unit 232 is the top three. A notification screen 301 indicating that it is a candidate for a registered player's face image is displayed by controlling the display unit 23.

  At this time, for example, a notification screen 301 as shown in FIG. 15 is displayed on the display unit 23.

  In the notification screen 301 of FIG. 15, the camera image display field 311 is provided in the middle left, and the face image supplied from the image processing unit 39 is displayed. Also, on the right side, face image display fields 312-1 to 312-3 of registered players of the top three in the similarity from the first candidate to the third candidate are provided in descending order of similarity. Further, similarity level display fields 313-1 to 313-3 are provided below the face image display fields 312-1 to 312-3 of each registered player, and the level of similarity is displayed. . In FIG. 15, the horizontal length of the area shown in black indicates the magnitude of the similarity.

  Further, ID display fields 314-1 to 314-3 are provided at corresponding positions below the similarity level display fields 313-1 to 313-3, and the face in the face image information DB 22 of each face image is provided. IDs for identifying images are displayed. In FIG. 15, “00051”, “00018”, and “00022” are displayed from the left.

  Furthermore, below the ID display fields 314-1 to 314-3, confirmation buttons 318-1 to 318-3 operated by the operation unit 225 when each candidate is selected are provided at the corresponding positions. It has been.

  A camera ID display field 315 for identifying the camera that captured the face image is provided under the camera image display field 311. In FIG. 15, the camera 38, the entrance camera 40, and the in-store camera 41 are displayed. “Camera 02” is displayed as a camera ID for identification. Further, a time display field 316 is provided below the camera ID display field 315, and the time taken by the entrance camera 40 is displayed. In FIG. 15, “18:23:32” is displayed. It is shown that the face image in the camera image display field 311 is captured at 18:23:32.

  Furthermore, another person button 317 is provided below the time display field 316, and the face image display fields 312-1 to 312- of the registered player whose face images of the camera image are the first to third candidates. 3 is operated by the operation unit 225 when it is regarded as not similar to any of 3.

  In step S27, the communication unit 224 displays on the display unit 23 whether or not any of the candidate face images has been selected by operating the operation unit 225, for example, the notification screen 301 shown in FIG. If it has been determined, it is determined whether any of the confirmation buttons 318-1 to 318-3 has been operated by the operation unit 225.

  In step S27, for example, when the confirm button 318-1 is operated, it is considered that the face image to be the first candidate has been selected, and in step S28, the communication unit 224 has the selected face image to be the first candidate. And the camera image imaged with the entrance camera 40 is transmitted to the portable terminal 20, and it notifies that the corresponding registered player came to the store.

  In step S41, the communication unit 271 determines whether or not the registered player has been notified of the store visit, and repeats the process until the notification is received. For example, when a registered player's visit is notified in step S41 by the process of step S28, the communication unit 271 notifies the registered player's visit sent from the face image recognition device 21 in step S42. , The face image of the registered player and the camera image captured by the entrance camera 40 transmitted together with the notification are supplied to the image processing unit 272. The image processing unit 272 processes information on the selected face image and the camera image captured by the entrance camera 40 into information in a format that can be displayed on the display unit 273, and displays the information on the display unit 273 in step S43. .

  By the above processing, the staff in the game store 1 can recognize the registered player's visit if the portable terminal 20 is possessed.

  In step S29, the face image acquisition unit 221 determines whether or not all of the supplied face images have been processed. If there is an unprocessed face image, the process returns to step S22. That is, the processes in steps S22 to S30 are repeated until the process is performed on all face images. If it is determined that the process has been completed for all face images, the process returns to step S21.

  On the other hand, no candidate face image is selected in step S27. For example, when the other person button 317 on the notification screen 301 in FIG. 15 is pressed, or in step S25, the similarity calculation unit 232 supplies the face image. Based on the information of the top three face images and the similarity, the highest similarity is not greater than a predetermined threshold, that is, even the face image of the most similar registered player If the similarity is less than the predetermined threshold, the similarity determination unit 233 supplies the face image supplied from the image processing unit 39 to the unregistered player database registration unit 223 in step S30. The unregistered player database registration unit 223 accesses the face image management database 3 through the biometric management information bus 6 and the public communication network 8 to register the supplied face image in the unregistered player DB.

  As a result of such processing, if the face image supplied from the image processing unit 39 by the face image recognition device 21 is not registered in the face image information DB 22, the face managed by the face image management center 2 is used. The face image of the unregistered player is registered in the unregistered player DB in the image management DB 3.

  With the above processing, among the feature values of the face image that are continuously supplied, three similar feature values that are first continuous after supplying the face image are used as dictionary information, and then supplied continuously. By comparing three types of similar feature quantities with the information of the person to be collated, it is possible to detect whether or not a person is replaced within the range imaged by the camera 38. As a result, the image processing unit 39 supplies only the face image supplied from the face image extracted from the image captured by the camera 38 to the face image authentication device 21 at the timing when the replacement occurs. Thus, it is possible to reduce the load of the face image authentication process in the face image authentication device 21.

  Incidentally, the series of monitoring processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.

  FIG. 16 shows a configuration example of a general-purpose personal computer. This personal computer incorporates a CPU (Central Processing Unit) 1001. An input / output interface 1005 is connected to the CPU 1001 via the bus 1004. A ROM (Read Only Memory) 1002 and a RAM (Random Access Memory) 1003 are connected to the bus 1004.

  The input / output interface 1005 includes an input unit 1006 including an input device such as a keyboard and a mouse for a user to input an operation command, an output unit 1007 for outputting a processing operation screen and an image of a processing result to a display device, a program and various data. A storage unit 1008 including a hard disk drive for storing data, a LAN (Local Area Network) adapter, and the like, and a communication unit 1009 for performing communication processing via a network represented by the Internet are connected. Also, a magnetic disk (including a flexible disk), an optical disk (including a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc)), a magneto-optical disk (including an MD (Mini Disc)), or a semiconductor A drive 1010 for reading / writing data from / to a removable medium 1011 such as a memory is connected.

  The CPU 1001 is read from a program stored in the ROM 1002 or a removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, installed in the storage unit 1008, and loaded from the storage unit 1008 to the RAM 1003. Various processes are executed according to the program. The RAM 1003 also appropriately stores data necessary for the CPU 1001 to execute various processes.

  In this specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series in the order described, but of course, it is not necessarily performed in time series. Or the process performed separately is included.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

It is a figure which shows the structure of one Embodiment of the monitoring system to which this invention is applied. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure explaining the structural example of the image processing unit of FIG. It is a figure explaining the structural example of the face image information recognition apparatus of FIG. It is a figure explaining the structural example of the portable terminal of FIG. It is a flowchart explaining a registered player visit monitoring process. It is a flowchart explaining a replacement determination process. It is a flowchart explaining a collation replacement determination process. It is a flowchart explaining a similarity calculation process. It is a figure explaining the example of a display of an alerting | reporting image. And FIG. 11 is a diagram illustrating a configuration example of a personal computer.

Explanation of symbols

1, 1-1 to 1-n amusement store 2 face image management center 3 face image management database 4 third party amusement store management center 5 third party amusement store management database 6 face image management bus 7 third party amusement store management bus 8, 9 Public communication line network 21 Facial image information recognition device 22 Facial image information database 24 Amusement store management device 26 Amusement table management database 27 Media rental management device 29 Media rental management database 30 Amusement store management information bus 31 Facial image information bus 33 Settlement machine 34 Lending machine 35 Counter 36, 36-1 to 36-m Game stand 37, 37-1 to 37-m Game stand peripheral terminal 38, 38-1 to 38-m Camera 39, 39-1 to 39- m Image processing unit

Claims (8)

Accumulation means for accumulating a face image as an accumulator face image in an accumulator database;
Acquisition means for continuously acquiring the face image of the person to be verified;
Face image extracting means for extracting the face image acquired by the acquiring means;
Feature amount extraction means for extracting feature amounts from the face image extracted by the face image extraction means;
Dictionary data storage means for storing a predetermined number of feature values of the face image acquired continuously as dictionary data;
Collation data for storing, as the collation data, a predetermined number of feature values of the face image continuously acquired after the feature data of the predetermined number of face images are stored as the dictionary data by the dictionary data storage means. Storage means;
When the feature quantity of the predetermined number of face images in the dictionary data and the feature quantity of the predetermined number of face images in the collation data are not similar, the face image of the person to be collated acquired at that timing, and the storage means A collation means for calculating and collating the similarity with the accumulated face image of the accumulator;
And a similarity determination unit that determines whether or not the face image of the person to be collated is the face image of the accumulator by comparing the similarity that is the collation result of the collation unit with a predetermined threshold value. . A feature amount comparison / determination unit that determines whether or not the feature amount of the predetermined number of face images in the dictionary data is similar to the feature amount of the predetermined number of face images in the matching data;
When it is determined that the feature amount comparison and determination unit does not resemble the collation unit, the similarity between the face image of the person to be collated acquired at that timing and the face image of the accumulator stored in the storage unit The monitoring device according to claim 1, wherein the monitoring device calculates and collates. An average feature amount calculating means for calculating an average feature amount of a predetermined number of face images in the dictionary data as an average feature amount;
The feature amount comparison / determination unit compares the feature amount of each of the predetermined number of face images in the collation data with the average feature amount, and if none of them is similar, the predetermined number of face images in the dictionary data The monitoring device according to claim 2, wherein it is determined that the feature amount of the image and the feature amount of the predetermined number of face images in the matching data are not similar. When the feature quantity of the predetermined number of face images in the dictionary data and the feature quantity of the predetermined number of face images in the collation data are not similar, at that timing, the dictionary data storage means is stored as the dictionary data. The feature quantity of the predetermined number of face images is deleted, and the feature quantity of the predetermined number of face images stored as the collation data is stored as new dictionary data by the collation data storage means. The monitoring device described in 1. The dictionary data storage means, when the feature quantity of the face image acquired continuously and the feature quantity stored as the dictionary data and having the lowest priority are not similar as dictionary data The monitoring apparatus according to claim 1, wherein the stored feature quantity with the lowest priority is updated with the feature quantity of the acquired face image. The collation data storage means, as the collation data, when the feature quantity of the face image acquired continuously and the feature quantity stored as the collation data and having the lowest priority are not similar The monitoring apparatus according to claim 1, wherein the stored feature quantity with the lowest priority is updated with the feature quantity of the acquired face image. Accumulation means for accumulating a face image as an accumulator face image in an accumulator database;
Acquisition means for continuously acquiring the face image of the person to be verified;
Face image extracting means for extracting the face image acquired by the acquiring means;
Feature amount extraction means for extracting feature amounts from the face image extracted by the face image extraction means;
Dictionary data storage means for storing a predetermined number of feature values of the face image acquired continuously as dictionary data;
Collation data for storing, as the collation data, a predetermined number of feature values of the face image continuously acquired after the feature data of the predetermined number of face images are stored as the dictionary data by the dictionary data storage means. Storage means;
When the feature quantity of the predetermined number of face images in the dictionary data and the feature quantity of the predetermined number of face images in the collation data are not similar, the face image of the person to be collated acquired at that timing, and the storage means A collation means for calculating and collating the similarity with the accumulated face image of the accumulator;
And a similarity determination unit that determines whether or not the face image of the person to be collated is the face image of the accumulator by comparing the similarity that is the collation result of the collation unit with a predetermined threshold value. In the monitoring method of
An acquisition step of continuously acquiring the face image of the person to be verified by the acquisition means;
A face image extraction step of extracting the face image acquired by the process of the acquisition step by the previous face image extraction step;
A feature amount extraction step for extracting a feature amount from the face image extracted by the processing of the face image extraction step by the feature amount extraction means;
A dictionary data storage step of storing a predetermined number of feature quantities of the face image continuously acquired by the dictionary data storage means as dictionary data;
After the feature data of the predetermined number of face images are stored as the dictionary data by the processing of the dictionary data storage step by the collation data storage means, A collation data storage step for storing as many pieces of collation data;
When the feature quantity of the predetermined number of face images in the dictionary data and the feature quantity of the predetermined number of face images in the match data are not similar by the matching means, the face image of the person to be matched acquired at that timing A collation step of calculating and collating the similarity with the face image of the accumulator accumulated in the accumulation means;
Similarity that determines whether or not the face image of the person to be collated is the face image of the accumulator by comparing the similarity that is the collation result in the collation step with a predetermined threshold by the similarity degree determination unit A monitoring method including a determination step. Accumulation means for accumulating a face image as an accumulator face image in an accumulator database;
Acquisition means for continuously acquiring the face image of the person to be verified;
Face image extracting means for extracting the face image acquired by the acquiring means;
Feature amount extraction means for extracting feature amounts from the face image extracted by the face image extraction means;
Dictionary data storage means for storing a predetermined number of feature values of the face image acquired continuously as dictionary data;
Collation data for storing, as the collation data, a predetermined number of feature values of the face image continuously acquired after the feature data of the predetermined number of face images are stored as the dictionary data by the dictionary data storage means. Storage means;
When the feature quantity of the predetermined number of face images in the dictionary data and the feature quantity of the predetermined number of face images in the collation data are not similar, the face image of the person to be collated acquired at that timing, and the storage means A collation means for calculating and collating the similarity with the accumulated face image of the accumulator;
And a similarity determination unit that determines whether or not the face image of the person to be collated is the face image of the accumulator by comparing the similarity that is the collation result of the collation unit with a predetermined threshold value. To control the computer,
An acquisition step of continuously acquiring the face image of the person to be verified by the acquisition means;
A face image extraction step of extracting the face image acquired by the process of the acquisition step by the previous face image extraction step;
A feature amount extraction step for extracting a feature amount from the face image extracted by the processing of the face image extraction step by the feature amount extraction means;
A dictionary data storage step of storing a predetermined number of feature quantities of the face image continuously acquired by the dictionary data storage means as dictionary data;
After the feature data of the predetermined number of face images are stored as the dictionary data by the processing of the dictionary data storage step by the collation data storage means, A collation data storage step for storing as many pieces of collation data;
When the feature quantity of the predetermined number of face images in the dictionary data and the feature quantity of the predetermined number of face images in the match data are not similar by the matching means, the face image of the person to be matched acquired at that timing A collation step of calculating and collating the similarity with the face image of the accumulator accumulated in the accumulation means;
Similarity that determines whether or not the face image of the person to be collated is the face image of the accumulator by comparing the similarity that is the collation result in the collation step with a predetermined threshold by the similarity degree determination unit A program that executes a process including a judgment step.

Images