JP5577729B2 - Monitoring device and method, and program - Google Patents

Description

  The present invention relates to a monitoring apparatus and method, a monitoring system, and a program, and more particularly, to a monitoring apparatus and method, a monitoring system, and a program that can improve the recognition rate based on facial images.

  Even at amusement stores such as pachinko and pachislot stores, suspicious people have been pulled out of gaming media, which is a factor that puts pressure on the management of amusement stores. Such damage is an urgent issue in the amusement store industry.

  In order to overcome this situation, a mechanism has been established in which a suspicious person is discovered and the game is stopped, and suspicious person entrance monitoring using a face image is also performed.

  However, in the face image authentication system, due to an error in authentication, the recognition rate is not close to 100%, and there is a problem in reliability.

  More specifically, as a problem of reliability in the face image authentication system, a so-called rejection error that is rejected even though it is a face image of the same person as the acquired face image, and an acquired face image There is a so-called other person acceptance error that accepts a face image of another person who is not the same person as the same person.

  Of these, other person acceptance errors are detected as unsuspicious persons as suspicious persons, and therefore, other person acceptance errors occur unnecessarily to an amusement store staff. In some cases, visual confirmation by an attendant was required after all.

  Due to the occurrence of such an error that could hinder sales, there is an example where the distrust of the system increases, and as a result, the introduction and use of the face image authentication system has been discontinued. However, the actual situation was that it was not effective.

  Therefore, in order to cope with such a problem, there has been proposed a technique for suppressing a rejection error and a stranger acceptance error by improving the face orientation and the robustness to illumination (see Patent Documents 1 and 2).

JP 2009-157767 A JP 2001-202516 A

  However, in addition to the face orientation and lighting, the cause of the rejection error and other person acceptance error described above may be caused by the fact that the facial feature points are hidden when the hair touches the eyes, or the shadow is caused by the hair hitting the contour of the face. In other words, people who do not have similar characteristics in the actual face image by hiding part of the face image or changing the shadow The face images may be erroneously authenticated as being similar, or conversely, even face images of people who are originally similar may be erroneously authenticated as being dissimilar.

  In addition, when the face image is originally captured from the front or when the person goes to collate, it is possible to wait until the conditions are met. Captures images from the top (table lamp) or the left side of the game table (sand), and only captures face images that are difficult to match because there are few cases of capturing the face from the front and the person is unaware of being authenticated. There is a risk that authentication failure or misauthentication may occur.

  The present invention has been made in view of such a situation, and makes it possible to improve authentication accuracy in authentication processing using a face image.

  The monitoring apparatus according to one aspect of the present invention includes a storage unit that stores a face image as a face image of an accumulator in an accumulator database, an image capturing unit that captures an image, and an image captured by the image capturing unit. A face image extracting means for extracting a person's face image, an organ extracting means for extracting a part of each organ constituting the face from the face image of the person to be collated, and the reliability of the organ extracted by the organ extracting means A reliability calculation unit that calculates the level, a determination unit that determines whether the face image of the person to be collated is appropriate based on the reliability calculated by the reliability calculation unit, and the appropriate determination unit The collation means for calculating and collating the similarity between the face image of the person to be collated determined to be appropriate and the face image of the accumulator accumulated in the accumulation means, and the collation result of the collation means Similarity and predetermined Similarity determination means for determining whether or not the face image of the person to be collated is the face image of the accumulator by comparison with a threshold value, and the face image of the person to be collated is stored by the similarity degree determination means. A notification means for notifying that the person to be collated is the accumulator.

  Based on the reliability calculated by the reliability calculation means, an angle estimation means for estimating the angle of the face image with respect to the imaging device, and based on the angle of the face image with respect to the imaging device estimated by the angle estimation means, It can be determined whether or not the face image of the person to be collated is appropriate.

  And a feature amount extracting unit that extracts a feature amount from the face image of the person to be collated, and the collating unit includes a face image of the person to be collated acquired by the acquiring unit, The similarity is calculated using the feature amount with the registrant's face image stored in the storage means, the face image of the person to be collated acquired by the acquisition means, and the registrant's face stored in the storage means A face image can be collated.

  The organ extracting means can extract left and right eyes and mouths of the parts of each organ constituting the face from the face image.

  The reliability calculation means can calculate the reliability by increasing the weights of the reliability of the left and right eyes and the mouth among the organs extracted by the organ extraction means.

  The monitoring method according to one aspect of the present invention includes: an accumulation unit that accumulates a face image as an accumulator's face image in an accumulator database; an imaging unit that captures an image; and an image captured by the imaging unit. A face image extracting means for extracting a person's face image, an organ extracting means for extracting a part of each organ constituting the face from the face image of the person to be collated, and the reliability of the organ extracted by the organ extracting means A reliability calculation unit that calculates the level, a determination unit that determines whether the face image of the person to be collated is appropriate based on the reliability calculated by the reliability calculation unit, and the appropriate determination unit The collation means for calculating and collating the similarity between the face image of the person to be collated determined to be appropriate and the face image of the accumulator accumulated in the accumulation means, and the collation result of the collation means Similarity and predetermined Similarity determination means for determining whether or not the face image of the person to be collated is the face image of the accumulator by comparison with a threshold value, and the face image of the person to be collated is stored by the similarity degree determination means. And a notifying means for notifying that the person to be collated is the accumulator, when the face image is determined to be a person's face image, wherein the face image is accumulated in the accumulating means From the accumulation step of accumulating in the accumulator database as a person's face image, the imaging step of capturing an image in the imaging unit, and the image captured by the processing of the imaging step in the face image extracting unit A face image extracting step for extracting a person's face image, and an organ extraction for extracting a part of each organ constituting the face from the face image of the person to be collated in the organ extracting means Calculated by the reliability calculation step of calculating the reliability of the organ extracted by the processing of the organ extraction step in the reliability calculation means, and the reliability calculation step of the appropriateness determination means Based on the reliability, an appropriate determination step for determining whether or not the face image of the verification target person is appropriate, and a verification target person determined to be appropriate by the processing of the appropriateness determination step in the acquisition unit A step of acquiring a facial image of the person, and calculating a similarity between the face image of the person to be collated acquired by the process of the obtaining step in the collating unit and the face image of the accumulating person accumulated in the accumulating unit In comparison between the collation step to be collated and the similarity that is the collation result in the collation step processing in the similarity determination means and a predetermined threshold value Thus, the similarity determination step for determining whether or not the face image of the collation target person is the face image of the accumulator, and the face image of the collation target person by the processing of the similarity determination step in the notification means A notification step of notifying that the person to be collated is the accumulator.

  The program according to one aspect of the present invention includes: an accumulation unit that accumulates a face image as an accumulator's face image in an accumulator database; an imaging unit that captures an image; and an image captured by the imaging unit, A face image extracting means for extracting a face image of the subject, an organ extracting means for extracting a part of each organ constituting the face from the face image of the person to be collated, and the reliability of the organ extracted by the organ extracting means. A reliability calculation means for calculating, a correctness determination means for determining whether the face image of the person to be collated is appropriate based on the reliability calculated by the reliability calculation means, and the appropriateness determination means. The similarity between the face image of the person to be collated determined to be appropriate and the face image of the accumulator accumulated in the accumulating means is calculated and collated, and the similarity that is the collation result of the collating means Degree and place A similarity determination unit that determines whether or not the face image of the person to be collated is the face image of the accumulator by comparison with the threshold value, and the face image of the person to be collated is determined by the similarity degree determination unit. When it is determined that the face image is an accumulator, the face image in the accumulator is accumulated in a computer that controls a monitoring device including a notifying unit that notifies that the person to be collated is the accumulator. From the accumulation step of accumulating in the accumulator database as a person's face image, the imaging step of capturing an image in the imaging unit, and the image captured by the processing of the imaging step in the face image extracting unit A face image extracting step for extracting a person's face image, and an organ extraction unit for extracting a part of each organ constituting the face from the face image of the person to be collated in the organ extracting means. Calculated by the reliability calculation step of calculating the reliability of the organ extracted by the processing of the organ extraction step in the reliability calculation means, and the reliability calculation step of the appropriateness determination means Based on the reliability, an appropriate determination step for determining whether or not the face image of the verification target person is appropriate, and a verification target person determined to be appropriate by the processing of the appropriateness determination step in the acquisition unit A step of acquiring a facial image of the person, and calculating a similarity between the face image of the person to be collated acquired by the process of the obtaining step in the collating unit and the face image of the accumulating person accumulated in the accumulating unit And a comparison between the matching step for matching and the similarity as a matching result in the processing of the matching step in the similarity determination means and a predetermined threshold value The similarity determination step for determining whether or not the face image of the person to be collated is the face image of the accumulator, and the face image of the person to be collated by the processing of the similarity degree determination step in the notification means Is determined to be a face image of the accumulator, a process including a notification step of notifying that the collation target person is the accumulator is executed.

  In one aspect of the present invention, a face image is accumulated in the accumulator database as an accumulator face image, an image is captured, and the face image of the person to be collated is extracted from the captured image, and the person to be collated From each face image, the part of each organ constituting the face is extracted, the reliability of the extracted organ is calculated, and based on the calculated reliability, whether or not the face image of the person to be collated is appropriate The similarity between the face image of the person to be verified that has been determined to be appropriate and the face image of the accumulated person that has been accumulated is calculated and collated, and the similarity that is the result of the collation and a predetermined threshold value Is determined whether or not the face image of the person to be collated is the face image of the accumulator, and when it is determined that the face image of the person to be collated is the face image of the accumulator, Notification that the person to be collated is the accumulator

  That is, in the monitoring device according to one aspect of the present invention, the accumulation unit that accumulates a face image as an accumulator's face image in the accumulator database is, for example, a biological information DB, and The face image extraction unit that is a camera and extracts the face image of the person to be collated from the image captured by the image pickup unit is, for example, a face image extraction unit of an image processing unit, and The organ extracting means for extracting the part of each organ constituting the face from the face image is an organ extracting unit, and the reliability calculating means for calculating the reliability of the organ extracted by the organ extracting means is an organ An appropriate determination unit that is a score providing unit and determines whether the face image of the person to be collated is appropriate based on the reliability calculated by the reliability calculation unit is an appropriate determination unit. Previous The collating means for calculating and collating the similarity between the face image of the person to be collated determined to be appropriate by the adequacy determining means and the face image of the accumulator accumulated in the accumulating means is, for example, a collating unit What is similarity determination means for determining whether or not the face image of the person to be collated is the face image of the accumulator by comparing the similarity that is the collation result of the collation means and a predetermined threshold value? A similarity determination unit that, when the similarity determination means determines that the face image of the person to be collated is the face image of the accumulator, notifies that the person to be collated is the accumulator The notification means is, for example, a communication unit.

  That is, the organ score assigning unit assigns a score to each organ extracted from the face image of the image captured by the camera. The appropriateness determination unit determines whether the face image of the person to be collated is a face image suitable for the collation processing in the collation unit based on the score given to each organ. Then, the collation unit collates the face image determined by the appropriateness determination unit as the face image of the person to be collated suitable for collation with the face image stored in the biometric information DB.

  In the organ to be extracted, the left and right eyes and the mouth are particularly important, and only these may be extracted, and when extracting other organs, the left and right eyes, The weight of the mouth reliability may be increased.

  As a result, only the face image suitable for the matching process is used for the matching process, so that the recognition rate is improved and the face image not suitable for the matching is not used for the matching process, thereby reducing the load required for the matching process. And the processing speed can be improved.

  According to the present invention, it is possible to improve the recognition rate in authentication processing using a face image.

It is a figure which shows the structure of one Embodiment of the monitoring system to which this invention is applied. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the entrance camera and shop camera of FIG. It is a figure explaining the structural example of the image processing unit of FIG. It is a figure explaining the structural example of the biometric information recognition apparatus of FIG. It is a figure explaining the structural example of the biometric information management center and biometric information management DB of FIG. It is a figure explaining the structural example of the portable terminal of FIG. It is a flowchart explaining a registered player visit monitoring process. It is a figure explaining organ extraction. It is a flowchart explaining a similarity calculation process. It is a figure explaining the example of a display of an alerting | reporting image. And FIG. 11 is a diagram illustrating a configuration example of a personal computer.

  Embodiments of the present invention will be described below. Correspondences between the configuration requirements of the present invention and the embodiments described in the detailed description of the present invention are exemplified as follows. This description is to confirm that the embodiments supporting the present invention are described in the detailed description of the invention. Accordingly, although there are embodiments that are described in the detailed description of the invention but are not described here as embodiments corresponding to the constituent elements of the present invention, It does not mean that the embodiment does not correspond to the configuration requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

  That is, the monitoring device according to one aspect of the present invention includes an accumulation unit (for example, the biometric information DB 22 in FIG. 10) that accumulates a face image as an accumulator's face image in an accumulator database, and an imaging unit that captures an image (for example, 9, a face image extraction unit (for example, the face image extraction unit 202 in FIG. 9) that extracts the face image of the person to be collated from the images captured by the imaging unit, From the face image of the person to be collated, organ extraction means (for example, the organ extraction unit 203 in FIG. 9) that extracts the parts of each organ constituting the face, and the reliability of the organ extracted by the organ extraction means is calculated. Whether or not the face image of the person to be collated is appropriate based on the reliability calculated by the reliability calculating unit (for example, the organ score assigning unit 204 in FIG. 9) and the reliability calculated by the reliability calculating unit. Appropriate judgment The similarity between the means (for example, the appropriateness determination unit 206 in FIG. 9), the face image of the person to be collated determined to be appropriate by the appropriateness determination means, and the face image of the accumulator accumulated in the accumulation means The face image of the person to be collated is stored in the accumulator by comparing the collation unit (for example, the collation unit 222 in FIG. 10) that compares Similarity determination means (for example, similarity determination unit 233 in FIG. 10) for determining whether or not the face image of the person to be collated is the face image of the accumulator by the similarity determination means. In the case where it is determined that there is, it includes notification means (for example, the communication unit 224 in FIG. 10) for notifying that the person to be collated is the accumulator.

  Based on the reliability calculated by the reliability calculation unit, an angle estimation unit (for example, the angle estimation unit 205 in FIG. 9) that estimates an angle of the face image with respect to the imaging device, and the angle estimation unit estimates the angle. Based on the angle of the face image with respect to the imaging device, it can be determined whether or not the face image of the person to be collated is appropriate.

  A feature amount extraction unit (for example, a feature amount extraction unit 231 in FIG. 10) that extracts a feature amount from the face image of the person to be verified can be further included, and the acquisition unit includes the acquisition unit. The similarity is calculated using the feature amount between the face image of the person to be collated acquired by the above and the face image of the registrant accumulated in the accumulating unit, and the face image of the person to be collated acquired by the acquiring unit And the face image of the registrant stored in the storage means can be collated.

  The monitoring method according to one aspect of the present invention includes an accumulation step of accumulating the face image as an accumulator face image in an accumulator database in the accumulation unit, and an imaging step of capturing an image in the imaging unit (for example, FIG. 13 step S1), and a face image extraction step (for example, step S2 in FIG. 13) for extracting the face image of the person to be collated from the image captured by the image capturing step in the face image extracting means. An organ extracting step (for example, step S4 in FIG. 13) for extracting a part of each organ constituting the face from the face image of the person to be collated in the organ extracting means, and the organ in the reliability calculating means. A reliability calculation step (for example, step S5 in FIG. 13) for calculating the reliability of the organ extracted by the processing of the extraction step; An appropriate determination step for determining whether or not the face image of the person to be collated is appropriate based on the reliability calculated by the process of the reliability calculation step (for example, step S7 in FIG. 13). And an acquisition step (for example, step S21 in FIG. 13) of acquiring the face image of the person to be verified that is determined to be appropriate by the process of the appropriateness determination step in the acquisition unit, and the acquisition in the verification unit A collation step (for example, step S24 in FIG. 13) for calculating and collating the similarity between the face image of the person to be collated acquired by the processing of the step and the face image of the accumulator accumulated in the accumulating unit; The face image of the person to be collated is obtained by comparing the similarity, which is the collation result in the collation process, with a predetermined threshold value in the similarity determination unit. The similarity determination step (for example, step S25 in FIG. 13) for determining whether or not the face image of the accumulator is obtained, and the face image of the person to be collated is processed by the similarity determination step in the notification means. And a notification step (for example, step S28 in FIG. 13) for notifying that the person to be collated is the accumulator when it is determined that the face image is the accumulator.

  FIG. 1 is a diagram showing a configuration of an embodiment of a game shop monitoring system according to the present invention.

  The game stores 1-1 to 1-n are so-called pachinko stores, pachislot stores, or casino stores. These amusement stores 1-1 to 1-n are affiliated stores or member stores of the biometric information management center or the third party amusement store management center, and stores that need to manage a plurality of stores in an integrated manner. It is. The game stores 1-1 to 1-n are connected by a biological information management bus 6 and a third-party game store management bus 7, and the public communication line networks 8 and 9 represented by the bus and the Internet are used. Via each other, biological information and third-party amusement store management information are exchanged. In the following description, when there is no particular need to distinguish each of the game stores 1-1 to 1-n, the game store 1 is simply referred to, and the other configurations are also referred to in the same manner.

  The biometric information management bus 6 mainly functions as a transmission path for distributing biometric information managed by the biometric information recognition device 21 of each game store 1. The third-party game store management bus 7 functions as a transmission path for distributing medium lending management information managed mainly by the medium lending management device 27 of each amusement store 1.

  The biometric information management center 2 is a server used by a business operator who manages and operates the biometric information management center, and is a registered player DB 252 (hereinafter referred to as DB) 3 managed by a biometric information management database (hereinafter also referred to as DB). 11) is updated based on the unregistered player DB 251 (FIG. 11) generated by each gaming store 1, and the updated latest registered player DB 252 is updated to the biometric information recognition device 21 of each gaming store 1. And deliver.

  The third-party amusement store management center 4 is a server used by a business operator that manages and operates the third-party amusement store management center, and the medium lending management managed by the third-party amusement store management database (DB) 5 In addition to updating the information DB based on the information supplied from each game store 1, the updated latest medium lending management information is distributed to the medium lending management device 27 of each game store 1.

  The biometric information recognition apparatus 21 includes image processing units 39-1 to 39 based on images captured by the cameras 38-1 to 38-m, the entrance cameras 40-1 to 40-p, and the in-store cameras 41-1 to 41-q. Based on the face image information extracted by − (m + p + q) and supplied via the biometric information bus 31, the face image registered in advance in the biometric information database 22 is collated. The store visitor's visit is notified to the portable terminal 20 or displayed on the display unit 23 such as a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display). Further, when the face image registered in the biometric information database 22 is collated and does not match, the biometric information recognition device 21 accesses the biometric information management database 3 and registers it in the unregistered player DB 251 as an unregistered person. .

  The game store management device 24 is a so-called hall computer, and monitors the operations of the game machines 36-1 to 36-m via the game store management information bus 30. The game store management device 24 determines a predetermined amount according to the monitoring status such as information on the payout of the game table 36 or the payout of medals, the call information of the players on the game tables 36-1 to 36-m, or the occurrence of an error. The execution result is executed, and the execution result is displayed on the display unit 25 such as a CRT or LCD. The game store management device 24 identifies information supplied from the counter 35, the game machines 36-1 to 36-m, and the game machine peripheral terminals 37-1 to 37-m, respectively. For example, it is managed by the gaming machine management database 26 in association with the gaming machine number.

  The medium lending management device 27 manages the medium lending management information of the rented game media using the medium lending management database 29 based on the information from the payment machine 33 and the lending machine 34, and also the medium lending management database. When the medium lending management information registered in 29 is updated, the updated information is sent to the third party amusement store management center 4 via the third party amusement store management bus 7 and the public communication line network 9. Further, the medium lending management device 27 acquires the medium lending management information supplied by the third party amusement store management center 4 via the third party amusement store management bus 7 and the public communication line network 9, and manages the medium lending management. Store in the database 29.

  The lending machine 34 lends a number of game media according to the amount when the player accepts a predetermined amount by cash or a prepaid card when playing on the gaming table 36. At this time, the lending machine 34 supplies information on the number of lent gaming media to the medium lending management device 27 together with information such as the accepted cash and the remaining number of prepaid cards. Thereby, the medium lending management device 27 registers information on the number of lent gaming media in the medium lending management information database 29 together with information such as the received cash and the remaining number of prepaid cards.

  The checkout machine 33 sells prepaid cards with a frequency for borrowing a ball. At this time, the checkout machine 33 supplies the frequency of the prepaid card sold and the amount paid to the medium lending management device 27. Further, the settlement vending machine 33 settles and pays out cash based on the remaining number of game media lent as a frequency such as a prepaid card. At this time, the checkout machine 33 supplies the remaining number of prepaid cards and the amount of cash refunded to the medium lending management device 27.

  The counter 35 counts the number of game media acquired by the player playing on the game table 36, and outputs the counting result as a magnetic card or a receipt.

  The game machines 36-1 to 36-m execute a game when a predetermined operation is performed by the player, and pay out a game ball or a medal according to a so-called small hit or big hit.

  The game console peripheral terminals 37-1 to 37-m are so-called pedestrian machines provided corresponding to the respective game machines 36-1 to 36-m. The same as the lending machine 34) is provided. Further, the gaming machine peripheral terminal 37 acquires biological information such as a face image of the player who plays the gaming machine 36 and transmits it to the biological information recognition device 21 together with the gaming machine identification information (game machine number). FIG. 1 shows an example in which cameras 38-1 to 38-m that acquire a player's face image are provided as a function of acquiring biometric information.

  For example, as shown in FIG. 2, the cameras 38-1 to 38-m are arranged below the table display lamps 61-1 to 61-4 provided on the respective upper sides of the game tables 36-1 to 36-4. As shown in FIG. 3, a face image may be taken so that the player can take an image within the reading range δ. Can be used.

  Further, as shown in FIG. 4, the cameras 38-1 to 38-m are provided with convex portions 71-1 to 71-4 on the gaming machine peripheral terminals 37-1 to 37-4, as shown in FIG. As described above, it may be provided so that a player's face image can be captured within the reading range θ.

  Further, for example, as shown in FIG. 6, the cameras 38-1 to 38-m may be provided at the center of the game table 36 (on the board surface of the game table 36) to take an image. That is, by installing the camera 38 in the installation unit 81 of FIG. 6, the player is imaged within the reading range φ as shown in FIG.

  The entrance cameras 40-1 to 40-p and the in-store cameras 41-1 to 41-q are installed at entrances and predetermined places in the store of the amusement store 1, and imaged units 39- (m + 1) to 39- (m + p + q).

  The entrance cameras 40-1 to 40-p and the in-store cameras 41-1 to 41-q are set as shown in FIG. 8, for example. FIG. 8 shows an installation example of the entrance cameras 40-1 to 40-p and the in-store cameras 41-1 to 41-q in the game store 1.

  That is, in FIG. 8, entrances 112-1 to 112-3 are provided, and the entrance cameras 40-1 to 40-3 image players who enter the store from each entrance and exit 112. Further, the in-store cameras 41-1 to 41-10 are set at positions at which both surfaces of the island facilities 111-1 to 111-5 can be imaged in a row. The island facilities 111 are provided with game machines 36 on both sides, that is, so as to sandwich the island facilities 111 in the vertical direction in the figure. Since the camera 38, the entrance camera 40, and the in-store camera 41 all have a pan / tilt / zoom function, the in-store cameras 41-1 to 41-10 are arranged as shown in FIG. All players who play at 36 can take an image with any of the in-store cameras 41-1 to 41-10.

  Further, the in-store camera 40-a is provided in front of the lending machine 34, the in-store camera 40-b is provided in front of the checkout machine 33, and the in-store camera 41-c is the counter of the counter 35. It is provided in front, and a player who uses the rental machine 34, the checkout machine 33, and the counter 35 can be imaged.

  That is, as shown in FIG. 8, in the game store 1, a player who visits the store, a player who plays on the game stand 36, and a player who uses the lending machine 34, the checkout machine 33, and the counter 35. The camera 38, the entrance camera 40, and the in-store camera 41 are installed so that almost all of the actions that the player is supposed to take in the gaming store 1 can be monitored.

  Next, a configuration example of the image processing unit 39 will be described with reference to FIG.

  The image acquisition unit 201 acquires an image captured by the camera 38 (or the entrance camera 40 or the in-store camera 41) and supplies it to the face image extraction unit 202. The face image extraction unit 202 extracts a rectangular image composed of the face image from the image supplied from the image acquisition unit 201 using a pattern such as the arrangement of the parts constituting the face, and sends it to the organ extraction unit 203 and the transmission unit 207. Supply.

  The organ extraction unit 203 extracts organs constituting the face from the rectangular face image, that is, organs such as left and right eyes, nose, mouth, and ears, and supplies them to the organ score assigning unit 204. More specifically, the organ extraction unit 203 extracts organ feature points set in parts constituting the outline of these organs from the face image, and classifies the extracted organ feature point information for each organ. , And supplied to the organ score assigning unit 204 as organ information. As for the organ feature points, positions and points to be obtained for each organ are set in advance. In other words, organ feature points are arranged along the contours that make up the organ. For example, in the case of the eye, the central feature of the eye is configured such as the center position of the upper eyelid, the center position of the lower eyelid, and the eyelid. A predetermined number of points are specified at a predetermined position. Then, the organ extracting unit 203 extracts each organ by extracting information on the position and the number of organ feature points that can be read for each organ from the image among the organ feature points set in this way. To do.

  The organ score assigning unit 204 assigns a score to each organ based on the organ feature point information, which is information on the organ supplied from the organ extracting unit 203, and supplies the information to the angle estimating unit 205. . That is, as described above, the organ information extracted by the organ extraction unit 203 is information on the organ feature points that can be specified on the image among the set organ feature points and the positions thereof. For this reason, the organ score assigning unit 204 is based on the ratio of the number of organ feature points that can be specified from the face image to the total number of organ feature points set for each organ. Calculate and assign a score. Of the organs that make up the face, the eyes and mouth are a major factor in determining whether they are appropriate for recognizing the face image. The weight may be set so that the score is higher than that of the organ. In order to reduce the calculation speed and load, only the eyes and mouth may be handled as organs.

  Based on the score set for each organ, the angle estimation unit 205 estimates the angle of the imaging position with respect to the face-to-face direction as the extracted face image as the imaging angle, and information on the estimated imaging angle and the organ Information on the score set for each time is supplied to the appropriateness determination unit 206.

  The appropriateness determination unit 206 uses the extracted face image for the similarity calculation processing of the subsequent face image based on the score and imaging angle information set for each organ supplied from the angle estimation unit 205. It is determined whether or not it is appropriate, and when it is appropriate, the transmission unit 207 is controlled to supply the face image to the biological information recognition apparatus 21 at the subsequent stage. More specifically, the appropriateness determination unit 206 determines whether the estimated imaging angle is within a predetermined range close to the facing direction and the total score set for each organ is higher than a predetermined value. Whether or not the face image is an appropriate face image to be used for the similarity calculation processing is determined based on whether or not the face image is used. The transmission unit 207 transmits the face image to the biological information recognition device 21 based on the determination result of the appropriateness determination unit 206.

  Next, a configuration example of the biological information recognition device 21 will be described with reference to FIG.

  The face image acquisition unit 221 acquires the face image supplied from the image processing unit 39 and supplies it to the collation unit 222. The collation unit 222 collates the face image acquired by the face image acquisition unit 221 with the registered player's face image registered in advance in the biometric information DB 22, and if there is a face image that is a candidate with a high degree of similarity. The face images up to the third candidate are displayed on the display unit 23 as the matching result. Further, the collation unit 222 supplies the supplied face image to the unregistered player database registration unit 223 when there is no face image that is a candidate with high similarity.

  More specifically, the feature amount extraction unit 231 of the matching unit 222 extracts a feature amount for identifying a face image, and supplies the feature amount together with the face image to the similarity calculation unit 232. The similarity calculation unit 232 extracts the feature amount of the registered player's face image registered in the biometric information DB 22 and uses the feature amount supplied from the feature amount extraction unit 231 to use the biometric information DB 22. The degree of similarity with the face images of all registered players registered in the face image is obtained, and the face images supplied from the face image acquisition unit 221 and the face images with the third highest degree of similarity are determined by the similarity degree determination unit. 233. More specifically, the similarity calculation unit 232, for example, based on various facial feature amounts such as the ratio between the distance between the eyes, the distance from the chin to the forehead, and the length from the chin to the nose, Each difference sum, average ratio, or ratio sum is obtained as the similarity.

  The similarity determination unit 233 sequentially accumulates the similarities supplied from the similarity calculation unit 232 in the buffer 233a, and with respect to each of the similarities of the top three face images, When the similarity is compared with a predetermined threshold, and the registered face image that is ranked first is similar to the face image supplied from the face image acquisition unit 221 based on the comparison result (similarity When the similarity is higher than the predetermined threshold in the case of similarity indicating that the similarity is higher, or when the similarity is lower than the predetermined threshold in the case of similarity indicating that the similarity is lower as the similarity is lower. Information on the similarity between the face image and the top three is supplied to the display unit 23, and is supplied to the communication unit 224 when displayed. In addition, the similarity determination unit 233 compares the similarity between the face image ranked first and a predetermined threshold value, and the registered face image ranked first is compared with the face image acquisition unit 221 based on the comparison result. When the face image supplied is not similar, the face image supplied from the face image acquisition unit 221 is supplied to the unregistered player database registration unit 223.

  The unregistered player database registration unit 223 registers the face image, which has been supplied as being unregistered by the verification unit 222, in the unregistered player DB 251 of the biometric information management database 3.

  The operation unit 225 includes buttons, a mouse, a keyboard, and the like. The operation unit 225 is operated when any one of the face images displayed on the display unit 23 with the above-mentioned similarity ranked in the top three is selected. Is supplied to the communication unit 224. The communication unit 224 includes a modem or the like, and distributes the selected face image to the mobile terminal 20 based on an operation signal from the operation unit 225.

  Here, for example, the similarity indicates a higher value as it is closer to the face image registered as a registered player as indicated by the ratio sum, and the similarity is a value higher than a predetermined threshold value. In this case, an example will be described in which it is determined that the face image of the registered player corresponding to the similarity is. However, for example, when the similarity is expressed as a difference sum in each feature amount between the captured face image and the face image registered as the registered player, the similarity determination unit 233 determines that the similarity is a threshold value. If it is smaller than that, the captured face image is regarded as a registered player's face image, or in the case of an average ratio or the like, it is greater than or equal to a predetermined value in the range of 0 to 1 and close to 1. If it is a value, it can be considered that they are the same person.

  When a new registered player database 252 (FIG. 11) is distributed from the biometric information management center 2, the database management unit 226 updates the biometric information DB 22 based on the new registered player database 252.

  Next, a configuration example of the biological information management center 2 will be described with reference to FIG.

  The biometric information management center 2 includes a DB distribution unit 241, a DB update unit 242, and a DB update determination unit 243. The registered player DB 252 stored in the biometric information management DB 3 is transferred to the unregistered player DB 251. Update based on. More specifically, the DB update determination unit 243 has a built-in RTC (Real Time Clock) 243a that generates time information, and when a predetermined period has passed based on the built-in RTC 243a, the biological information management DB3 is accessed and it is determined whether or not a new unregistered player is registered in the unregistered player DB 251.

  When an unregistered player is newly registered in the unregistered player DB 251, the DB update determination unit 243 controls the feature amount extraction unit 243 b to register the unregistered player registered in the unregistered player DB 251. The feature amount is extracted from the face image of the player, and the similarity calculation unit 243c is controlled to select one of the face images of the unregistered player registered in the unregistered player DB 251 and other unregistered players. The similarity with the player's face image is calculated. Next, the DB update determination unit 243 controls the similarity determination unit 243d, and based on the comparison between the similarity calculated by the similarity calculation unit 243c and a predetermined threshold, the face of another unregistered player It is determined whether each of the images is similar to the face image of the selected one unregistered player. Then, the DB update determination unit 243 controls the number determination unit 243e so that the number of other unregistered player face images that are similar to the selected one unregistered player face image is equal to or greater than a predetermined number. By determining whether or not the number of times that the degree of similarity has exceeded a predetermined threshold is equal to or greater than a predetermined number of times, if it is equal to or greater than the predetermined number of times, the DB update unit 242 is notified accordingly. The DB update unit 242 accesses the unregistered player DB 251, reads out unregistered players who have been registered a predetermined number of times or more, and registers them in the registered player DB 252. Update. Further, when the registered player DB 252 is updated, the DB update unit 242 notifies the DB distribution unit 241 that it has been updated.

  When notified from the DB update unit 242 that the registered player DB 252 has been updated, the DB distribution unit 241 accesses the biometric information management DB 3 and reads the registered player DB 252 to recognize the biometric information of each gaming store 1. Delivered to the device 21.

  Next, a configuration example of the mobile terminal 20 will be described with reference to FIG.

  The communication unit 271 is configured by a modem or the like, and exchanges data with the biometric information recognition device 21 via a wireless communication network in the game store 1. In addition, the communication unit 271 acquires information indicating that a player with a facial image similar to the facial image supplied from the image processing unit 39, which is distributed from the biometric information recognition device 21, has visited the store. 272.

  The image processing unit 272 is a display constituted by an LCD or the like based on information indicating that a player with a face image similar to the face image supplied from the image processing unit 39 supplied from the communication unit 271 has visited the store. In addition to generating an image to be displayed on the unit 273, the image is displayed on the display unit 273.

  Next, the registered player visit monitoring process will be described with reference to the flowchart of FIG.

  In step S <b> 1, the entrance camera 40 captures an image of the installed range and supplies the captured image to the image processing unit 39. The image acquisition unit 201 of the image processing unit 39 acquires the supplied image and supplies it to the face image extraction unit 202.

  In step S <b> 2, the face image extraction unit 202 extracts a player's face image from the supplied image and supplies it to the organ extraction unit 203 and the transmission unit 207. More specifically, the face image extraction unit 202 extracts a face image from, for example, an arrangement of a characteristic part such as an eye or a nose, where the skin is exposed from the color of the captured image. To the organ extracting unit 203 and the transmitting unit 207.

  In step S3, the organ extraction unit 203 determines whether or not a face image has been extracted from the face image extraction unit 202. If no face image is supplied, the process returns to step S1. That is, the processes in steps S1 to S3 are repeated until a face image is extracted. If a face image is extracted in step S3, the process proceeds to step S4.

  In step S <b> 4, the organ extracting unit 203 extracts an organ from the supplied face image, and supplies the extracted organ information to the organ score assigning unit 204. More specifically, the organ extraction unit 203 detects organ feature points as indicated by the black dots of the three types of face images in FIG. 14, for example. That is, in FIG. 14, examples of a face image facing from the left to the left in the figure, a face image facing down in the figure, and a face image facing from the left in the figure and the right eye in the figure being hidden by the hair. It is shown. As shown in the center of FIG. 14, for example, there are 11 organ feature points on the left and right. In other words, if the face is facing the front, eleven organ feature points are detected. However, as shown in the left part of FIG. 14, in the case of a face image facing in the left direction in the figure, the left eye constitutes the outline of the organ of the eye because the face is directed in the left direction in the figure. Some of the organ feature points cannot be detected, and only 6 organ feature points are extracted. In this case, the organ extraction unit 203 extracts, as information on the left eye organ, that the extracted organ feature points are six points in the right part of the left eye in the figure. Therefore, in the case of the left part of FIG. 14, the right eye has 11 points and the mouth has 10 points.

  Similarly, in the central part of FIG. 14, the left and right eyes have 11 points and the mouth has 4 points. Further, in the right part of FIG. 14, the left eye has 6 points, the right eye has 7 points, and the mouth has 10 points.

  In step S5, when the organ score assigning unit 204 acquires the extracted information of the organ feature point and the score of each organ supplied from the organ extracting unit 203 as the extracted organ information, the organ feature of each organ is acquired. Based on the number of extracted points, a score is assigned and supplied to the angle estimation unit 205 together with the acquired information.

  That is, for example, in the case of the left face image of FIG. 14, the left eye has 6 extracted organ feature points. For example, when the perfect score for each organ is 100 points, the score Is set to 55 (≈100 × 6/11). Similarly, if the score of the right eye is set to 100 points, and the mouth has a total of 12 organ feature points, the score is set to 83 (≈100 × 10/12).

  In the case of the face image in the center of FIG. 14, the scores of the left eye, the right eye, and the mouth are set to 100 points, 100 points, and 33 points, respectively. Similarly, in the case of the right face image in FIG. 14, the scores of the left eye, the right eye, and the mouth are set to 55 points, 64 points, and 83 points, respectively.

  That is, since the organ feature points set in advance form the contour of each organ, it is considered that the contour of the organ is accurately reproduced if all the organ feature points are extracted. Therefore, since the score given by the organ score assigning unit 204 is the ratio of the number of extracted points to the preset number of organ feature points, it can be said that the score is the reliability of each organ in the face image. Therefore, it can be considered that the higher the score is, the easier it is for each organ to be recognized correctly, so that it can be considered suitable for the similarity calculation process.

  In step S <b> 6, the angle estimation unit 205 estimates the imaging angle from the face-to-face position of the face image, and supplies the estimated imaging angle and information on the score of each organ to the appropriateness determination unit 206. As used herein, the term “facing” refers to the position facing the front of the face image that is the subject. That is, since the angle estimation unit 205 stores in advance the number of extracted organ feature points of each organ and its distribution for each imaging angle, the imaging angle is calculated from the number of extracted organ feature points and their distribution. Is estimated. For example, in the case of the left face image of FIG. 14, among the organ feature points constituting the left eye contour, the left five points cannot be extracted, and the right eye is all extracted, For the mouth, the two points on the left cannot be extracted. The angle estimation unit 205 estimates the imaging angle from the number of extracted organ feature points and the distribution thereof.

  In step S7, the appropriateness determination unit 206 determines whether or not the extracted face image is a face image suitable for similarity calculation processing described later based on the supplied imaging angle and score information of each organ. Determine. More specifically, the appropriateness determination unit 206 determines that the imaging angle of the supplied face image is within a predetermined angle in the vicinity of the face-to-face direction of the human face of the face image with respect to the horizontal direction and the vertical direction. And, when the total score of the scores for each organ is higher than a predetermined threshold value, it is regarded as appropriate.

  Here, whether or not the face image is suitable for similarity calculation processing is determined based on the following conditions. That is, even if the score for each organ is high, if the imaging angle of the face image is not within a predetermined angle that is close to a certain degree, it is considered that there is a large distortion in the image of the shape of each organ. The first condition is that the imaging angle is within a predetermined angle near the facing direction. However, as described above, it can be considered that the number of extracted organ feature points tends to decrease as the angle deviates from the angle near the facing direction, so the score may be low if the angle is not within the angle near the facing direction. It is considered that the nature is high.

  Furthermore, the second condition is that the imaging angle is within a predetermined angle near the facing direction, and the total score for each organ is higher than a predetermined threshold. That is, when the outline of an organ is hidden by hair or the like, the reliability of information on each organ is low even if the imaging angle is within a predetermined angle near the facing direction.

  Therefore, the appropriateness determination unit 206 determines that the face image satisfying these two conditions is an appropriate face image for the similarity calculation process.

  For example, in the case of the left face image of FIG. 14, the score for each organ is 238 (= 55 + 100 + 83), which is the sum of the left and right eye and mouth scores. In the case of the central portion of FIG. 14, 233 (= 100 + 100 + 33) is obtained. Furthermore, in the case of the right part of FIG. 14, 152 (= 55 + 64 + 33). Therefore, when the imaging angle is within a predetermined angle in the vicinity of the facing direction, when the threshold value of the score is 200, the left and center face images in FIG. 14 are appropriate face images. Although it is considered, in the right part of FIG. 14, it is not regarded as a proper face image. As a matter of course, the threshold value setting is not limited to the above score, and other threshold values may be used.

  When it is determined in step S7 that the extracted face image is appropriate for the similarity calculation process, in step S8, the appropriateness determination unit 206 controls the transmission unit 207 to be supplied from the face image extraction unit 202. The face image thus transmitted is transmitted to the biometric information recognition device 21. At this time, the transmitting unit 207 adds information such as a camera ID for identifying each of the camera 38, the entrance camera 40, or the in-store camera 41 and information on transmission time to the face image and transmits it to the biological information recognition apparatus 21. .

  On the other hand, when it is determined in step S7 that the extracted face image is not appropriate for the similarity calculation process, the process of step S8 is skipped, and the process returns to step S1.

  That is, in the image processing unit 39, among the extracted face images, only the face images whose imaging angle is within a predetermined angle range in the vicinity of the facing direction and whose total score is larger than a predetermined threshold value, The biometric information recognition device 21 is supplied. As a result, it is possible to improve accuracy in similarity calculation processing described later, and it is not necessary to perform similarity calculation processing using a face image with low accuracy, thereby reducing processing load and speeding up processing. It becomes possible to do.

  In the above description, the example using only the eyes and the mouth has been described as the organ extraction. However, as a matter of course, information on other organs, that is, the nose and the ear may be used. is there. However, in similarity calculation processing using facial images, the reliability of the eyes and mouth has a greater effect on the accuracy of the similarity calculation processing than other organs. When used, the weights for the eye and mouth scores may be increased. Further, since the influence on the accuracy of the similarity calculation process is large, conversely, only the eyes and the mouth may be used in order to reduce the calculation load.

  In step S21, the face image acquisition unit 221 of the biological information recognition apparatus 21 acquires a face image. In step S <b> 22, the face image acquisition unit 221 extracts any unprocessed one of the supplied face images and supplies the extracted unprocessed one to the feature amount extraction unit 231.

  In step S23, the feature amount extraction unit 231 of the collation unit 222 extracts the feature amount from the supplied face image, and supplies the feature amount together with the face image to the similarity calculation unit 232.

  In step S24, the similarity calculation unit 232 executes similarity calculation processing.

  Here, the similarity calculation processing will be described with reference to the flowchart of FIG.

  In step S51, the similarity calculation unit 232 extracts a face image of one unprocessed registered player from the face images registered in the registered player DB 252 of the biometric information DB 22, and sets it as a processing target.

  In step S52, the similarity calculation unit 232 extracts a feature amount similar to the feature amount supplied from the feature amount extraction unit 231 from the face image registered in the registered player DB 252 set as the processing target. .

  In step S53, the similarity calculation unit 232 determines the ratio of the eye-to-eye spacing, the length from the chin to the forehead, and the length from the chin to the nose for the face image supplied from the feature amount extraction unit 231. Using the various feature values of the face and the similar feature values in the face image registered in the registered player DB 252 registered in the biometric information DB 22, the difference sum, the average ratio, the ratio sum, etc. Is calculated as the similarity, and in step S54, the similarity with the face image registered in the registered player DB 252 as the calculation result is supplied to the similarity determination unit 233 and registered in the buffer 233a.

  In step S55, the similarity calculation unit 232 determines whether or not an unprocessed registered player face image exists in the registered player DB 252 of the biometric information DB 22, and an unprocessed registered player face image exists. If so, the process returns to step S51. That is, the processes in steps S51 to S55 are repeated until the similarity with the face images of all registered players in the registered player DB 252 is calculated. If it is determined in step S55 that there is no unprocessed registered player face image in the registered player DB 252, the similarity calculation process ends.

  Now, the description returns to the flowchart of FIG.

  In step S25, the similarity determination unit 233 obtains a rank based on the calculation result of the similarity registered in the buffer 233a, and the top three faces among the face images registered in the registered player DB 252. Extract information about image and similarity. Then, the similarity determination unit 233 determines whether or not the highest similarity is greater than a predetermined threshold based on the top three face images and similarity information. That is, the similarity determination unit 233 is a registered player who is most similar (a registered game that is most similar to the facial image acquired by the facial image acquisition unit 221 among facial images registered in the biometric information DB 22). In this case, the similarity of the registered player having the highest similarity is compared with a predetermined threshold.

  Note that, as described above, the similarity between the face image of the registered player who is most similar to the captured face image is not necessarily the highest because of the definition of the similarity. And the threshold value may differ from those in this example.

  If it is determined in step S25 that the highest similarity is greater than a predetermined threshold, in step S26, the similarity determination unit 233 determines that the top third facial image supplied from the similarity calculation unit 232 is the top three. A notification screen 301 indicating that it is a candidate for a registered player's face image is displayed by controlling the display unit 23.

  At this time, for example, a notification screen 301 as shown in FIG. 16 is displayed on the display unit 23.

  In the notification screen 301 of FIG. 16, a camera image display field 311 is provided in the middle left, and the face image supplied from the image processing unit 39 is displayed. Also, on the right side, face image display fields 312-1 to 312-3 of registered players of the top three in the similarity from the first candidate to the third candidate are provided in descending order of similarity. Further, similarity level display fields 313-1 to 313-3 are provided below the face image display fields 312-1 to 312-3 of each registered player, and the level of similarity is displayed. . In FIG. 15, the horizontal length of the area shown in black indicates the magnitude of the similarity.

  Further, ID display fields 314-1 to 314-3 are provided at corresponding positions below the similarity level display fields 313-1 to 313-3, and the face images in the biological information DB 22 of each face image. In FIG. 15, “00051”, “00018”, and “00022” are displayed from the left in FIG.

  Furthermore, below the ID display fields 314-1 to 314-3, confirmation buttons 318-1 to 318-3 operated by the operation unit 225 when each candidate is selected are provided at the corresponding positions. It has been.

  A camera ID display field 315 for identifying the camera that captured the face image is provided under the camera image display field 311. In FIG. 15, the camera 38, the entrance camera 40, and the in-store camera 41 are displayed. “Camera 02” is displayed as a camera ID for identification. Further, a time display field 316 is provided below the camera ID display field 315, and the time taken by the entrance camera 40 is displayed. In FIG. 15, “18:23:32” is displayed. It is shown that the face image in the camera image display field 311 is captured at 18:23:32.

  Furthermore, another person button 317 is provided below the time display field 316, and the face image display fields 312-1 to 312- of the registered player whose face images of the camera image are the first to third candidates. 3 is operated by the operation unit 225 when it is regarded as not similar to any of 3.

  In step S27, the communication unit 224 displays on the display unit 23 whether or not any of the candidate face images has been selected by operating the operation unit 225, for example, the notification screen 301 shown in FIG. If it has been determined, it is determined whether any of the confirmation buttons 318-1 to 318-3 has been operated by the operation unit 225.

  In step S27, for example, when the confirm button 318-1 is operated, it is considered that the face image to be the first candidate has been selected, and in step S28, the communication unit 224 has the selected face image to be the first candidate. And the camera image imaged with the entrance camera 40 is transmitted to the portable terminal 20, and it notifies that the corresponding registered player came to the store.

  In step S41, the communication unit 271 determines whether or not the registered player has been notified of the store visit, and repeats the process until the notification is received. For example, if a registered player's visit is notified in step S41 by the process in step S28, the communication unit 271 notifies the registered player's visit sent from the biometric information recognition device 21 in step S42. , The face image of the registered player and the camera image captured by the entrance camera 40 transmitted together with the notification are supplied to the image processing unit 272. The image processing unit 272 processes information on the selected face image and the camera image captured by the entrance camera 40 into information in a format that can be displayed on the display unit 273, and displays the information on the display unit 273 in step S43. .

  By the above processing, the staff in the game store 1 can recognize the registered player's visit if the portable terminal 20 is possessed.

  In step S29, the face image acquisition unit 221 determines whether or not all of the supplied face images have been processed. If there is an unprocessed face image, the process returns to step S22. That is, the processes in steps S22 to S30 are repeated until the process is performed on all face images. If it is determined that the process has been completed for all face images, the process returns to step S21.

  On the other hand, no candidate face image is selected in step S27. For example, when the other person button 317 on the notification screen 301 in FIG. 15 is pressed, or in step S25, the similarity calculation unit 232 supplies the face image. Based on the information of the top three face images and the similarity, the highest similarity is not greater than a predetermined threshold, that is, even the face image of the most similar registered player If the similarity is less than the predetermined threshold, the similarity determination unit 233 supplies the face image supplied from the image processing unit 39 to the unregistered player database registration unit 223 in step S30. The unregistered player database registration unit 223 accesses the biometric information management database 3 through the biometric management information bus 6 and the public communication line network 8 to register the supplied face image in the unregistered player DB 251.

  If the face image supplied from the image processing unit 39 by the biometric information recognition device 21 is not registered in the biometric information DB 22 by the above processing, the biometric information management managed by the biometric information management center 2 is performed. The face image of the unregistered player is registered in the unregistered player DB 251 in DB3.

  Incidentally, the series of monitoring processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.

  FIG. 17 shows a configuration example of a general-purpose personal computer. This personal computer incorporates a CPU (Central Processing Unit) 1001. An input / output interface 1005 is connected to the CPU 1001 via the bus 1004. A ROM (Read Only Memory) 1002 and a RAM (Random Access Memory) 1003 are connected to the bus 1004.

  The input / output interface 1005 includes an input unit 1006 including an input device such as a keyboard and a mouse for a user to input an operation command, an output unit 1007 for outputting a processing operation screen and an image of a processing result to a display device, a program and various data. A storage unit 1008 including a hard disk drive for storing data, a LAN (Local Area Network) adapter, and the like, and a communication unit 1009 for performing communication processing via a network represented by the Internet are connected. Also, a magnetic disk (including a flexible disk), an optical disk (including a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc)), a magneto-optical disk (including an MD (Mini Disc)), or a semiconductor A drive 1010 for reading / writing data from / to a removable medium 1011 such as a memory is connected.

  The CPU 1001 is read from a program stored in the ROM 1002 or a removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, installed in the storage unit 1008, and loaded from the storage unit 1008 to the RAM 1003. Various processes are executed according to the program. The RAM 1003 also appropriately stores data necessary for the CPU 1001 to execute various processes.

  In this specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series in the order described, but of course, it is not necessarily performed in time series. Or the process performed separately is included.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

1, 1-1 to 1-n amusement store 2 biometric information management center 3 biometric information management database 4 third party amusement store management center 5 third party amusement store management database 6 biometric information management bus 7 third party amusement store management bus 8, 9 Public communication network 21 Biometric information recognition device 22 Biometric information database 24 Game store management device 26 Game table management database 27 Medium lending management device 29 Medium lending management database 30 Amusement store management information bus 31 Biometric information bus 33 Settling machine 34 Lending machine 35 Counter 36,36-1 thru 36-m Game stand 37,37-1 thru 37-m Game stand peripheral terminal 38,38-1 thru 38-m Camera 39,39-1 thru 39- (m + p + q) Image processing unit 40, 40-1 to 40-p Entrance camera 41, 41-1 to 41-q In-store camera

Claims (8)

From image captured by an imaging unit, a face image extracting means for extracting a face image,
From the face image, and organ extracting means for extracting a portion of each organ constituting the face,
Reliability calculation means for calculating the reliability of each organ extracted by the organ extraction means;
An angle estimating means for estimating an angle of the face image with respect to the imaging means based on the reliability of each organ calculated by the reliability calculating means;
When the total reliability of each organ calculated by the reliability calculation means is higher than a predetermined threshold, and the angle of the face image estimated by the angle estimation means is within a predetermined angle , and Value determination means for the face image is determined to be appropriate as a comparison target,
It said face image is determined to be appropriate by the proper determination means calculates a similarity between the stored accumulated's face image storing means for storing in the storage's database the face image as the accumulation's face image , Matching means for matching;
Matching the results similarity and a comparison with a predetermined threshold value, the monitoring device including said face image similarity determination means for determining whether a face image of the storage's said collation means. A feature amount extraction unit that extracts a feature amount from the face image extracted by the face image extraction unit ;
The comparison means includes the face image extracted by the face image extracting means, using the feature amount to calculate the similarity between the stored registrant's face image in the storage means, by the face image extractor The monitoring apparatus according to claim 1, wherein the extracted face image is collated with a registrant's face image stored in the storage unit. 3. The organ extracting unit according to claim 1 or 2 , wherein left and right eyes and a mouth are extracted from a part of each organ constituting the face from the face image extracted by the face image extracting unit . Monitoring device. The reliability calculation unit, said among the organs extracted by the organ extracting unit, right or left eye, and the weight of the reliability of the mouth is increased to claims 1 to calculate the reliability of claim 3 1 a monitoring device as claimed in One. A notification for notifying that the face image extracted by the face image extracting means is the face image of the accumulator when the similarity determination means determines that the face image is the face image of the accumulator The monitoring apparatus according to any one of claims 1 to 4, further comprising means. The program for functioning a computer as a monitoring apparatus as described in any one of Claims 1-5. Facial image extraction means, from the image taken by the imaging means, a face image extraction step of extracting a facial image,
An organ extracting means, from the face image, and the organ extracting a portion of each organ constituting the face,
Is Shin Yoriyukido calculating means, a reliability calculating step of calculating reliability of each organ extracted by the processing of the organ extracting step,
An angle estimating unit that estimates an angle of the face image with respect to the imaging unit based on the reliability of each organ calculated by the process of the reliability calculating step;
Suitable positive determination means, the reliability the calculated by the processing calculation step higher than the total predetermined threshold reliability of each organ, and the angle estimation step of the imaging unit of the estimated the face image by processing If the angle with respect to is within a predetermined angle, and proper determination step of the face image is determined to be appropriate as a comparison target, collation means, said face is determined to be appropriate by the processing of the proper judgment step A collation step of calculating and collating the similarity between the image and the face image of the accumulator accumulated in the accumulator means for accumulating the face image as an accumulator face image in the accumulator database;
S similarity score determination unit, wherein by comparison with the collation result is a similarity with a predetermined threshold value in the process of verification steps, the similarity determination determines whether the facial image is a facial image of the storage's A monitoring method comprising steps and When the notification means determines that the face image is the face image of the accumulator by the processing of the similarity determination step, the face image extracted by the processing of the face image extraction step is the face of the accumulator The monitoring method according to claim 7, further comprising a notification step of notifying that the image is displayed.

Images