JP2007293741A - Monitoring device and method, registrant collation device, attribute estimation device and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce a processing load, and to improve a processing speed. <P>SOLUTION: In this monitor, an image acquisition part 201 acquires an image imaged by an entrance camera 43, and supplies it to a face detection frame setting part 231. The face detection frame setting part 231 adjusts the image so that only an area including the face image of a player remains passing through an entrance registered in entrance passenger position information supplied from a position recognition server 42, and is supplied to a face image extraction part 202. A face detection part 211 of the face image extraction part 202 detects a portion wherein the face image exists from the images supplied from the face detection frame setting part 231, and supplies a detection result thereof to a face image cutout part 212. The face image cutout part 212 extracts the rectangular face image from the portion wherein the face image exists, and supplies it to a transmission part 203. The face image cutout part 212 extracts the rectangular face image from the portion wherein the face image exists, and supplies it to the transmission part 203. The transmission part 203 transmits the cut-out face image to a bioinformation recognition device 21. The monitor can be applied to a monitoring system. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a monitoring device and a monitoring method, a registrant verification device, an attribute estimation device and method, and a program, and more particularly, a monitoring device that can reduce the load of face matching processing and improve the processing speed of face matching processing. And a method, a registrant verification device, an attribute estimation device and method, and a program.

  Currently, in many stores and public facilities, staff members collect information on customers or visitors by visual inspection.

  However, the above-described method is labor intensive and it is difficult to obtain accurate information. Therefore, there has been proposed a method in which a person to be collated is captured by a camera installed near the entrance, a face image is extracted, and the face image of the registrant and the face image of the person to be collated are collated with the face image. (See Patent Document 1). More specifically, a camera installed in the store captures the face image of the person to be collated entering from the entrance or exiting from the exit, and the captured face image and a pre-registered face image are displayed. By comparing and collating, it is determined whether or not the person to be collated is the registrant.

JP 2002-279540 A

  By the way, in the above method, the face of the person to be collated entering from the entrance may be imaged in a state unsuitable for collation, for example, by being oblique or downward with respect to the imaging direction of the camera. It was necessary to improve collation accuracy by acquiring more collation target images. That is, it is necessary to acquire more face images of the person to be collated by setting a wide area as an imaging area.

  However, when the imaging area is widened, a person who does not need to be collated in the imaging area regardless of the presence or absence of the person to be collated, that is, for example, collation of a deaf person (verified collation target person) or a clerk A face image of a person who is not required is also collated. As a result, a load more than necessary may be applied to the face image matching process. In addition, when performing so-called attribute estimation processing that uses face images to count the number of visitors of the verification target person based on the age or sex of the verification target person, the verified verification target person is regarded as a new verification target person. In some cases, the number of visitors could not be counted correctly due to misrecognition.

  The present invention has been made in view of such circumstances, and eliminates unnecessary face image matching processing to reduce the load of face matching processing and improve the processing speed of face matching processing. Is.

  The monitoring device according to the first aspect of the present invention includes an accumulating unit that accumulates a pre-registered registrant's face image, an image obtaining unit that obtains an image, and a position acquisition that obtains the position of the person to be verified in the image. And the person to be collated acquired by the position acquisition means pass through a predetermined area, or pass through the predetermined area continuously after passing through the predetermined area. If the determination unit determines whether or not the verification target person is a person who has passed through the predetermined area based on the determination result of the determination unit, An image adjusting unit that adjusts an image so that only the determined person to be collated remains, and a face image of the person to be collated determined as the person who has passed through the predetermined area is extracted from the image adjusted by the image adjusting unit. An image extraction unit; a collation unit that calculates and collates the similarity between the face image of the person to be collated extracted by the face image extraction unit and the face image of the registrant accumulated in the accumulation unit; and the collation A similarity determination unit that determines whether or not the face image of the person to be verified is a face image of a registrant by comparing the similarity that is a comparison result of the unit with a predetermined threshold value, and the similarity determination unit Notification means for notifying that the verification target person is the registrant when it is determined that the verification target person is the registrant.

  It is possible to further include a feature amount extraction unit that extracts a feature amount from the face image of the person to be collated determined as the person who has passed the predetermined region, and the collation unit includes the face image extraction unit. The degree of similarity is calculated using the extracted feature amount of the face image of the person to be collated determined as a person who passes the predetermined area and the face image of the registrant stored in the storage unit, and the acquisition unit The acquired face image of the person to be collated can be collated with the face image of the registrant stored in the storage means.

  The position acquisition means can acquire the position of the person to be verified in the image by a stereo image sensor.

  A person position information generating means for generating person position information based on the position of the person to be collated in the image acquired by the position acquiring means can be further included. Based on the position information, the person to be collated acquired by the position acquisition means passes through a predetermined area, or after passing through the predetermined area, the predetermined person that is continuously present in the image as it is It can be determined whether or not the person has passed through the area.

  When the similarity determination unit determines that the collation target person is not the registrant, it further includes an unregistered person registration unit that registers the face image of the collation target person as an unregistered person. Can do.

  The accumulating means can accumulate a face image of a registrant registered in advance as a database. The face of an unregistered person registered more than a predetermined number of times by the unregistered person registering means is stored in the database. An image can be registered as the face image of the registrant.

  The monitoring method according to the first aspect of the present invention includes an image acquisition step for acquiring an image, a position acquisition step for acquiring a position of a verification target person in the image, and the verification target acquired by the processing of the position acquisition step. A determination step of determining whether or not a person is passing a predetermined area or is a person who has passed through the predetermined area and continuously exists in the image as it is, and the determination If it is determined that the person to be verified is a person who has passed the predetermined area based on the determination result in the processing of the step, only the person to be verified that is determined to be a person who has passed the predetermined area in the image is left. An image adjustment step for adjusting the image to the face, and a face for extracting the face image of the person to be collated determined as a person who has passed the predetermined region from the image adjusted by the processing of the image adjustment step An image extraction step, a collation step of calculating and collating the similarity between the face image of the person to be collated extracted by the processing of the face image extraction step and the accumulated face image of the registrant, and A similarity determination step for determining whether or not the face image of the person to be collated is a face image of the registrant by comparing the similarity that is the collation result in the process with a predetermined threshold, and the similarity determination step A notification step of notifying that the verification target person is the registrant when it is determined that the verification target person is the registrant.

  The program according to the first aspect of the present invention includes an image acquisition step of acquiring an image, a position acquisition step of acquiring a position of a verification target person in the image, and the verification target person acquired by the processing of the position acquisition step. A determination step for determining whether or not a user passes through a predetermined region, or after passing through the predetermined region, a user who has passed through the predetermined region continuously in the image, and the determination step When it is determined that the person to be collated is a person who has passed the predetermined area based on the result of the determination in the above process, only the person to be collated that has been determined to be a person who has passed the predetermined area in the image is left. An image adjustment step for adjusting an image, and a face image of the person to be collated determined as a person who has passed the predetermined region is extracted from the image adjusted by the processing of the image adjustment step. An image extraction step; a collation step for calculating and collating the similarity between the face image of the person to be collated extracted by the processing of the face image extraction step and the accumulated face image of the registrant; A similarity determination step for determining whether or not the face image of the person to be collated is a face image of the registrant by comparing the similarity that is the collation result in the process with a predetermined threshold, and the similarity determination step When it is determined that the person to be collated is the registrant by the process, the computer is caused to execute a process including a notification step of notifying that the person to be collated is the registrant.

  The registrant verification device according to the second aspect of the present invention acquires an accumulating means for accumulating a pre-registered registrant's face image, an image acquiring means for acquiring an image, and a position of a verification target person in the image. The predetermined area that is present in the image as it is after the position acquisition means and the person to be acquired acquired by the position acquisition means pass through the predetermined area or after passing through the predetermined area When it is determined that the person to be verified is a person who has passed the predetermined area based on a determination unit that determines whether or not the person is a passer, and a determination result of the determination unit, An image adjustment unit that adjusts an image so as to leave only the person to be collated determined to be present, and a face image of the person to be collated determined as a person who has passed through the predetermined region from an image adjusted by the image adjustment unit Extraction A face image extracting means, a matching means for calculating and comparing the similarity between the face image of the person to be collated extracted by the face image extracting means and the face image of the registrant accumulated in the accumulating means, Similarity determination means for determining whether or not the face image of the person to be collated is a face image of the registrant by comparing the similarity as the collation result of the collation means with a predetermined threshold, and the similarity determination And means for notifying that the verification target person is the registrant when the verification target person is determined to be the registrant.

  The attribute estimation apparatus according to the third aspect of the present invention includes an image acquisition unit that acquires an image, a position acquisition unit that acquires a position of a verification target person in the image, and the verification target person acquired by the position acquisition unit. Determining means for determining whether or not a person has passed through a predetermined area, or has passed through the predetermined area, and is a person who has passed through the predetermined area continuously in the image, and If it is determined that the person to be collated is a person who has passed the predetermined area, the image is adjusted so that only the person to be collated determined to be a person who has passed the predetermined area is left among the images. An image adjusting means for extracting, a face image extracting means for extracting a face image of the person to be collated determined as a person who has passed through the predetermined area from an image adjusted by the image adjusting means, and extraction by the face image extracting means Based on the collation object's face image, including an estimation unit that estimates the attributes of the collation object person.

  It is possible to further include a feature amount extraction unit that extracts a feature amount from the face image of the person to be collated determined as the person who has passed the predetermined region, and the attribute estimation unit includes the face image extraction unit. The attribute of the collation target person acquired by the acquisition unit can be estimated using the feature amount of the face image of the collation target person determined as the person who passed the predetermined area.

  The attribute estimation means can estimate gender or age as the attribute of the person to be collated.

  The attribute estimation method according to the third aspect of the present invention includes an image acquisition step of acquiring an image, a position acquisition step of acquiring a position of a person to be verified in the image, and the verification acquired by the processing of the position acquisition step. A determination step of determining whether the target person is passing through a predetermined area, or after passing through the predetermined area, whether or not the person is a predetermined area passing person continuously present in the image; When it is determined that the person to be verified is a person who has passed the predetermined area based on the determination result in the process of the determination step, only the person to be verified who has been determined to be a person who has passed the predetermined area in the image is left. An image adjustment step for adjusting the image as described above, and extracting the face image of the person to be collated determined as a person who has passed the predetermined region from the image adjusted by the processing of the image adjustment step A face image extraction step, on the basis of the collation object person of the face image extracted by the processing of the facial image extraction step includes an estimation step of estimating an attribute of the collation object person.

  The program according to the third aspect of the present invention includes an image acquisition step of acquiring an image, a position acquisition step of acquiring a position of the verification target person in the image, and the verification target person acquired by the processing of the position acquisition step. A determination step for determining whether or not a user passes through a predetermined region, or after passing through the predetermined region, a user who has passed through the predetermined region continuously in the image, and the determination step When it is determined that the person to be collated is a person who has passed the predetermined area based on the result of the determination in the above process, only the person to be collated that has been determined to be a person who has passed the predetermined area in the image is left. An image adjustment step for adjusting an image, and a face image of the person to be collated determined as a person who has passed the predetermined region is extracted from the image adjusted by the processing of the image adjustment step. An image extraction step, the face image extraction based on the step of the collation object's face image extracted by the processing to execute the processing including an estimation step of estimating an attribute of the collation object person to the computer.

  In the monitoring apparatus, method, and program according to the first aspect of the present invention, pre-registered registrant face images are accumulated, images are acquired, and the position of the person to be verified in the images is acquired and acquired. In addition, it is determined whether or not the person to be collated has passed through a predetermined area, or is a person who has passed through the predetermined area and continuously passes through the predetermined area as it is in the image. As a result, when it is determined that the person to be collated is a person who has passed the predetermined area, the image is adjusted so that only the person to be collated determined to be a person who has passed the predetermined area is left among the images. Then, the face image of the person to be collated determined as a person who has passed through the predetermined area is extracted from the adjusted image, and the similarity between the extracted face image of the person to be collated and the accumulated face image of the registrant is calculated. Being Then, it is determined whether or not the collation target person's face image is a registrant's face image by comparing the similarity as a collation result with a predetermined threshold, and the collation target person is the registrant. If determined, it is notified that the person to be collated is the registrant.

  In the registrant verification device according to the second aspect of the present invention, pre-registered registrant face images are accumulated, an image is acquired, the position of the verification target person in the image is acquired, and the acquired verification It is determined whether or not the target person is passing through a predetermined area, or after passing through the predetermined area, is a person who has passed through the predetermined area continuously in the image. When it is determined that the verification target person is a person who has passed the predetermined area, the image is adjusted and adjusted so that only the verification target person who is determined to be the predetermined area passer is left among the images. The face image of the person to be collated determined as a person who passed the predetermined area is extracted from the obtained image, and the similarity between the extracted face image of the person to be collated and the registered registrant face image is calculated, Matched and is the matching result By comparing the similarity with a predetermined threshold value, it is determined whether or not the face image of the person to be collated is a face image of the registrant, and when it is determined that the person to be collated is the registrant, It is notified that the person to be verified is the registrant.

  In the attribute estimation apparatus, method, and program according to the third aspect of the present invention, an image is acquired, the position of the person to be verified in the image is acquired, and the acquired person to be verified passes through a predetermined region. Or after passing through the predetermined area, it is determined whether or not the person who has passed through the predetermined area is continuously present in the image. When it is determined that the person is a person who has passed the predetermined area, the image is adjusted so that only the person to be verified who has been determined to be a person who has passed the predetermined area is left. The face image of the person to be collated is extracted, and the attribute of the person to be collated is estimated based on the extracted face image of the person to be collated.

  In the monitoring device according to the first aspect of the present invention and the registrant verification device according to the second aspect, the storage means for storing the face image of the registrant registered in advance as the registrant database is, for example, a registration game The image acquisition means for acquiring an image is, for example, an entrance camera, and the position acquisition means for acquiring the position of the person to be verified in the image is, for example, an entrance stereo image sensor, and the position Whether or not the person to be collated acquired by the acquisition means passes through a predetermined area, or is a person who has passed through the predetermined area continuously after passing through the predetermined area For example, the determination means for determining whether the person to be collated is a person who has passed the predetermined area is determined by the determination result of the determination means. The image adjustment means for adjusting the image so that only the person to be collated determined to be a person who has passed the predetermined area in the image is, for example, a face detection frame setting unit, and the image adjustment means The face image extraction unit that acquires the face image of the person to be collated determined as the person who has passed the predetermined region from the adjusted image is, for example, a face image extraction unit, and the collation target extracted by the face image extraction unit The collating means for calculating and collating the similarity between the person's face image and the registrant's face image stored as the registrant database in the accumulating means is, for example, a collating unit, and collating by the collating means The similarity determination means for determining whether or not the face image of the person to be collated is a face image of the registrant by comparing the similarity as a result with a predetermined threshold is, for example, a similarity determination unit , Similar to the above If the collation object person is determined to be the registered person by the determination means, and notification means for notifying that the collation object person is the registrant is a communication unit.

  That is, the entrance passer position information generated by the entrance passer position information generation unit based on the position information of the check target person acquired by the entrance stereo image sensor among the face images of the check target person captured by the entrance camera The face image of the person to be collated is extracted in a state where only the person to be collated included in the information is left in the image, and the similarity determination unit registers only the face image of the person to be collated included in the passer position information. A case where it is determined whether the similarity with the face image of the registrant stored as the player database is the same person by comparing with a predetermined threshold, and it is determined that the person to be collated is the registrant The communication unit notifies that the verification target person is the registrant.

  In the attribute estimation apparatus according to the third aspect of the present invention, the image acquisition means for acquiring an image is, for example, an entrance camera, and the position acquisition means for acquiring the position of the person to be verified in the image is, for example, The entrance stereo image sensor, and the person to be collated acquired by the position acquisition means passes through a predetermined area or continuously exists in the image as it passes through the predetermined area. The determination means for determining whether or not a predetermined area passing person is, for example, an entrance passer position information generation unit, and the verification target person is the predetermined area passing person based on a determination result of the determination means. The image adjustment means for adjusting the image so as to leave only the person to be collated determined to be a person who has passed through the predetermined area in the image, for example, a face detection frame setting unit The face image extracting means for extracting the face image of the person to be collated determined as a person who has passed through the predetermined area from the image adjusted by the image adjusting means is, for example, a face image extracting unit, and the face image Based on the face image of the person to be collated extracted by the extracting means, the estimating means for estimating the attribute of the person to be collated is, for example, an attribute estimation unit.

  That is, the entrance passer position information generated by the entrance passer position information generation unit based on the position information of the check target person acquired by the entrance stereo image sensor among the face images of the check target person captured by the entrance camera The face image of the person to be collated is extracted while only the person to be collated included in the information is left in the image, and the attribute estimation unit estimates the attributes of the person to be collated such as gender and age based on the face image.

  As a result, only the face image of the person to be verified who has passed through the entrance or has been recognized as an entrance passer in the image after passing through the entrance is compared with the registrant. Or the attribute is estimated, so that the face image of a person who is not an entrance passer, for example, a person who has already been verified, is not recognized as a person to be verified, and the face image is not verified. As a result, it is possible to reduce the load on the face image collation process and the attribute estimation process, and it is possible to improve the processing speed for face image collation by eliminating unnecessary processes.

According to the present invention, it is possible to reduce the load on processing using a face image, and to improve the processing speed for collating face images.

  Embodiments of the present invention will be described below. Correspondences between the configuration requirements of the present invention and the embodiments described in the detailed description of the present invention are exemplified as follows. This description is to confirm that the embodiments supporting the present invention are described in the detailed description of the invention. Accordingly, although there are embodiments that are described in the detailed description of the invention but are not described here as embodiments corresponding to the constituent elements of the present invention, It does not mean that the embodiment does not correspond to the configuration requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

  In other words, the monitoring apparatus according to one aspect of the present invention includes a storage unit (for example, the biometric information DB 22 in FIG. 13) that stores a pre-registered registrant's face image, and an image acquisition unit (for example, FIG. 10) that acquires an image. The entrance camera 43), position acquisition means for acquiring the position of the person to be verified in the image (for example, the entrance stereo image sensor 41 in FIG. 12), and the person to be verified acquired by the position acquisition means Determination means (for example, passing the entrance in FIG. 12) that determines whether or not the user has passed through the predetermined area, or has passed through the predetermined area, and continuously passes through the predetermined area. When the person to be collated is determined to be a person who has passed through the predetermined area based on the determination result of the person position information generation unit 273) and the determination unit, the person is determined to be a person who has passed through the predetermined area in the image. The image adjustment means for adjusting the image so as to leave only the person to be collated (for example, the face detection frame setting unit 231 in FIG. 10) and the image adjusted by the image adjustment means are determined as those who have passed the predetermined area. In addition, the face image extracting means (for example, the face image extracting unit 202 in FIG. 10) for extracting the face image of the person to be collated, the face image of the person to be collated acquired by the face image extracting means, and the storage means A collation unit (for example, the similarity calculation unit 322 in FIG. 13) that calculates and collates the similarity with the accumulated face image of the registrant, and a similarity that is a collation result of the collation unit and a predetermined threshold value. Similarity determination means (for example, the similarity determination unit 323 in FIG. 13) that determines whether the face image of the person to be collated is a face image of the registrant by comparison, and the collation by the similarity determination means. The target person In the case where it is determined that the person is a recorder, notification means (for example, the communication unit 304 in FIG. 13) for notifying that the person to be verified is the registrant is included.

  A feature amount extraction unit (for example, a feature amount extraction unit 321 in FIG. 13) that extracts a feature amount from the face image of the person to be collated determined as the person who has passed the predetermined region can be included. In the collation means (for example, the similarity calculation unit 322 in FIG. 13), feature amounts of the face image of the person to be collated acquired by the acquisition means and the face image of the registrant accumulated in the accumulation means are stored. It is possible to calculate the degree of similarity and to collate the face image of the person to be collated acquired by the obtaining unit with the face image of the registrant accumulated in the accumulating unit.

  The position acquisition means (for example, the entrance stereo image sensor 41 in FIG. 12) can acquire the position of the person to be verified in the image by the stereo image sensor.

  A person position information generating unit (for example, a person position information generating unit 272 in FIG. 12) that generates person position information based on the position of the person to be collated in the image acquired by the position acquiring unit is further included. In the determination means (for example, the entrance passer position information generation unit 273 in FIG. 12), based on the predetermined area passer information, the verification target person acquired by the position acquisition means It is possible to determine whether or not the user has passed through a predetermined area, or has passed through the predetermined area, and continues to pass through the predetermined area as it is in the image.

  When the similarity determination means determines that the person to be verified is not the registrant, unregistered person registration means (for example, the unregistered game in FIG. 13) registers the face image of the person to be verified as an unregistered person. A user database registration unit 303).

  The monitoring method and program according to an aspect of the present invention include an image acquisition step (for example, step S141 in FIG. 20) for acquiring an image, and a position acquisition step (for example, in FIG. 20) for acquiring the position of the person to be verified in the image. Step S122) and the person to be collated acquired by the processing of the position acquisition step passes through a predetermined area, or continuously exists in the image as it is after passing through the predetermined area. A determination step (for example, step S214 in FIG. 21) for determining whether or not the person has passed the predetermined area and a determination result in the process of the determination step determine that the person to be collated is the person who has passed the predetermined area. If so, an image adjustment step for adjusting the image so as to leave only the person to be collated determined to be a person who has passed the predetermined area in the image ( For example, step S144 in FIG. 20 and a face image extraction step (for example, FIG. 20) that extracts the face image of the person to be collated determined as a person who has passed the predetermined area from the image adjusted by the process of the image adjustment step. Step S145), and a collation step of calculating and comparing the similarity between the face image of the person to be collated acquired by the processing of the face image extraction step and the face image of the registrant stored in the accumulating means ( For example, whether or not the face image of the person to be collated is a face image of the registrant based on a comparison between the similarity that is the collation result in the process of the collation step and a predetermined threshold value in step S164) of FIG. It is determined that the person to be collated is the registrant by the similarity determination step (for example, step S165 in FIG. 20) and the processing of the similarity determination step. If, and a notification step of the collation object person is notified that the said registered person (e.g., step S168 in FIG. 20).

  The registrant verification device according to the second aspect of the present invention includes a storage unit (for example, the biometric information DB 22 in FIG. 13) that stores a pre-registered registrant's face image, and an image acquisition unit (for example, an image acquisition unit). The entrance camera 43 in FIG. 10), position acquisition means (for example, the entrance stereo image sensor 41 in FIG. 12) for acquiring the position of the person to be verified in the image, and the person to be verified acquired by the position acquisition means Determination means for determining whether or not the user has passed through a predetermined area, or has passed through the predetermined area, and continuously passes through the predetermined area in the image (for example, FIG. 12 When it is determined by the determination result of the entrance passer position information generation unit 273) and the determination means that the verification target person is the predetermined area passer, it is determined that the predetermined area passer is included in the image. The The image adjustment means for adjusting the image so as to leave only the person to be collated (for example, the face detection frame setting unit 231 in FIG. 10) and the image adjusted by the image adjustment means are determined as those who have passed the predetermined area. The face image extraction means (for example, the face image extraction unit 202 in FIG. 10) that extracts the face image of the person to be collated, the face image of the person to be collated acquired by the acquisition means, and the storage means. By comparing the degree of similarity with the registrant's face image and collating it (for example, the similarity calculating unit 322 in FIG. 13), and comparing the similarity as a result of the collation with the predetermined threshold. Similarity determination means (for example, similarity determination unit 323 in FIG. 13) for determining whether or not the face image of the person to be collated is the face image of the registrant, and the person to be collated by the similarity degree determination means Is the registrant In the case where it is determined that there is, a notification means (for example, the communication unit 304 in FIG. 13) for notifying that the person to be collated is the registrant is included.

  The attribute estimation apparatus according to the third aspect of the present invention includes an image acquisition unit (for example, the entrance camera 43 in FIG. 10) that acquires an image, and a position acquisition unit (for example, FIG. 10) that acquires the position of the person to be verified in the image. 12 entrance stereo image sensors 41) and the person to be collated acquired by the position acquisition means pass through a predetermined area, or after passing through the predetermined area, Based on the determination result (for example, the entrance passer position information generation unit 273 in FIG. 12) and the determination result of the determination means, the person to be verified is determined to be the predetermined area. When it is determined that the person has passed, image adjustment means for adjusting the image so that only the verification target person who has been determined to pass the predetermined area is left among the images (for example, the face detection frame of FIG. 10). Face setting unit 231) and face image extracting means for extracting the face image of the person to be collated determined as a person who has passed the predetermined area from the image adjusted by the image adjusting means (for example, face image acquisition in FIG. 10) Unit 202) and estimation means (for example, attribute estimation unit 307 in FIG. 13) for estimating the attribute of the verification target person based on the face image of the verification target person extracted by the face image extraction means.

  A feature amount extraction unit (for example, a feature amount extraction unit 321 in FIG. 13) that extracts a feature amount from the face image of the person to be collated determined as the person who has passed the predetermined region can be further included. The attribute estimation unit (for example, the attribute estimation unit 307 in FIG. 13) uses the feature amount of the face image of the person to be collated determined by the predetermined area extracted by the face image extraction unit, The attribute of the person to be collated acquired by the acquisition unit can be estimated.

  The attribute estimation means (for example, the attribute estimation unit 307 in FIG. 13) can be made to estimate gender or age as the attribute of the person to be collated.

  The attribute estimation method according to the third aspect of the present invention includes an image acquisition step (for example, step S141 in FIG. 20) for acquiring an image and a position acquisition step (for example, FIG. 20) for acquiring the position of the person to be verified in the image. Step S122) and the person to be collated acquired by the processing of the position acquisition step passes through the predetermined area, or continuously exists in the image as it passes through the predetermined area. A determination step (for example, step S214 in FIG. 21) for determining whether or not the person who has passed the predetermined area is determined, and the result of determination in the process of the determination step indicates that the person to be collated is the person who has passed the predetermined area If determined, an image adjustment step for adjusting the image so as to leave only the verification target person determined to be a person who has passed the predetermined area in the image (for example, 20 step S144) and a face image extraction step (for example, step S145 of FIG. 20) for extracting the face image of the person to be collated determined as a person who has passed the predetermined area from the image adjusted by the processing of the image adjustment step. ) And an estimation step (for example, steps S116 to S118 in FIG. 19) for estimating the attribute of the collation target person based on the face image of the collation target person extracted by the processing of the face image extraction step.

  FIG. 1 is a diagram showing a configuration of an embodiment of a game shop monitoring system according to the present invention.

  The game stores 1-1 to 1-n are so-called pachinko stores or pachislot stores. These amusement stores 1-1 to 1-n are affiliated stores or member stores of the biometric information management center or the third party amusement store management center, and stores that need to manage a plurality of stores in an integrated manner. It is. The game stores 1-1 to 1-n are connected by a biological information management bus 6 and a third-party game store management bus 7, and the public communication line networks 8 and 9 represented by the bus and the Internet are used. Via each other, biological information and third-party amusement store management information are exchanged. In the following description, when there is no particular need to distinguish each of the game stores 1-1 to 1-n, the game store 1 is simply referred to, and the other configurations are also referred to in the same manner.

  The biometric information management bus 6 mainly functions as a transmission path for distributing biometric information managed by the biometric information recognition device 21 of each game store 1. The third-party amusement store management bus 7 functions as a transmission path for distributing medium lending management information mainly managed by the medium lending device management device 27 of each amusement store 1.

  The biometric information management center 2 is a server used by a business operator who manages and operates the biometric information management center, and is a registered player DB 352 (hereinafter referred to as DB) 3 managed by a biometric information management database (hereinafter also referred to as DB). 14) is updated based on the unregistered player DB 351 (FIG. 14) generated by each game store 1, and the updated latest registered player DB 352 (FIG. 14) is updated with the biometric information of each game store 1. Delivered to the recognition device 21.

  The third-party amusement store management center 4 is a server used by a business operator that manages and operates the third-party amusement store management center, and the medium lending management managed by the third-party amusement store management database (DB) 5 In addition to updating the information DB based on the information supplied from each game store 1, the updated latest medium lending management information is distributed to the medium lending management device 27 of each game store 1.

  The biological information recognition device 21 is extracted by the image processing units 39-1 to 39- (m + p) from the images captured by the cameras 38-1 to 38-m and the in-store cameras 40-1 to 40-p, and the biological information recognition device 21 Based on the face image information supplied via the information bus 31, the face image registered in advance in the biometric information database 22 is checked, and if they match, the registered player's visit is notified to the portable terminal 45. Or displayed on the display unit 23 such as CRT (Cathode Tube Ray) or LCD (Liquid Crystal Display). Further, when the face image registered in the biometric information database 22 is collated and does not match, the biometric information recognition device 21 accesses the biometric information management database 3 and unregistered player DB 351 (FIG. 14) as an unregistered person. ).

  The game store management device 24 is a so-called hall computer, and monitors the operations of the game machines 36-1 to 36-m via the game store management information bus 30. The game store management device 24 determines a predetermined amount according to the monitoring status such as information on the payout of the game table 36 or the payout of medals, the call information of the players on the game tables 36-1 to 36-m, or the occurrence of an error. The execution result is executed, and the execution result is displayed on the display unit 25 such as a CRT or LCD. The game store management device 24 is supplied from each of the checkout vending machine 33, the lending machine 34, the counter 35, the game machines 36-1 to 36-m, and the game machine peripheral terminals 37-1 to 37-m. The incoming information is managed by the gaming machine management database 26 in association with identification information (for example, gaming machine number) for identifying each information.

  The medium lending management device 27 manages the medium lending management information of the rented game media using the medium lending management database 29 based on the information from the payment machine 33, the lending machine 34, and the counter 35. When the medium lending management information registered in the medium lending management database 29 is updated, the updated information is sent to the third party amusement store management center 4 via the third party amusement store management bus 7 and the public communication line network 9. Send to. Further, the medium lending management device 27 acquires the medium lending management information supplied by the third party amusement store management center 4 via the third party amusement store management bus 7 and the public communication line network 9, and manages the medium lending management. Store in the database 29.

  The lending machine 34 lends a number of game media according to the amount when the player accepts a predetermined amount by cash or a prepaid card when playing on the gaming table 36. At this time, the lending machine 34 supplies information on the number of lent gaming media to the medium lending management device 27 together with information such as the accepted cash and the remaining number of prepaid cards. Thereby, the medium lending management device 27 registers information on the number of lent gaming media in the medium lending management information database 29 together with information such as the received cash and the remaining number of prepaid cards.

  The checkout machine 33 sells prepaid cards with a frequency for borrowing a ball. At this time, the checkout machine 33 supplies the frequency of the prepaid card sold and the amount paid to the medium lending management device 27. Further, the settlement vending machine 33 settles and pays out cash based on the remaining number of game media lent as a frequency such as a prepaid card. At this time, the checkout vending machine 33 supplies the information on the remaining number of prepaid cards and the amount of cash refunded to the medium lending management device 27.

  The counter 35 counts the number of game media acquired by the player playing on the game table 36, and outputs the counting result as a magnetic card or a receipt.

  The game machines 36-1 to 36-m execute a game when a predetermined operation is performed by the player, and pay out a game ball or a medal according to a so-called small hit or big hit.

  The game console peripheral terminals 37-1 to 37-m are so-called pedestrian machines provided corresponding to the respective game machines 36-1 to 36-m. The same as the lending machine 34) is provided. Further, the gaming machine peripheral terminal 37 acquires biological information such as a face image of a player who plays the gaming machine 36 and transmits it to the gaming machine management device 24 together with the gaming machine identification information (game machine number). FIG. 1 shows an example in which cameras 38-1 to 38-m that acquire a player's face image are provided as a function of acquiring biometric information.

  For example, as shown in FIG. 2, the cameras 38-1 to 38-m are arranged below the table display lamps 61-1 to 61-4 provided on the respective upper sides of the game tables 36-1 to 36-4. As shown in FIG. 3, a face image may be taken so that the player can take an image within the reading range δ. Can be used.

  Further, as shown in FIG. 4, the cameras 38-1 to 38-m are provided with convex portions 71-1 to 71-4 on the gaming machine peripheral terminals 37-1 to 37-4, as shown in FIG. As described above, it may be provided so that a player's face image can be captured within the reading range θ.

  Further, for example, as shown in FIG. 6, the cameras 38-1 to 38-m may be provided at the center of the game table 36 (on the board surface of the game table 36) to take an image. That is, by installing the camera 36 in the installation unit 81 of FIG. 6, the player is imaged within the reading range φ as shown in FIG.

  The in-store cameras 40-1 to 40-p are installed at predetermined locations in the store of the game store 1, and supply captured images to the image processing units 39- (m + 1) to 39- (m + p), respectively.

  The entrance cameras 43-1 to 43-q are installed at positions where the entrance / exit 112 in the store of the game store 1 can be imaged, and supply the captured images to the entrance image processing units 44-1 to 44-q, respectively.

  The entrance stereo image sensors 41-1 to 41-q are provided at positions corresponding to the entrance cameras 43-1 to 43-q, respectively, and are almost the same as images captured by the entrance cameras 43-1 to 43-q. The same image is taken and the spatial position information of the person in the image is supplied to the position recognition servers 42-1 to 42-q. The position recognition servers 42-1 to 42-q use the position information in the images supplied from the entrance stereo image sensors 41-1 to 41-q, respectively, as the position information of the person passing through the entrance / exit 112. Based on the information, the entrance passer position information is generated and supplied to the entrance image processing units 44-1 to 44-q.

  The image processing units 39-1 through 39- (m + p) extract face images from the images supplied from the cameras 38-1 through 38-m and the in-store cameras 40-1 through 40-p to extract the biometric information recognition device 21. To supply.

  The basic functions of the entrance image processing units 44-1 to 44-q are the same as those of the image processing unit 39, but the positions of the images supplied from the entrance cameras 43-1 to 43-q are the same. Only the face images of the players passing through the entrances 112-1 to 112-3 are extracted from the recognition servers 42-1 to 42-q and supplied to the biological information recognition apparatus 21.

  The in-store cameras 40-1 to 40-p, the entrance cameras 43-1 to 43-p, and the entrance stereo image sensors 41-1 to 41-q are set as shown in FIG. 8, for example. FIG. 8 shows an installation example of the in-store cameras 40-1 to 40-p, the entrance cameras 43-1 to 43-q, and the entrance stereo image sensors 41-1 to 41-q in the game store 1.

  That is, in FIG. 8, entrances 112-1 to 112-3 through which general players enter and exit are provided, and the entrance cameras 43-1 to 43-3 are provided by the respective entrances 112-1 to 112-3. The player who enters the store is imaged. In addition, the entrance stereo image sensors 41-1 to 41-q include areas that can be imaged by the entrance cameras 43-1 to 43-3, or images substantially the same area using a plurality of cameras, and triangulation is performed. The position of the player in the image is measured based on the same principle as the measurement method according to, and the position information of the player in the image as the measurement result is supplied to the position recognition servers 42-1 to 42-q.

  Further, the in-store cameras 40-1 to 40-10 are set at positions where each side of the island facilities 111-1 to 111-5 can be imaged in a row. The island facilities 111 are provided with game machines 36 on both sides, that is, so as to sandwich the island facilities 111 in the vertical direction in the figure. Since the camera 38, the in-store camera 40, and the entrance camera 43 all have a pan / tilt / zoom function, as shown in FIG. 8, the in-store cameras 40-1 to 40-10 are arranged so that the game table All players who play at 36 can take images with any of the in-store cameras 40-1 to 40-10.

  Further, the in-store camera 40-a is provided in front of the lending machine 34, the in-store camera 40-b is provided in front of the checkout machine 33, and the in-store camera 40-c is the counter of the counter 35. It is provided in front, and it is possible to take an image of a player who uses the checkout machine 33, the lending machine 34, and the counter 35, respectively.

  That is, as shown in FIG. 8, in the game store 1, a player who visits the store, a player who plays on the game stand 36, and a player who uses the checkout vending machine 33, the lending machine 34, and the counter 35. The camera 38, the in-store camera 40, and the entrance camera 43 are installed so that almost all of the actions that the player is supposed to take in the gaming store 1 can be monitored. In addition, an employee entry / exit 113 is provided, and an employee of the amusement shop enters or exits the amusement shop 1 through the employee entry / exit 113. As a result, since only the persons involved in the amusement store can enter and leave the employee entrance / exit 113, there is no need to provide the entrance camera 43 for detecting the registrant. Therefore, there is no person related to the game store among the players in the image captured by the entrance camera 43. As a result, the image captured by the entrance camera 43 does not include the face images of the persons concerned including the employees of the game shop that do not need to be processed. Therefore, it is possible to reduce the processing load.

  Next, a configuration example of the image processing unit 39 will be described with reference to the functional block diagram of FIG.

  The image acquisition unit 201 acquires an image captured by the camera 38 (or the in-store camera 40) and supplies it to the face image extraction unit 202. The face image extraction unit 202 includes a face detection unit 211 and a face image cutout unit 212. The face detection unit 211 detects a part including a face image in the image supplied from the image acquisition unit 201 based on a pattern such as the arrangement of the parts constituting the face, and supplies the detection result to the face image clipping unit 212. . The face image cutout unit 212 extracts a rectangular image including only the face image based on information on a part including the face image detected from the captured image, and supplies the extracted image to the transmission unit 203. The transmission unit 203 transmits the face image to the biometric information recognition device 21.

  Next, a configuration example of the entrance image processing unit 44 will be described with reference to the functional block diagram of FIG. In FIG. 10, the same components as those of the image processing unit 39 described with reference to FIG. 9 are denoted by the same reference numerals, the description thereof will be omitted as appropriate, and the other components will be similarly described. Shall be handled.

  The entrance image processing unit 44 in FIG. 10 differs from the image processing unit 39 in FIG. 9 in that a face detection frame setting unit 231 is provided between the face image acquisition unit 201 and the face image extraction unit 202. .

  The face detection frame setting unit 231 is a player who has not passed through the entrance / exit 112 out of images supplied from the image acquisition unit 201 based on the entrance passer position information supplied from the position recognition server 42. The frame is set (adjusted) and supplied to the face image extraction unit 202 so that only the face image of the entrance passer can be extracted by deleting the player image not included in the entrance passer information. As a result, in the entrance image processing unit 44, the face image extraction unit 202 extracts only the face image of the player who has passed through the entrance 112.

  Next, a configuration example of the entrance stereo image sensor 41 will be described with reference to the functional block diagram of FIG.

  As shown in FIG. 11, the entrance stereo image sensor 41 includes cameras 251-1 and 251-2, each of which captures the same region from different angles and outputs it to the arithmetic processing unit 252. The arithmetic processing unit 252 is imaged by performing arithmetic processing using parallax on images with different angles supplied from the cameras 251-1 and 251-2 (so-called arithmetic processing using trigonometry). The spatial position of the subject in the image is obtained as three-dimensional coordinates and supplied to the position recognition server 42 for each player who becomes the subject.

  Next, a configuration example of the position recognition server 42 will be described with reference to the functional block diagram of FIG.

  The person detection unit 271 acquires the player's spatial three-dimensional position information supplied from the entrance stereo image sensor 41, and adds a person ID to each piece of position information to generate person position information. To the unit 272.

  The person position information generation unit 272 includes a coordinate conversion unit 272a and an entrance detection determination unit 272b. The person position information generation unit 272 controls the coordinate conversion unit 272a to convert the 3D position information supplied together with the person ID into the entrance camera. In addition to the conversion to the two-dimensional coordinates in the image captured by 43, the entrance detection determination unit 272b is controlled to determine whether the person exists at the entrance 112 for each person ID from the position information in the image. The person position information is generated and stored in the memory 274, and the fact that the person position information is generated in the memory 274 is supplied to the entrance passer position information generating unit 273.

  The entrance passer position information generation unit 273 includes an entrance passer determination unit 273a, and when notified that the person location information stored in the memory 274 has been updated, the person location stored in the memory 274 is stored. The information 274 is read out, the entrance passer determination unit 273a is controlled, and the entrance consisting only of the person ID and the location information of the player who has passed through the entrance 112 out of the information of the person ID stored in the person location information 274. Passer position information is generated and supplied to the entrance image processing unit 44.

  Next, a configuration example of the biological information recognition device 21 will be described with reference to the functional block diagram of FIG.

  The face image acquisition unit 301 acquires the face image supplied from the image processing unit 39 or the entrance image processing unit 44 and supplies it to the collation unit 302. The collation unit 302 collates the face image acquired by the face image acquisition unit 301 with the registered player's face image registered in advance in the biometric information DB 22, and if there is a face image that is a candidate with a high degree of similarity. The face images up to the third candidate are displayed on the display unit 23 as the matching result. The collation unit 302 supplies the supplied face image to the unregistered player database registration unit 303 when there is no face image that is a candidate with a high degree of similarity.

  More specifically, the feature amount extraction unit 321 of the matching unit 302 extracts a feature amount for identifying a face image, and supplies the feature amount together with the face image to the similarity calculation unit 322 and the attribute estimation unit 307. The similarity calculation unit 322 extracts the feature amount of the registered player's face image registered in the biometric information DB 22 and uses the feature amount supplied from the feature amount extraction unit 321 to use the biometric information DB 22. The degree of similarity with the face images of all registered players registered in the face image is obtained, and the face images supplied from the face image acquisition unit 301 and the face images with the highest degree of similarity are ranked in the similarity degree determination unit. 323. More specifically, the similarity calculation unit 322, for example, based on various facial feature amounts such as the ratio between the distance between the eyes, the distance from the chin to the forehead, and the length from the chin to the nose, Each difference sum, average ratio, or ratio sum is obtained as the similarity.

  The similarity determination unit 323 includes a similarity with the face image that is ranked first among the similarities of the face images with the highest degree of similarity supplied from the similarity calculation unit 322, and a predetermined threshold value. And the first registered face image is similar to the face image supplied from the face image acquisition unit 301 based on the comparison result (the higher the degree of similarity, the more similar the face image is) If the similarity is higher than a predetermined threshold, or if the similarity is lower, the similarity is lower than the predetermined threshold) Information on the degree of similarity with the image is supplied to the display unit 23 and is displayed to the communication unit 304 as it is displayed. In addition, the similarity determination unit 323 compares the similarity with the first-ranked face image and a predetermined threshold, and the registered face image that is first-ranked from the face image acquisition unit 301 based on the comparison result. When the supplied face image is not similar, the face image supplied from the face image acquisition unit 301 is supplied to the unregistered player database registration unit 303.

  The unregistered player database registration unit 303 registers the face image that has been supplied as being unregistered by the verification unit 302 in the unregistered player DB 351 of the biometric information management database 3.

  The operation unit 305 includes buttons, a mouse, a keyboard, and the like. The operation unit 305 is operated when any one of the face images displayed on the display unit 23 with the above-mentioned similarity ranked in the top three is selected. Is supplied to the communication unit 304. The communication unit 304 includes a modem or the like, and distributes the selected face image to the mobile terminal 45 based on an operation signal from the operation unit 305.

  Here, for example, the similarity indicates a higher value as it is closer to the face image registered as a registered player as indicated by the ratio sum, and the similarity is a value higher than a predetermined threshold value. In this case, an example will be described in which it is determined that the face image of the registered player corresponding to the similarity is. However, for example, when the similarity is expressed as a difference sum in the respective feature amounts of the face image captured as a registered player and the face image registered as a registered player, the similarity determination unit 323 determines that the similarity is a threshold value. If it is smaller than that, the captured face image is regarded as a registered player's face image, or in the case of an average ratio or the like, it is greater than or equal to a predetermined value in the range of 0 to 1 and close to 1. If it is a value, it can be considered that they are the same person.

  When a new registered player database 352 (FIG. 14) is distributed from the biometric information management center 2, the database management unit 306 updates the biometric information DB 22 based on the new registered player database 352.

  The attribute estimation unit 307 includes a gender estimation unit 307a and an age estimation unit 307b. The gender estimation unit 307a and the age estimation unit 307b estimate the gender and age of the player of the face image based on the feature amount supplied from the feature amount extraction unit 321 of the collation unit 302, and display the estimation result 23 To display.

  Next, a configuration example of the biological information management center 2 will be described with reference to the functional block diagram of FIG.

  The biometric information management center 2 includes a DB distribution unit 341, a DB update unit 342, and a DB update determination unit 343. The registered player DB 352 stored in the biometric information management DB 3 is transferred to the unregistered player DB 351. Update based on. More specifically, the DB update determination unit 343 has a built-in RTC (Real Time Clock) 343a that generates time information. When a predetermined time elapses based on the built-in RTC 343a, the biological information management DB3 is accessed, and it is determined whether or not a new unregistered player is registered in the unregistered player DB 351.

  When a new unregistered person is registered in the unregistered player DB 351, the DB update determination unit 343 notifies the DB update unit 342 to that effect. The DB update unit 342 accesses the unregistered player DB 351, determines whether there is an unregistered player that has been registered a predetermined number of times or more among registered unregistered players, and the predetermined number of times or more. When there is an unregistered player registered as an unregistered player, the registered player DB 352 is read, and an unregistered player registered for a predetermined number of times or more is registered in the registered player DB 352 and updated. Further, when the registered player DB 352 is updated, the DB update unit 342 notifies the DB distribution unit 341 of the update.

  When notified by the DB update unit 342 that the registered player DB 352 has been updated, the DB distribution unit 341 accesses the biometric information management DB 3 and reads the registered player DB 352 to recognize the biometric information of each gaming store 1. Delivered to the device 21.

  Next, a configuration example of the mobile terminal 45 will be described with reference to FIG.

  The communication unit 371 is configured by a modem or the like, and exchanges data with the biological information recognition device 21 via a wireless communication network in the game store 1. In addition, the communication unit 371 receives information indicating that a player with a facial image similar to the facial image supplied from the image processing unit 39 or the entrance image processing unit 44 is delivered from the biological information recognition apparatus 21. Obtained and supplied to the image processing unit 372.

  Based on the information indicating that the player of the face image similar to the face image supplied from the image processing unit 39 or the entrance image processing unit 44 supplied from the communication unit 371 has come to the store, the image processing unit 372 In addition to generating an image to be displayed on the display unit 373 configured by the above, the display unit 373 displays the image.

  Next, a registered player visit monitoring process when the camera 38 (or in-store camera 40) is used will be described with reference to the flowchart of FIG.

  In step S <b> 1, the camera 38 determines whether or not a game is started by the game table 36 based on a signal from the game table 36, and repeats the process until it is determined that the game is started.

  If it is determined in step S <b> 1 that the game has started, in step S <b> 2, the camera 38 captures an image of the installed range and supplies the captured image to the image processing unit 39. The image acquisition unit 201 of the image processing unit 39 acquires the supplied image and supplies it to the face image extraction unit 202.

  In step S <b> 3, the face detection unit 211 of the face image extraction unit 202 detects a part where the player's face image exists from the supplied image, and supplies the detection result to the face image cutout unit 212. The face image cutout unit 212 extracts a face image by cutting out a rectangular face image from the image based on the information indicating the position of the part where the detected face image exists, and supplies the face image to the transmission unit 203. More specifically, the face detection unit 211 of the face image extraction unit 202 is, for example, a part where the skin is exposed from the color of the captured image, and the face is determined based on the color scheme of characteristic parts such as eyes and nose. A part in the image where the image exists is detected, and the face image cutout unit 212 extracts the rectangular image from the image based on the information on the part where the face image exists as a detection result, The data is supplied to the transmission unit 203.

  In step S <b> 4, the transmission unit 203 transmits the face image supplied from the face image extraction unit 202 to the biological information recognition device 21. At this time, the transmission unit 203 adds information such as a camera ID for identifying each of the camera 38 and the in-store camera 40 and information on transmission time to the face image and transmits the information to the biometric information recognition apparatus 21.

  In step S21, the face image acquisition unit 301 of the biological information recognition device 21 acquires a face image. In step S <b> 22, the face image acquisition unit 301 extracts one unprocessed one of the supplied face images and supplies the extracted unprocessed one to the feature amount extraction unit 321.

  In step S23, the feature amount extraction unit 321 of the collation unit 302 extracts the feature amount from the supplied face image and supplies it to the similarity calculation unit 322 together with the face image.

  In step 24, the similarity calculation unit 322 determines, for the face image supplied from the feature amount extraction unit 321, the ratio of the distance between eyes, the length from the chin to the forehead, and the length from the chin to the nose. Obtain various facial feature quantities, calculate the difference sum, average ratio, or ratio sum of each as the similarity, calculate the ranking of the similarity as the calculation result, and the top three face images and similarities Is supplied to the similarity determination unit 323 together with the face image supplied from the feature amount extraction unit 321.

  In step S25, the similarity determination unit 323 determines whether the highest similarity is larger than a predetermined threshold based on the top three face images and similarity information supplied from the similarity calculation unit 322. Determine whether or not. That is, the similarity determination unit 323 is the most similar registered player (the registered game most similar to the face image acquired by the face image acquisition unit 301 among the face images registered in the biometric information DB 22). In this case, the similarity of the registered player having the highest similarity is compared with a predetermined threshold.

  Note that, as described above, the similarity between the face image of the registered player who is most similar to the captured face image is not necessarily the highest because of the definition of the similarity. And the threshold value may differ from those in this example.

  In step S <b> 26, the similarity determination unit 323 displays a notification screen 401 indicating that the top three face images supplied from the similarity calculation unit 322 are candidates for registered player's face images, and displays the display unit 23. Control and display.

  At this time, for example, a notification screen 401 as shown in FIG. 17 is displayed on the display unit 23.

  In the notification screen 401 of FIG. 17, the camera image display field 411 is provided in the middle left, and the face image supplied from the image processing unit 39 is displayed. Also, on the right side, face image display fields 412-1 to 412-3 of registered players in the top three ranks of similarity from the first candidate to the third candidate are provided in descending order of similarity. Further, similarity level display fields 413-1 to 413-3 are provided below the face image display fields 412-1 to 412-3 of each registered player, and the level of similarity is displayed. Yes. In FIG. 17, the horizontal length of the area shown in black indicates the degree of similarity.

  Further, ID display fields 414-1 to 414-3 are provided at corresponding positions below the similarity level display fields 413-1 to 413-3, and the face images in the biological information DB 22 of each face image. In FIG. 17, “00051”, “00018”, and “00022” are displayed from the left in FIG.

  Furthermore, below the ID display fields 414-1 to 414-3, the confirmation buttons 418-1 to 418-3 confirmation buttons operated by the operation unit 305 when the respective candidates are selected at the corresponding positions. 418-1 to 418-3 are provided.

  A camera ID display field 415 for identifying the camera that captured the face image is provided under the camera image display field 411. In FIG. 17, the camera 38 and the in-store camera 40 are identified. “Camera 02” is displayed as the camera ID. Further, a time display field 416 is provided below the camera ID display field 415, and the time taken by the camera 38 set as “camera 02” as the camera ID is displayed. In FIG. , “18:23:32” is displayed, indicating that the face image in the camera image display field 411 is captured at 18:23:32.

  Further, another person button 417 is provided below the time display field 416, and the face image of the registered player, whose face image of the camera image is the first to third candidates, is displayed. 3 is operated by the operation unit 305.

  In step S27, the communication unit 304 displays on the display unit 23 whether or not any of the candidate face images has been selected by operating the operation unit 305, that is, for example, the notification screen 401 illustrated in FIG. If it has been determined, it is determined whether any of the confirmation buttons 418-1 to 418-3 has been operated by the operation unit 305.

  In step S27, for example, when the confirm button 418-1 is operated, it is considered that the face image to be the first candidate has been selected, and in step S28, the communication unit 304 selects the face image to be the first candidate selected. And the camera image imaged with the camera 38 is transmitted to the portable terminal 45, and it notifies that the corresponding registered player came to the store.

  In step S41, the communication unit 371 determines whether or not the registered player has been notified of the store visit, and repeats the processing until the notification is received. For example, in step S41, when the registered player's visit is notified by the process of step S28, in step S42, the communication unit 371 notifies the registered player's visit sent from the biometric information recognition device 21. , The face image of the registered player and the camera image captured by the camera 38 transmitted together with the notification are supplied to the image processing unit 372. The image processing unit 372 processes information on the selected face image and the camera image captured by the camera 38 into information in a format that can be displayed on the display unit 373, and displays the information on the display unit 373 in step S43.

  By the above processing, the staff in the game store 1 can recognize the visit of the registered player if the portable terminal 45 is possessed.

  In step S29, the face image acquisition unit 301 determines whether or not all of the supplied face images have been processed. If there is an unprocessed face image, the process returns to step S22. That is, the processes in steps S22 to S30 are repeated until the process is performed on all face images. If it is determined that the process has been completed for all face images, the process returns to step S21.

  On the other hand, no candidate face image is selected in step S27. For example, when a different person button 417 on the notification screen 401 in FIG. 17 is pressed, or in step S25, the similarity calculation unit 322 supplies the face image. Based on the information of the top three face images and the similarity, the highest similarity is not greater than a predetermined threshold, that is, even the face image of the most similar registered player If the similarity is less than the predetermined threshold, the similarity determination unit 323 supplies the face image supplied from the image processing unit 39 to the unregistered player database registration unit 303 in step S30. The unregistered player database registration unit 303 accesses the biometric information management database 3 through the biometric management information bus 6 and the public communication line network 8 to register the supplied face image in the unregistered player DB 351.

  If the face image supplied from the image processing unit 39 by the biometric information recognition device 21 is not registered in the biometric information DB 22 by the above processing, the biometric information management managed by the biometric information management center 2 is performed. The face image of the unregistered player is registered in the unregistered player DB 351 in DB3.

  Next, update processing of the registered player DB 352 will be described with reference to the flowchart of FIG.

  In step S61, the DB update determination unit 343 of the biometric information management center 2 makes an inquiry to the RTC 343a, determines whether or not a predetermined time has elapsed, and repeats the process until the predetermined time has elapsed. For example, if it is determined in step S61 that the predetermined time has elapsed, in step S62, the DB update determination unit 343 accesses the unregistered player DB 351 of the biometric information management DB 3, and an unregistered player is newly added. It is determined whether or not it is registered, and the processes of steps S61 and S62 are repeated until it is determined that a new unregistered player is registered.

  In step S62, for example, when an unregistered player is newly registered by the process of step S30, the DB update determination unit 343 in step S63, the unregistered game newly registered in the unregistered player DB 351. It is determined whether or not one of the users has been registered a predetermined number of times or more. If not, the process returns to step S61.

  On the other hand, in step S63, for example, when it is determined that there is an unregistered player registered in the unregistered player DB 351 a predetermined number of times or more, the DB update determination unit 343 determines in step S64 that the DB update unit 342 Then, the registered player DB 352 is notified that it should be updated. In response to this, the DB update unit 342 accesses the unregistered player DB 351, reads all the face image information of unregistered players that have been registered a predetermined number of times, and accesses the registered player DB 352. Then, an ID for identifying the face image is added to the information of the face image of the unregistered player who has been registered a predetermined number of times or more, and the information is additionally registered in the registered player DB 352 and updated. Furthermore, the DB update unit 342 notifies the DB distribution unit 341 that the registered player DB 352 has been updated. When the update of the registered player DB 352 is completed, unregistered players that have been registered a predetermined number of times or more are deleted from the unregistered player DB 351.

  In step S65, the DB distribution unit 341 accesses and reads the newly updated registered player DB 352 of the biometric information management DB 3, and reads the registered player DB 352 read to the database management unit 306 of the biometric information recognition device 21. To deliver.

  In step S81, the database management unit 306 determines whether or not a new registered player DB 352 updated from the biometric information management center 2 has been distributed, and repeats the processing until it is distributed.

  In step S81, for example, when the newly updated registered player DB 352 is distributed from the DB distribution unit 341 of the biometric information management center 2 by the processing in step S65, the database management unit 306 distributes in step S82. In addition to acquiring the registered player DB 352, the biometric information DB 22 is updated based on the information of the registered player DB 352. More specifically, the database management unit 306 copies and overwrites the newly distributed information of the registered player DB 352 in the biometric information DB 22.

  As a result of the above processing, when one of the game stores 1-1 to 1-n is registered in the unregistered player DB 351 as a non-registered player more than a predetermined number of times, by being additionally registered in the registered player DB 352, Since the database is registered in the biometric information DB 22 of any of the biometric information recognition devices 21 of the game stores 1-1 to 1-n, registration is performed when images are captured by the camera 38 in a plurality of game stores 1 as well. The visit to the store will be notified as a player. In addition, since the registered player DB 352 is updated based on information from each of the game stores 1-1 to 1-n, it is possible to generate customer store records at a plurality of stores, and integrate customer store records. Management is possible.

  In the above, as shown in the process of step S1 in the flowchart of FIG. 16, the face image in the image captured at the timing when the game is started on the game table 36, and the living body that is the same database as the registered player DB 352 When it is determined whether or not the customer has visited the store by comparing the face image in the information DB 22, if it is not a registered player, it is registered in the unregistered player DB 351 as an unregistered player and registered as an unregistered player a predetermined number of times. In this case, the example registered in the registered player DB 352 has been described. However, the timing at which an image is captured is not limited to this. For example, it is detected that a banknote is jammed in the game stand peripheral terminal 37. In the game table 36, the camera 38 is detected at a timing when an abnormal operation such as a ball jam or a medal jam is detected. The in-store camera 41 may be used to capture an image. Thus, for example, when a specific player is registered during an abnormal operation, for example, when an illegal action is performed using the abnormal operation. It becomes possible to detect this.

  Further, in the above, as shown in the process of step S1 in the flowchart of FIG. 16, the example of extracting the face image at the timing when the game is started has been described, but without specially determining the timing of imaging, If a face image is extracted while continuing to image, it may be collated.

  According to the above, it is possible to realize not only customer visit information at each store but also collection of customer visit records at a plurality of stores and integrated management.

  Next, the estimation process will be described with reference to the flowchart of FIG. The processes in steps S101 to S104 and the processes in steps S111 to S113 and S120 in the flowchart of FIG. 19 are the same as the processes in steps S1 to S4 and the processes in steps S21 to S23 and S29 in the flowchart of FIG. Therefore, the description thereof will be omitted.

  That is, when the feature amount is extracted in step S113, the gender estimation unit 307a of the attribute estimation unit 307 in step S114 includes the gender feature pattern of the male face or the female face and the feature supplied from the feature amount extraction unit 321. The amount is compared, and the similarity with the gender feature pattern is scored. More specifically, the gender estimation unit 307a determines the similarity between the nose and the pattern that protrudes immediately above the eyes and then tilts backward for the forehead, and the width of the face for the mouth. Each item, such as the density of the area where the cheeks grow on the cheeks, is scored to obtain a comparison result with the male face feature pattern.For the forehead, the size of the area As for the color depth and the degree of thinness of the upper lip with respect to the lower lip, and for the cheek, each item such as how the cheekbone protrudes and the size of the area is scored to obtain a comparison result with the female face feature pattern.

  In step S115, the gender estimation unit 307a determines whether or not the face image of the feature amount supplied from the feature amount extraction unit 321 is a male by comparing the scores obtained from the male face feature pattern and the female face feature pattern, respectively. For example, if the score obtained from the female face feature pattern is higher, it is determined in step S116 that the gender is not male but female. On the other hand, if the score obtained from the male face feature pattern is higher in step S115, the gender estimation unit 307a determines that the gender is male in step S117.

  In step S118, the age feature estimation unit 307b estimates the age from the feature of the face image. More specifically, the age feature estimation unit 307b has, for example, wrinkles, dullness, or facial expression wrinkles for the forehead, wrinkles, wrinkles in the eyelids, or sagging of the corners of the eyes, and wrinkles for the skin. Assuming that pores, sagging, or dullness, sagging, or area of the jaw, the smaller the age, the younger the age, and the greater the age, the older the age, for example, Estimated age.

  In step S119, the attribute estimation unit 307 supplies the estimation result of the face image of the feature amount supplied from the feature amount extraction unit 321 estimated by the gender estimation unit 307a and the age estimation unit 307b to the display unit 23. Display.

  Through the above process, the face ID is issued to the player, the player inserts a card etc. on which the member ID is recorded, and the face image is used without relying on the visual inspection of the staff. It is possible to acquire information on customers who come to the amusement store based on the gender estimated by the image, the estimated age, and the like, and it is possible to analyze the customer segment that has so far only depended on the visual observation of the staff.

  In the above, the registered player visit monitoring process when the camera 38 or the in-store camera 40 is used has been described with reference to the flowchart of FIG. 16, but the registered player visit monitoring process is performed using the entrance camera 43. It is also possible to execute. However, in that case, the entrance image recognition server 44 is used in place of the image processing unit 39, and the processing of the position recognition server 42 is further added.

  Then, with reference to the flowchart of FIG. 20, the registration player visit monitoring process at the time of using the entrance camera 43 is demonstrated. The processes in steps S161 through S170 and the processes in steps S191 through S193 in the flowchart of FIG. 20 are the same as the processes in steps S21 through S30 and steps S41 through S43 in the flowchart of FIG. Shall be omitted.

  In step S121, the arithmetic processing unit 252 of the entrance stereo image sensor 41 determines whether or not an image for one frame is captured by the cameras 251-1 and 251-2, and an image for one frame is captured. The process is repeated until it is determined. For example, in step S121, when an image for one frame is captured by the cameras 251-1 and 251-2 and supplied to the arithmetic processing unit 252, the process proceeds to step S122.

  In step S122, the arithmetic processing unit 252 calculates the position information of the subject in the three-dimensional space in the image by trigonometry based on the two images respectively supplied from the cameras 251-1 and 251-2, The two images are supplied to the position recognition server 42.

  In step S <b> 123, the person detection unit 271 of the position recognition server 42 determines whether a person is captured based on the image supplied from the entrance stereo image sensor 41. More specifically, the human detection unit 271 determines the presence or absence of an area that includes elements that can be distinguished by color schemes such as eyes, nose, mouth, ears, chin, etc., constituting a human face image in the supplied image. To do.

  If it is determined in step S123 that there is no region including the elements constituting the human face image, the process returns to step S121. On the other hand, if it is determined in step S123 that there is a region including an element constituting the person's face image, in step S124, the person detection unit 271 performs the individual region including the element constituting the person's face image. The person ID is added to the person position information generation unit 272 together with information indicating a region where the face image of the person exists in the image.

  In step S125, the person position information generation unit 272 executes a person position information generation process to generate person position information.

  Here, the person position information generation processing will be described with reference to the flowchart of FIG.

  In step S <b> 211, the person position information generation unit 272 is an unprocessed piece of information that is supplied from the person detection unit 271 and that indicates individual areas including elements constituting the face image of the person to which the person ID is added. The person ID is extracted and recorded as person position information in the memory 274.

  In step S212, the person position information generation unit 272 controls the coordinate conversion unit 272a to set the three-dimensional position information corresponding to the extracted person ID to the position corresponding to the entrance stereo image sensor 41 that generated the position information. The information is converted into two-dimensional position information on the image picked up by the entrance camera 43 that has been set, and recorded as person position information in an area corresponding to the person ID recorded in the memory 274 in step S213.

  More specifically, for example, when an image as shown in FIG. 22 is supplied from the entrance stereo image sensor 41, the person position information generation unit 272 controls the coordinate conversion unit 272a, and the number shown in FIG. Three-dimensional coordinates (X1, Y1, Z1) for the area indicated by 1, coordinates of the area indicated by number 2 (X2, Y2, Z2), coordinates of the area indicated by number 3 (X3, Y3, Z3) Are captured by an entrance camera 43 that captures substantially the same image as the image captured by the camera 251-1 or 251-2 of the entrance stereo image sensor 41, for example, 2 on the image shown in FIG. Convert to dimensional coordinates. Therefore, the person position information generation unit 272 obtains position coordinates on the images of the face images of the players A, B, and C in the image that will be captured by the entrance camera 43.

  That is, since the image captured by the entrance camera 43 is an image captured by the cameras 251-1 and 251-2 of the entrance stereo image sensor 41, it is a two-dimensional image in the space captured from the angle of the entrance camera 43. By associating the face image with the position information with the person ID so that the position information matches the three-dimensional position information detected by the entrance stereo image sensor 41, for example, as shown in FIG. In addition, the player of face image B is associated with the position of number 1, the player of face image A is associated with the position of number 2, and the player of face image C is associated with the position of number 3. . At this time, the coordinate conversion unit 272a captures the three-dimensional coordinates (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3) of numbers 1 to 3 by the entrance camera 43, respectively. The coordinates on the image are converted to (x1, y1), (x2, y2), (x3, y3). Based on the conversion result, the person position information generation unit 272 records the two-dimensional coordinate position on the player image corresponding to the person ID as person position information, as shown in FIG. The coordinates are described as being the center position indicating each area, but may be coordinates of another position specifying the area.

  In FIG. 25, for each frame from the first frame to the third frame (Flame No. 1 to 3), a person ID column, a detection position column, and an entrance detection Flag column are provided from the right. In the third frame, 0001, 0002,... Are recorded as person IDs from the top. Further, (x1, y1), (x2, y2),... Are recorded as detection positions from the top. Furthermore, “Yes” and “No” are recorded from the top as the entrance detection flag. Therefore, in the case of FIG. 25, the face image of the person ID = 0001 exists on the two-dimensional coordinates (x1, y1) on the image captured by the entrance camera 43, and the entrance detection flag is “Yes”. It is shown. In addition, it is indicated that the face image with the person ID = 0002 exists on the two-dimensional coordinates (x2, y2) on the image captured by the entrance camera 43, and the entrance detection flag is “No”. . Details of the inlet detection flag will be described later.

  In step S214, the person position information generation unit 272 controls the entrance detection determination unit 272b to determine whether the face image is detected in the area at the entrance 112 based on the two-dimensional position information. That is, the two-dimensional coordinates handled here are converted from the position of the face image in the three-dimensional space measured by the entrance stereo image sensor 41 to the coordinates of the two-dimensional space on the image captured by the entrance camera 43. Therefore, it is possible to detect whether or not it is a face image at the entrance / exit 112 by comparing the coordinates in the three-dimensional space of the entrance / exit 112 with those converted into coordinates in the two-dimensional space by the same method. It is. Therefore, the person position information generation unit 272 controls the entrance detection determination unit 272b to determine whether the face image is detected at the entrance 112 based on the position information of the face image in the two-dimensional space.

  In step S214, for example, when it is determined that the detection is performed at the entrance 112, the person position information generation unit 272 sets the entrance detection flag to Yes in step S215. In step S214, if no face image is detected at the entrance 112, the person position information generation unit 272 sets the entrance detection flag to No in step S216.

  In step S217, the person position information generation unit 272 determines whether or not an unprocessed person ID exists. If there is an unprocessed person ID, the process returns to step S211. That is, until it is determined that all the person IDs have been processed, the processes in steps S211 to S217 are individually repeated for the face image in which the person ID is set, and registered in the person position information.

  In step S217, when the face images for which all the person IDs are set are registered in the person position information, the person position information generation unit 272 indicates that the person position information generation process has been completed, The process ends when the information generation unit 273 is notified.

  With the above processing, the position information is registered by the coordinates in the image captured by the entrance camera 43 for the person detected by the entrance stereo image sensor 41, that is, the person in the image captured by the entrance camera 43. Person position information is generated.

  Now, the description returns to the flowchart of FIG.

  When the person position information is generated in step S125, in step S126, the entrance passer position information generation unit 273 executes entrance passer position information generation processing to generate entrance passer position information.

  Here, the entrance passer position information generation processing will be described with reference to the flowchart of FIG.

  In step S231, when the entrance passer position information generation unit 273 receives a notification from the person position information generation unit 272 indicating that the person position information has been generated in the memory 274 by the person position information processing, it accesses the memory 274, Unprocessed person ID information is extracted from the information registered in the person position information.

  In step S232, the entrance passer position information generation unit 273 controls the passage determination unit 273a to determine whether or not the extracted person ID is registered in the person position information of the immediately preceding frame. Determine if it is an ID. For example, if it is determined in step S232 that it is the first detection, that is, it is not registered in the person position information in the immediately preceding frame, in step S236, the entrance passer position information generation unit 273 determines the passer determination unit 273a. To determine whether the person position information in the current frame has been detected at the entrance, that is, whether the entrance detection Flag is Yes. For example, if the entrance detection flag in the person position information of the current frame of the extracted person ID is Yes, it is considered that the person has passed the entrance, and the process proceeds to step S234, where the extracted person ID Information is registered in the entrance passer position information as an entrance passer.

  On the other hand, in step S236, if it is not an entrance passer, that is, it is the first detection and the entrance detection flag of the person position information in the current frame is not Yes, it is regarded as not an entrance passer, and the process The process proceeds to S235.

  In step S236, for example, when the extracted person ID is 0001, the entrance detection Flag is Yes, so it is determined that the person has passed the entrance. In step S234, the entrance passer position information generation unit 273, for example, As shown in FIG. 27, the extracted person ID and detection position information are registered as entrance passer position information. That is, in this case, since the person ID is 0001, (x1, y1) is registered as the detected position read from the person position information as the passer position information of the third frame (Flame No. 3). The On the other hand, if it is determined in step S236 that the person is not an entrance passer, the process of step S234 is skipped, and the extracted person ID information is not registered in the entrance passer position information.

  In step S235, the entrance passer position information generation unit 273 determines whether or not an unprocessed person ID exists in the person position information. If there is an unprocessed person ID, the process returns to step S231. If there is no unprocessed person ID in the person position information and the process is completed for all person IDs, the process ends.

  On the other hand, if it is determined in step S232 that, for example, the person position information of the previous frame exists, that is, it is not the first detected person ID, the entrance passer position information generation unit 273 determines whether or not the passage determination is made in step S233. The unit 273a is controlled to determine whether or not the person having the extracted person ID has created the entrance passer position information in the last few frames, and determine whether or not the entrance passer position information has been created. That is, the entrance passer position information is newly registered for a certain person ID when it is determined in step S233 that the person has been detected at the entrance. It is determined as an entrance passer.

  Only the position information of the person deemed to have passed through the entrance / exit 112 is registered in the entrance passer position information. That is, for example, as shown in the left part of FIG. 28, a three-dimensional coordinate position considered to pass through the entrance 112 in the image captured by the entrance stereo image sensor 41 is set in the region Z1. The image captured by the corresponding entrance camera 43 is expressed as shown in the right part of FIG.

  For example, as shown in the upper left part of FIG. 29, when the person H1 is captured by the cameras 251-1 and 251-2 of the entrance stereo image sensor 41 as being present in the area Z1, the entrance stereo image sensor 41, since the person H1 is considered to exist at the entrance 112, the entrance detection flag is registered in the state of Yes when registered in the person position information. At this time, the entrance camera 43 captures an image as shown in the uppermost right part of FIG. In FIG. 29, as in FIG. 28, an image captured by the entrance stereo image sensor 41 is shown on the left, and an image captured by the entrance camera 43 is shown on the right, respectively. , And the image that is continuous one frame at a time in the lowest order is displayed.

  In this case, since it is the first detection in the entrance passer position information processing, it is regarded as the first detection in step S232, but since it exists at the entrance 112 in the current frame, Since the entrance detection flag is registered as Yes, it is regarded as an entrance passer and is registered in the entrance passer position information in the process of step S234.

  In addition, in the case of the persons H2 and H3 shown in the middle left part and the lowermost part of FIG. 29, the entrance stereo image sensor 41 has created the entrance passer position information in the uppermost process, so in step S233. 29, even when the entrance camera 43 captures images such as those shown in the middle right and bottom of FIG. 29, the entrance passer position information is generated in step S234. The unit 273 is regarded as an entrance passer and registers in the entrance passer position information. That is, as shown in the right part of FIG. 29, entrance passer position information is created for all of the persons H1 to H3 that have passed through the entrance 112 and are continuously being imaged.

  On the other hand, as shown in the left part and the right part of FIG. 30, after passing through the entrance / exit 112, that is, after entering the store, the person H11 does not pass through the entrance / exit 112 again, and the entrance stereo image sensor 41 and the entrance camera. In the case of passing through the 43 imaging areas, it is determined in step S232 that the detection is the first detection. However, in step S236, the entrance detection flag of the person position information is not set to Yes. It is determined that there is no entry and is not registered in the entrance passer position information.

  As a result, according to the above processing, only when the player visits the store through the entrance 112, it is registered in the entrance passer position information.

  Now, the description returns to the flowchart of FIG.

  When the entrance passer position information generation process is executed in step S126 and the entrance passer position information is generated, in step S127, the entrance passer position information generation unit 273 converts the generated entrance passer position information into the entrance image. Supply to the processing unit 44.

  In step S <b> 141, the entrance camera 43 captures an image around the entrance 112 and supplies the captured image to the entrance image processing unit 44. The image acquisition unit 201 of the entrance image processing unit 44 acquires the supplied image and supplies it to the face detection frame setting unit 231. Here, the image captured by the entrance camera 43 is an image captured at the same timing as the image captured by the entrance stereo image sensor 41 in the immediately preceding step S121.

  In step S142, the face detection frame setting unit 231 acquires entrance passer position information supplied from the position recognition server 42 by the process of step S127.

  In step S143, the face detection frame setting unit 231 determines whether or not an entrance passer is registered based on the entrance passer position information, and if not, the process returns to step S141. If there is an entrance passer in step S143, the supplied image is adjusted based on the entrance passer position information in step S144. That is, based on the entrance passer position information, the face detection frame setting unit 231 is indicated by frames F1 to F3 according to the detection positions included in the entrance passer position information, for example, as shown in the right part of FIG. The frame image is adjusted so as to leave a region including such a face image, and is supplied to the face image extraction unit 202.

  In step S145, the face detection unit 211 of the face image extraction unit 202 detects the player's face image from the supplied image, and supplies the detection result to the face image cutout unit 212. The face image clipping unit 212 extracts the detected face image by clipping it from the image, and supplies the extracted face image to the transmission unit 203.

  In step S146, the transmission unit 203 determines whether or not a face image has been extracted from the face image extraction unit 202. If no face image has been extracted, the process returns to step S141. On the other hand, when it is determined in step S146 that a face image has been extracted, in step S147, the transmission unit 203 transmits the face image supplied from the face image extraction unit 202 to the biological information recognition apparatus 21. At this time, the transmission unit 203 adds information such as a camera ID for identifying each of the entrance cameras 43 and information on transmission time to the face image and transmits the information to the biometric information recognition apparatus 21.

  Of the face images captured by the entrance camera 43 by the entrance stereo image sensor 41 and the position recognition server 42 by the above processing, the location information of the player who has entered the store through the entrance 112 is used as entrance passer location information. Since it is registered, the entrance image processing unit 44 transmits only the face image of the person registered as entrance passer position information to the biometric information recognition device 21.

  For this reason, only the face image that passes through the entrance 112 and comes to the store is subjected to the face image recognition processing in the biometric information recognition device 21.

  Even in the above-described estimation process, it is possible to process only the face image of the person who has passed through the entrance / exit 112 among the face images captured by the entrance camera 43.

  In addition, about the estimation process using the entrance camera 43, the process of FIG.19 S111 thru | or S120 is performed after the process of FIG.19 S147, About each process, FIG. Since it is the same as that described with reference to the flowchart of FIG. 20, the description thereof will be omitted.

  As a result, it becomes possible to reduce the processing load in the biometric information recognition device 21 and, at the same time, it is possible to improve the processing speed by omitting unnecessary processing.

  Incidentally, the series of monitoring processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.

  FIG. 31 shows a configuration example of a general-purpose personal computer. This personal computer includes a CPU (Central Processing Unit) 501. An input / output interface 505 is connected to the CPU 501 via the bus 504. A ROM (Read Only Memory) 502 and a RAM (Random Access Memory) 503 are connected to the bus 504.

  The input / output interface 505 includes an input unit 506 including an input device such as a keyboard and a mouse for a user to input an operation command, an output unit 507 for outputting a processing operation screen and an image of the processing result to a display device, a program and various data. A storage unit 508 including a hard disk drive to be stored, a LAN (Local Area Network) adapter, and the like are connected to a communication unit 509 that executes communication processing via a network represented by the Internet. Also, a magnetic disk (including a flexible disk), an optical disk (including a CD-ROM (compact disc-read only memory), a DVD (digital versatile disc)), a magneto-optical disk (including an MD (mini disc)), or a semiconductor A drive 510 for reading / writing data from / to a removable medium 511 such as a memory is connected.

  The CPU 501 is read from a program stored in the ROM 502 or a removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, installed in the storage unit 508, and loaded from the storage unit 508 to the RAM 503. Various processes are executed according to the program. The RAM 503 also appropriately stores data necessary for the CPU 501 to execute various processes.

  In this specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series in the order described, but of course, it is not necessarily performed in time series. Or the process performed separately is included.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

It is a figure which shows the structure of one Embodiment of the monitoring system to which this invention is applied. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the camera of FIG. It is a figure which shows the example of installation of the entrance camera store camera of FIG. 1, and an entrance stereo image sensor. It is a figure explaining the structural example of the image processing unit of FIG. It is a figure explaining the structural example of the entrance image processing unit of FIG. It is a figure explaining the structural example of the entrance stereo image sensor of FIG. It is a figure explaining the structural example of the position recognition server of FIG. It is a figure explaining the structural example of the biometric information recognition apparatus of FIG. It is a figure explaining the structural example of the biometric information management center and biometric information management DB of FIG. It is a figure explaining the structural example of the portable terminal of FIG. It is a flowchart explaining the registration player visit monitoring process using the camera of FIG. It is a figure explaining the example of a display of an alerting | reporting image. It is a flowchart explaining a registration player DB update process. It is a flowchart explaining an estimation process. It is a flowchart explaining the registration player visit monitoring process using the entrance camera and entrance stereo image sensor of FIG. It is a flowchart explaining a person position information generation process. It is a figure explaining person position information generation processing. It is a figure explaining person position information generation processing. It is a figure explaining person position information generation processing. It is a figure explaining person position information. It is a flowchart explaining entrance passing person position information generation processing. It is a figure explaining entrance passage person position information. It is a figure explaining entrance passing person position information generation processing. It is a figure explaining entrance passing person position information generation processing. It is a figure explaining entrance passing person position information generation processing. It is a figure explaining a medium.

Explanation of symbols

1, 1-1 to 1-n amusement store 2 biometric information management center 3 biometric information management database 4 third party amusement store management center 5 third party amusement store management database 6 biometric information management bus 7 third party amusement store management bus 8, 9 Public communication network 21 Biometric information recognition device 22 Biometric information database 24 Game store management device 26 Game table management database 27 Medium lending management device 29 Medium lending management database 30 Amusement store management information bus 31 Biometric information bus 33 Settling machine 34 Lending machine 35 Counter 36,36-1 thru 36-m Game stand 37,37-1 thru 37-m Game stand peripheral terminal 38,38-1 thru 38-m Camera 39,39-1 thru 39- (m + p) Image processing unit 40, 40-1 to 40-p In-store camera 41, 41-1 to 41-q Entrance stereo image sensor 42 42-1 to 42-q position recognition server 43,43-1 to 43-q entrance camera 44,44-1 to 44-q inlet image processing unit

Claims (14)

Accumulating means for accumulating pre-registered registrant face images;
Image acquisition means for acquiring images;
Position acquisition means for acquiring the position of the person to be verified in the image;
Whether the person to be collated acquired by the position acquisition means has passed through a predetermined area, or is a person who has passed through the predetermined area continuously after passing through the predetermined area Determining means for determining whether or not;
When it is determined by the determination result of the determination means that the verification target person is a person who has passed the predetermined area, only the verification target person determined to be the predetermined area passer of the image is left. Image adjusting means for adjusting the image;
Face image extraction means for extracting the face image of the person to be collated determined as a person who has passed through the predetermined area from the image adjusted by the image adjustment means;
Collation means for calculating and collating the similarity between the face image of the person to be collated extracted by the face image extraction means and the face image of the registrant accumulated in the accumulation means;
A similarity determination unit that determines whether or not the face image of the person to be collated is a face image of the registrant by comparing the similarity that is the collation result of the collation unit and a predetermined threshold;
And a notifying means for notifying that the verification target person is the registrant when the similarity determination means determines that the verification target person is the registrant. A feature amount extraction means for extracting a feature amount from the face image of the person to be collated determined as the person who has passed the predetermined region,
The collation means uses feature amounts of the face image of the person to be collated determined as a person who has passed through the predetermined area, extracted by the face image extraction means, and the face image of the registrant accumulated in the accumulation means. The monitoring apparatus according to claim 1, wherein the similarity is calculated and the face image of the person to be collated acquired by the acquisition unit is collated with the face image of the registrant accumulated in the accumulation unit. The monitoring apparatus according to claim 1, wherein the position acquisition unit acquires a position of a person to be verified in the image by a stereo image sensor. Further including person position information generating means for generating person position information based on the position of the person to be collated in the image acquired by the position acquiring means;
The determination means is based on the person position information, and the person to be collated acquired by the position acquisition means passes through a predetermined area or continuously after passing through the predetermined area. The monitoring apparatus according to claim 1, wherein it is determined whether or not the person has passed through a predetermined area existing in the image. The unregistered person registration means for registering the face image of the verification target person as an unregistered person when the similarity determination means determines that the verification target person is not the registrant. Monitoring device. The storage means stores a registered registrant's face image as a database,
The monitoring apparatus according to claim 5, wherein a face image of an unregistered person who has been registered a predetermined number of times or more by the unregistered person registration unit is registered in the database as the face image of the registrant. An image acquisition step of acquiring an image;
A position acquisition step of acquiring the position of the person to be verified in the image;
The person to be collated acquired by the processing of the position acquisition step is a person who has passed through a predetermined area, or has passed through the predetermined area, and has continuously passed through the predetermined area as it is in the image. A determination step of determining whether or not there is;
When it is determined by the determination result in the process of the determination step that the person to be collated is a person who has passed the predetermined area, only the person to be collated that has been determined to be a person who has passed the predetermined area in the image. An image adjustment step for adjusting the image to leave,
A face image extraction step of extracting a face image of the person to be collated determined as a person who has passed through the predetermined area from an image adjusted by the process of the image adjustment step;
A collation step of calculating and collating the similarity between the face image of the person to be collated extracted by the processing of the face image extraction step and the accumulated face image of the registrant;
A similarity determination step of determining whether or not the face image of the person to be verified is a registrant's face image by comparing the similarity that is the result of the verification in the process of the verification step with a predetermined threshold;
And a notification step of notifying that the person to be collated is the registrant when the person to be collated is determined to be the registrant by the processing of the similarity determination step. An image acquisition step of acquiring an image;
A position acquisition step of acquiring the position of the person to be verified in the image;
The person to be collated acquired by the processing of the position acquisition step is a person who has passed through a predetermined area, or has passed through the predetermined area, and has continuously passed through the predetermined area as it is in the image. A determination step of determining whether or not there is;
When it is determined by the determination result in the process of the determination step that the person to be collated is a person who has passed the predetermined area, only the person to be collated that has been determined to be a person who has passed the predetermined area in the image. An image adjustment step for adjusting the image to leave,
A face image extraction step of extracting a face image of the person to be collated determined as a person who has passed through the predetermined area from an image adjusted by the process of the image adjustment step;
A collation step of calculating and collating the similarity between the face image of the person to be collated extracted by the processing of the face image extraction step and the accumulated face image of the registrant;
A similarity determination step of determining whether or not the face image of the person to be verified is a registrant's face image by comparing the similarity that is the result of the verification in the process of the verification step with a predetermined threshold;
A program for causing a computer to execute a process including a notification step of notifying that the person to be collated is the registrant when the person to be collated is determined to be the registrant by the process of the similarity determination step. . Accumulating means for accumulating pre-registered registrant face images;
Image acquisition means for acquiring images;
Position acquisition means for acquiring the position of the person to be verified in the image;
Whether the person to be collated acquired by the position acquisition means has passed through a predetermined area, or is a person who has passed through the predetermined area continuously after passing through the predetermined area Determining means for determining whether or not;
When it is determined by the determination result of the determination means that the verification target person is a person who has passed the predetermined area, only the verification target person determined to be the predetermined area passer of the image is left. Image adjusting means for adjusting the image;
Face image extraction means for extracting the face image of the person to be collated determined as a person who has passed through the predetermined area from the image adjusted by the image adjustment means;
Collation means for calculating and collating the similarity between the face image of the person to be collated extracted by the face image extraction means and the face image of the registrant accumulated in the accumulation means;
A similarity determination unit that determines whether or not the face image of the person to be collated is a face image of the registrant by comparing the similarity that is the collation result of the collation unit and a predetermined threshold;
A registrant verification device comprising: notification means for notifying that the verification target person is the registrant when the similarity determination unit determines that the verification target person is the registrant. Image acquisition means for acquiring images;
Position acquisition means for acquiring the position of the person to be verified in the image;
Whether the person to be collated acquired by the position acquisition means has passed through a predetermined area, or is a person who has passed through the predetermined area continuously after passing through the predetermined area Determining means for determining whether or not;
When it is determined by the determination result of the determination means that the verification target person is a person who has passed the predetermined area, only the verification target person determined to be the predetermined area passer of the image is left. Image adjusting means for adjusting the image;
Face image extraction means for extracting the face image of the person to be collated determined as a person who has passed through the predetermined area from the image adjusted by the image adjustment means;
An attribute estimation apparatus comprising: an estimation unit configured to estimate an attribute of the collation target person based on the face image of the collation target person extracted by the face image extraction unit. A feature amount extraction means for extracting a feature amount from the face image of the person to be collated determined as the person who has passed the predetermined region,
The attribute estimation unit uses the feature amount of the face image of the person to be collated determined as the person who has passed the predetermined area, which is extracted by the face image extraction unit, and the attribute of the person to be collated acquired by the acquisition unit The attribute estimation device according to claim 10. The attribute estimation device according to claim 10, wherein the attribute estimation unit estimates gender or age as the attribute of the person to be collated. An image acquisition step of acquiring an image;
A position acquisition step of acquiring the position of the person to be verified in the image;
The person to be collated acquired by the processing of the position acquisition step is a person who has passed through a predetermined area, or has passed through the predetermined area, and has continuously passed through the predetermined area as it is in the image. A determination step of determining whether or not there is;
When it is determined by the determination result in the process of the determination step that the person to be collated is a person who has passed the predetermined area, only the person to be collated that has been determined to be a person who has passed the predetermined area in the image. An image adjustment step for adjusting the image to leave,
A face image extraction step of extracting a face image of the person to be collated determined as a person who has passed through the predetermined area from an image adjusted by the process of the image adjustment step;
An attribute estimation method comprising: an estimation step of estimating an attribute of the collation target person based on the face image of the collation target person extracted by the processing of the face image extraction step. An image acquisition step of acquiring an image;
A position acquisition step of acquiring the position of the person to be verified in the image;
The person to be collated acquired by the processing of the position acquisition step is a person who has passed through a predetermined area, or has passed through the predetermined area, and has continuously passed through the predetermined area as it is in the image. A determination step of determining whether or not there is;
When it is determined by the determination result in the process of the determination step that the person to be collated is a person who has passed the predetermined area, only the person to be collated that has been determined to be a person who has passed the predetermined area in the image. An image adjustment step for adjusting the image to leave,
A face image extraction step of extracting a face image of the person to be collated determined as a person who has passed through the predetermined area from an image adjusted by the process of the image adjustment step;
A program that causes a computer to execute a process including an estimation step of estimating an attribute of the person to be collated based on the face image of the person to be collated extracted by the process of the face image extracting step.

Images