JP4797517B2 - Monitoring device and method, and program - Google Patents

Description

  The present invention relates to a monitoring device, a monitoring method, and a program, and more particularly, to a monitoring device, a method, and a program that reduce false detection of unauthorized registrants and improve the detection rate.

  Amusement stores represented by pachinko or pachislot stores are popular.

  In a game store, a player plays a favorite game machine by himself among a plurality of game machines arranged. However, since the game machines are not evenly set with so-called jackpot probabilities, the player searches for a game machine that is likely to have a jackpot as much as possible.

  However, crimes by fraudulent players who deliberately give a device to a gaming machine and illegally generate a jackpot are unending.

  Regarding countermeasures against unauthorized players, a surveillance camera is arranged at the entrance and exit of a gaming store, and a facial image of a player's face image captured by the surveillance camera and a pre-registered unauthorized player's facial image are detected using facial recognition technology. It has been proposed to search for and issue a warning when an unauthorized player visits a store (see, for example, Patent Document 1).

  However, because the face recognition rate by face recognition technology is not high compared to other biometric authentication technologies that use vein patterns, retinal patterns, etc., misidentification (recognizing someone other than the person or the person other than the person) Misrecognition, such as recognizing) frequently occurs.

  Therefore, in order to improve the convenience for misidentification at the time of entering and leaving the room, for example, a technique has been proposed in which the setting of the threshold value for determination is changed according to the usage frequency of the registrant and the time of entering and leaving the room. (For example, refer to Patent Document 2).

JP 2001-178958 A JP 2002-183734 A

  However, in the above-described method, when other registrants or non-registrants enter or leave the room, there is a risk of being misidentified (identified as a person other than the principal) by a registrant who is frequently used. Moreover, although the above-mentioned method can be used for managing the entrance / exit, it is not possible to assume in advance the usage frequency or the usage time zone of a specific registrant for the purpose of monitoring. So it could not be used. Furthermore, when used for monitoring purposes in public facilities, since there are many persons to be identified, it was not possible to easily set the threshold value lower in order to avoid misidentification of non-registered persons.

  The present invention has been made in view of such a situation, and in particular, when the degree of similarity with a specific registrant is increased multiple times, it is considered that the registrant is likely to be detected. By re-setting the threshold value for recognizing the registrant, false positives are reduced.

The monitoring device according to one aspect of the present invention includes a storage unit that stores a face image of an unauthorized registrant, an acquisition unit that acquires a face image of the arrival person, the face image of the arrival person, and a storage unit that stores the face image. Similarity calculation means for calculating the similarity with the face image of the unauthorized registrant, and when the similarity calculated by the similarity calculation means is higher than a predetermined threshold , the similarity with the arrival person is calculated The determination means for determining that an unauthorized registrant has arrived, and the determination result of the determination means, when the similarity is lower than the threshold , the similarity is calculated as the similarity with the arrival. A storage unit that stores the face image of the registrant in association with a frequency detection unit that detects a frequency at which a face image of an unauthorized registrant whose similarity with the arrival person is lower than the threshold is stored in the storage unit; , fraud registration stored by the storage means Of the face images, the frequency detected by the frequency detecting means and a threshold updating means for updating lower the threshold for many incorrect registration's face image than a predetermined number of times.

The acquisition means acquires a camera ID for identifying a camera that has captured the arrival person's face image together with the arrival person's face image, and the storage means determines the similarity in the determination result of the determination means. Is lower than the threshold , the similarity is stored in association with the face image and camera ID of the unauthorized registrant whose similarity with the arrival is calculated, and the frequency detection means stores the storage The frequency with which a face image captured by a camera with the same camera ID is acquired from the face images of unauthorized registrants whose similarity to the arrival person stored in the means is lower than the threshold value is detected. be able to.

The said threshold updating means, the frequency detected by the frequency detecting means, when more than a predetermined number of times within a predetermined time, it is possible to so as to update lower the threshold.

The said threshold updating means, the frequency detected by the frequency detecting means, when more than a predetermined number of times, can be made to be updated lower weighting the threshold.

The said threshold updating means, the frequency detected by the frequency detecting means, when more than a predetermined number of times, can be made to be updated from the threshold by a predetermined value lower.

The monitoring method according to one aspect of the present invention includes an accumulating step of accumulating a face image of an unauthorized registrant, an acquiring step of obtaining an arrival person's face image, an arrival person's face image, and an accumulating step. A similarity calculation step for calculating the similarity with the face image of the unauthorized registrant, and the similarity with the arrival when the similarity calculated by the processing of the similarity calculation step is higher than a predetermined threshold. In the determination step of determining that an unauthorized registrant whose degree has been calculated has arrived and the determination result determined by the processing of the determination step, when the similarity is lower than the threshold , the similarity is determined as the arrival wherein a storage step of similarity is stored in correspondence with the calculated fraud registration's face image, incorrect registration's face image similarity is lower than the threshold value of the incoming user of the storage step process with A frequency detecting step of detecting a frequency to be more storage, of fraud registration's face image stored by the processing of the storage step, often incorrect registration than the detected frequency of a predetermined number of times by the process of the frequency detecting step A threshold update step of updating the threshold value of the person's face image to be low .

The program according to one aspect of the present invention is accumulated in the accumulation step of accumulating the face image of the unauthorized registrant, the acquisition step of obtaining the face image of the arrival person, the face image of the arrival person, and the process of the accumulation step. A similarity calculation step for calculating the similarity with the face image of the unauthorized registrant, and the similarity with the arrival when the similarity calculated by the processing of the similarity calculation step is higher than a predetermined threshold. When the highest similarity is lower than the threshold value in the determination step for determining that an unauthorized registrant for which has been calculated has arrived and the determination result determined by the processing of the determination step, the similarity is determined to be the arrival who said a storage step of similarity is stored in correspondence with the calculated fraud registration's face image, the incoming person and the similarity in the facial image of the lower invalid registrant than the threshold value the memory stearyl the A frequency detecting step of detecting a frequency to be stored by the processing of flops, the out of illegal subscribers of face images stored by the processing of the storage step, than the detected frequency of a predetermined number of times by the process of the frequency detecting step to execute processing including the threshold updating step of low update the threshold for many incorrect registration's face image to the computer.

In one aspect of the present invention, the face image of the unauthorized registrant is accumulated, the face image of the arriving person is acquired, and the similarity between the face image of the arriving person and the accumulated face image of the unauthorized registrant is calculated. When the calculated similarity is higher than a predetermined threshold , it is determined that an unauthorized registrant whose calculated similarity with the arrival has arrived. In the determination result, the similarity is greater than the threshold . The similarity is stored in association with the face image of the unauthorized registrant whose similarity with the arrival is calculated , and the similarity with the arrival is lower than the threshold . The frequency with which a face image is stored is detected, and among the stored face images of unauthorized registrants, the detected frequency is higher than a predetermined number of times and the similarity between the face image of an unauthorized registrant and the arrival person is The threshold is updated low .

In the monitoring device according to one aspect of the present invention, the storage unit that stores the unauthorized registrant's face image is, for example, an unauthorized player registration database, and the acquisition unit that acquires the arriving person's facial image is, for example, a face The similarity calculation means for calculating the similarity between the face image of the arrival person and the face image of the unauthorized registrant stored in the storage means is, for example, a similarity calculation section, When the similarity calculated by the similarity calculation means is higher than a predetermined threshold, the determination means that determines that an unauthorized registrant whose similarity with the arrival has been calculated is, for example, a threshold table A determination unit that compares the threshold and the similarity with each other to determine whether or not an illegal player has visited the store. When the similarity is lower than the threshold in the determination result of the determination unit, the similarity is determined as the arrival. And the similarity is calculated Fraud The registrant face memory means for storing in association with the image, for example, a threshold below result storage control unit, the arrival's and the similarity is low fraud registration's face image is the storage than the threshold value The frequency detection means for detecting the frequency stored in the means is, for example, a detection frequency analysis unit, and the frequency detected by the frequency detection means among the face images of unauthorized registrants stored in the storage means. The threshold value updating means for updating the threshold value for face images of unauthorized registrants more than a predetermined number of times is, for example, a threshold value changing unit.

That is, when the degree of similarity is lower than a predetermined threshold, the less than threshold result storage control unit stores the degree of similarity in association with the face image of the unauthorized registrant who has calculated the degree of similarity with the arrival person, and detects it. frequency analysis unit, the arrival's illegal registrant similarity is lower than the threshold of the face image to detect a frequency to be stored, the threshold changing unit, the detected frequency is incorrect registrant greater than a predetermined number of times Since the threshold value for the similarity between the face image and the arrival person is updated low, if a face image of a person similar to the face image registered as an unauthorized player is detected several times, the fraud by lowering the threshold for the registered face image as a player, online users of registered face images and similar face images as an unauthorized player will be determined as an unauthorized player.

  As a result, among face images of persons registered as fraudulent players, face images that are rarely used for determination are determined using normal threshold values, and an increase in false detection can be suppressed. It becomes possible. On the other hand, for a facial image registered as an unauthorized player that is frequently used for determination, the threshold value is lowered according to the number of detections. Therefore, if a face image is detected exceeding a predetermined number, it is detected as an unauthorized player. It becomes easy to improve the detection rate of unauthorized players.

  According to the present invention, it is possible to improve the detection rate of fraudulent players and reduce misidentification.

  Embodiments of the present invention will be described below. Correspondences between the configuration requirements of the present invention and the embodiments described in the detailed description of the present invention are exemplified as follows. This description is to confirm that the embodiments supporting the present invention are described in the detailed description of the invention. Accordingly, although there are embodiments that are described in the detailed description of the invention but are not described here as embodiments corresponding to the constituent elements of the present invention, It does not mean that the embodiment does not correspond to the configuration requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. Not something to do.

That is, the monitoring device according to one aspect of the present invention includes a storage unit (for example, the unauthorized player registration database 84 in FIG. 7) that stores the face image of an unauthorized registrant, and an acquisition unit (for example, that acquires the face image of an arrival person ). 7, a similarity calculation unit (for example, FIG. 7) that calculates the similarity between the arrival person's facial image and the unauthorized registrant's facial image stored in the storage unit. A similarity calculating unit 83) and a determining unit that determines that an unauthorized registrant whose calculated similarity with the arrival has arrived when the similarity calculated by the similarity calculating unit is higher than a predetermined threshold. (For example, the determination unit 85 in FIG. 7) and the determination result of the determination means, when the similarity is lower than the threshold value , the similarity is calculated as the similarity with the arrival and the unauthorized registration. storage means for storing in association with the person of the face image For example, a threshold smaller than the result of FIG storage control unit 87), the frequency detecting means for detecting the frequency of similarity between the incoming user face image of low fraud registrant than the threshold value stored in said storage means ( for example, the detection frequency analysis unit 89 of FIG. 7), of the illegal registration's face images stored by the storing means, the frequency of the detected by the frequency detection means is larger than a predetermined number of times incorrect registration's face image Threshold updating means (for example, a threshold changing unit 90 in FIG. 7) for updating the threshold with respect to low .

The acquisition unit (for example, the face image acquisition unit 81 in FIG. 7) acquires a camera ID for identifying the camera that captured the arrival person's face image together with the arrival person's face image, and the storage unit (for example, 7, when the similarity is lower than the threshold in the determination result of the determination unit, the similarity with the arrival is calculated as the similarity. The fraudulent registrant 's face image and camera ID are stored in association with each other, and the frequency detection means (for example, the detection frequency analysis unit 89 in FIG. 7) stores the similarity to the arrival person stored in the storage means. Among the face images of unauthorized registrants whose values are lower than the threshold, the frequency with which face images captured by cameras with the same camera ID are acquired can be detected.

The threshold updating unit (e.g., the threshold changing unit 90 of FIG. 7), the frequency of the detected by the frequency detection means, if more than a predetermined number of times within a predetermined time, be so as to update lower the threshold Can do.

The threshold updating unit (e.g., the threshold changing unit 90 of FIG. 7), the frequency detected by the frequency detecting means, when more than a predetermined number of times, be adapted to update lower weighting said threshold it can.

The threshold updating unit (e.g., the threshold changing unit 90 of FIG. 7), the frequency detected by the frequency detecting means, when more than a predetermined number of times, be adapted to update from the threshold by a predetermined value lower it can.

The monitoring method according to one aspect of the present invention includes an accumulating step of accumulating an unauthorized registrant's face image, an acquisition step of acquiring an arrival person's face image (for example, the process of step S41 in FIG. 9), and the arrival person's face image. A similarity calculation step (for example, the process in step S81 in FIG. 14) for calculating the similarity between the face image and the face image of the unauthorized registrant accumulated in the process in the accumulation step, and the process in the similarity calculation step A determination step (for example, the process of step S82 in FIG. 14) for determining that an unauthorized registrant whose similarity with the arrival has been calculated has arrived when the similarity calculated by the above is higher than a predetermined threshold; wherein the result of determination by the process of the determination step, the similarity is, when lower than the threshold value, versus the similarity incorrect registration's face image in which the similarity is calculated between the incoming party With storing step of storing (for example, step S84 in FIG. 14) and, the frequency of incorrect registration's face image similarity is lower than the threshold value of the incoming user is stored by the processing of the storage step frequency detecting step of detecting (for example, step S85 in FIG. 14) and, among the face images of illegal subscribers stored by processing of the storage step, the detected frequency is a predetermined number of times by the process of the frequency detecting step threshold update step of updating lowered the threshold for many incorrect registration's face image than (for example, step S88 in FIG. 14) and a.

  FIG. 1 is a diagram showing a configuration of an embodiment of a game shop monitoring system according to the present invention.

  The entrance stereo image sensors 1-1 to 1-3 and the island stereo image sensors 2-1 to 2-17 (FIG. 4) are composed of two cameras having different arrangement positions. An image with different angles obtained by imaging the same area with different cameras is processed using parallax, the spatial position of the imaged subject is detected, and supplied to the human tracking server 5. Note that the entrance stereo image sensors 1-1 to 1-3 and the island stereo image sensors 2-1 to 2-17 are simply different from the entrance stereo image sensor 1 and the island stereo image sensor 2 if it is not necessary to distinguish between them. Other configurations are also referred to in the same manner.

  More specifically, the entrance stereo image sensor 1 is composed of cameras 1a and 1b as shown in FIG. 2, and images the same area and outputs the same area to the arithmetic processing unit 1c. The arithmetic processing unit 1c performs arithmetic processing on images with different angles supplied from the cameras 1a and 1b using parallax (by arithmetic processing using a so-called trigonometric method), and subjects in the captured images Is obtained as a three-dimensional coordinate, and the person ID adding unit 1d attaches a person ID to each player who becomes a subject entering the store from the entrance and supplies it to the person tracking server 5. To do. In addition, when the player moves from its own monitoring area, the entrance stereo image sensor 1 supplies a person ID to the island stereo image sensor 2 that monitors the monitoring area adjacent to the direction in which the player has moved. . The person ID adding unit 1d adds a person ID only to a player who has entered the store from the entrance, but for other players, that is, a player who has once added a person ID. Do not add a person ID to

  As shown in FIG. 3, the island stereo image sensor 2 performs arithmetic processing using parallax on images having different angles supplied from the cameras 2 a and 2 b according to the same principle as the above-described entrance stereo image sensor 1. (By so-called arithmetic processing using trigonometry), the spatial position of the subject in the captured image is obtained as three-dimensional coordinates and supplied to the human tracking server 5.

  In addition, since the island stereo image sensor 2 takes over the person ID supplied from the entrance stereo image sensor 1 or the island stereo image sensor 2 for detecting the position of the player in the adjacent area, a person ID is added. Does not have a part.

  That is, the range in which one entrance stereo image sensor 1 or island stereo image sensor 2 can detect the spatial position of the subject is limited in the store, and each of them is, for example, as shown in FIG. Be placed. That is, the entrance stereo image sensor 1-1 including the cameras 1a-1 and 1b-1 detects the subject in the area Z1, that is, the spatial position of the player including the area where the game store is visited, and similarly. Thus, the island stereo image sensor 2-1 including the cameras 2a-1 and 2b-1 is in the region Z2, and the island stereo image sensor 2-3 including the cameras 2a-3 and 2b-3 is within the region Z3, and the camera 2a-5. , 2b-5 detects the spatial position information of each player in the area Z4. In FIG. 4, only the areas Z1 to Z4 are shown as the monitoring areas of the entrance stereo image sensor 1-1 and the island stereo image sensors 2-1, 2-3, 2-5. Yes, all the monitoring areas are not shown for the entrance stereo image sensors 1-1 to 1-3 and the island stereo image sensors 2-1 to 2-17, but the monitoring areas are set in the same manner. Further, as shown in FIG. 4, the monitoring areas are set so as to overlap, for example, as indicated by the hatched portion. FIG. 4 is a top view of the amusement store showing an arrangement example of the entrance stereo image camera 1, the island stereo image camera 2, and the camera 3 in the amusement store.

  The entrance stereo image sensor 1 and the island stereo image sensor 2 manage the player's position information in each monitoring area for each person ID, but deviate from each monitoring area as the player moves. Therefore, there may be a situation in which it is impossible to monitor with the person ID attached. For this reason, as described above, the monitoring areas are set so as to overlap each other.

  That is, for example, when a player visits the store at the upper left entrance in FIG. 4, the entrance stereo image sensor 1-1 including the cameras 1a-1 and 1b-1 (by the person ID adding unit 1d-1) A shall be attached. Thereafter, when the player moves and enters, for example, in the direction of the region Z3, the entrance stereo image sensor 1-1 detects the position information and the person ID of the moved player, and the adjacent monitor to which the player has moved. The image is supplied to the island stereo image sensor 2-3 including the cameras 2a-3 and 2b-3 for monitoring the area. As a result, in the island stereo image sensor 2-3, it is possible to monitor the same player as the player whose position information has been detected by the entrance stereo image sensor 1-1 with the same person ID.

  As described above, the entrance stereo image sensors 1-1 to 1-3 and the island stereo image sensors 2-1 to 2-17 monitor the position information of each player while cooperating with each other on the position information and the person ID. It has a configuration that can continue. Therefore, the entrance stereo image sensors 1-1 to 1-3 that monitor the area near the entrance / exit newly assign a person ID to the newly visited player, but other island stereo image sensors 2-1 to 2-1 As for 2-16, with the movement of the player, the person ID is supplied in a relay form from the entrance stereo image sensor 1 that monitors the area near one of the entrances. In the present embodiment, the entrance stereo image sensor 1 or the island stereo image sensor 2 has been described with respect to an example in which images captured by two cameras with different angles are processed in consideration of parallax. Considering the spatial position of the subject, it is sufficient if the configuration is such that, for example, more cameras with different angles are captured and processed in consideration of the respective parallaxes. May be requested. In addition, as for the entrance stereo image sensor 1 and the island stereo image sensor 2, an example in which a total of 20 are provided in the example of FIG. 4 is described, but other numbers are installed by changing the monitoring area. You may do it. Further, the function of detecting the position of the player is not limited to the entrance stereo image sensor 1 and the island stereo image sensor 2, and any function may be used as long as the spatial position of the player is required.

  As shown in FIG. 4, the cameras 3-1 to 3-15 are surveillance cameras provided near the entrance of a game shop or between island facilities where game machines in the game shop are installed. The player is imaged at each of the positions, and the captured images are supplied to the image processing units 4-1 to 4-15. In addition, in the example of FIG. 4, although the example in the case of installing in 15 places is shown, the number of installation locations may be other than that. Each camera 3 has a telephoto function and the like. In addition, it is desirable that the entire range in the amusement store be arranged so that any one of the cameras 3 can take an image.

The image processing unit 4 extracts the face image of the player who visits the store based on the captured image supplied from the camera 3 and supplies it to the face recognition server 6. Thereafter, the face recognition server 6 compares the supplied face image with a pre-registered unauthorized player's face image to determine whether or not the player of the supplied face image is an unauthorized player. The unauthorized player determination process to be executed is executed.

  In response to this, the person tracking server 5 supplies a person ID corresponding to the supplied face image (although details will be described later) to the image processing unit 4. Therefore, the image processing unit 4 attaches the supplied person ID to the captured face image and supplies it to the face recognition server 6.

  The person tracking server 5 acquires the position information with the person ID supplied from the entrance stereo image sensor 1 or the island stereo image sensor 2, sets the position management table composed of the person ID and the position information, and is sequentially supplied. The position information for each player coming is associated with the person ID and managed together with the acquired time information. Further, the person tracking server 5 performs position information for each face image based on the face image supplied from the image processing unit 4 and the position information supplied from the entrance stereo image sensor 1 or the island stereo image sensor 2. And the associated person ID, face image, and position information are supplied to the integrated monitoring server 7.

  The face recognition server 6 obtains the person ID, the camera ID, and the face image supplied from the image processing unit 4 and compares them with a face image of a pre-registered illegal player. It is determined whether or not the face image belongs to an unauthorized player. Further, the face recognition server 6 detects the imaging state of the captured face image, for example, the imaging state such as facing down or dark. The face recognition server 6 supplies the determination result, the face image, and the imaging state information to the integrated monitoring server 7 together with the person ID.

  The integrated monitoring server 7 acquires the person ID, face image, and position information supplied from the person tracking server 5, and based on the person ID, face image, and unauthorized player determination process supplied from the face recognition server 6. Information on determination results is acquired and managed as an integrated monitoring table. In this example, the same face image is supplied from both the human tracking server 5 and the face recognition server 6, but it may be supplied from either one.

  Further, the integrated monitoring server 7 monitors the integrated monitoring table, and when an unauthorized player is detected in the determination result column, the integrated monitoring server 7 detects the unauthorized player based on the face image indicating the unauthorized player corresponding to the person ID and the position information. Player visit information is generated and notified to the staff of the game store. In addition, when a player who has passed through the camera a predetermined number of times and has not been verified is detected (a player whose reconfirmation is reported as a determination result of an unauthorized player determination process described later is a predetermined number of times or more) Is detected), based on the image of the player with the corresponding person ID (here, the image facing down or sideways, which cannot be illegal player determination processing) and the position information, Unconfirmed player visit information is generated and notified to the staff of the game store.

  Next, a configuration of an embodiment of the image processing unit 4 will be described with reference to FIG.

  The image acquisition unit 21 acquires a captured image supplied from the camera 3 and supplies it to the face image extraction unit 22. The face image extraction unit 22 analyzes the image supplied from the image acquisition unit 21, extracts a part to be a player's face image, and supplies each face image and position information in the image to the transmission unit 23. The transmission unit 23 supplies the captured image to the human tracking server 5 in addition to the face image and position information in the image. Further, in response to this, the transmission unit 23 attaches the person ID supplied from the person tracking server 5 and the camera ID for identifying the captured camera 3 to the face image and transmits the face image to the face recognition server 6.

  Next, the configuration of an embodiment of the person tracking server 5 will be described with reference to FIG.

  The position information extraction unit 61 acquires position information in the game store corresponding to the player's person ID supplied from the entrance stereo image sensor 1 or the island stereo image sensor 2, and supplies it to the association unit 63. In addition, the position information extraction unit 61 stores the player's person ID and position information supplied from the entrance stereo image sensor 1 or the island stereo image sensor 2 and the time information thereof in the position management table 64.

  The face image acquisition unit 62 acquires the face image supplied from the image processing unit 4 and position information in the captured image, and supplies the acquired position information to the association unit 63.

  The associating unit 63 associates the position information for each person ID supplied from the entrance stereo image sensor 1 or the island stereo image sensor 2 with the face image supplied from the image processing unit 4 so as to associate the person ID and position. Information and a face image are supplied to the position information communication unit 65.

  The position information communication unit 65 supplies the person ID, the position information, and the face image supplied from the association unit 63 to the integrated monitoring server 7. The position information communication unit 65 supplies the person ID to the image processing unit 4 in association with the face image.

  Next, the configuration of one embodiment of the face recognition server 6 will be described with reference to FIG.

  The face image acquisition unit 81 acquires the face image with the person ID and the camera ID supplied from the image processing unit 4 and supplies them to the imaging state determination unit 82 and the similarity calculation unit 83. In response to the supplied face image, the imaging state determination unit 82 is in the imaging state (for example, in a state where the matching process can be performed reliably, it is “good” or the face direction is bad and the matching process is performed. If it is not suitable, an imaging state such as “bad” is determined, and the determination result is attached to the supplied face image and person ID and supplied to the similarity calculation unit 83 and the determination unit 85.

  The similarity calculation unit 83 stores the face image supplied from the face image acquisition unit 81 and the unauthorized player registration database 84 when the imaging state is “good” (when an unauthorized player determination process described later is possible). Each registered face image is compared and compared one by one, the similarity is calculated, and the calculation result is supplied to the determination unit 85. More specifically, the similarity calculation unit 83 obtains various facial feature quantities such as the ratio between the distance between the eyes, the distance from the chin to the forehead, and the length from the chin to the nose. Each difference sum, average ratio, or ratio sum is obtained as the similarity. Further, the similarity calculation unit 83 calculates the face image of the fraudulent player showing the highest similarity among the similarities to the face images of the fraudulent player obtained for each face image to which the person ID is attached. Information for identification (dictionary ID to be described later), the similarity between the face image of the unauthorized player corresponding to the dictionary ID and the captured face image, the person ID, the camera ID, the face image, and the imaging state are displayed in the determination unit 85. Supply. Needless to say, other methods may be used for calculating the similarity. Further, with respect to the fraudulent player's face image registered in the fraudulent player registration database 84, a new fraudulent player may be updated at any time.

  The determination unit 85 stores the face image registered in the unauthorized player registration database 84 stored in the threshold table 86 with respect to the similarity for each face image to which the person ID supplied from the similarity calculation unit 83 is attached. It is compared with a predetermined threshold value set every time, and based on the comparison result, it is determined whether or not the supplied face image is an unauthorized player's face image, and the determination result is supplied to the integrated monitoring server 7.

  Here, it is assumed that the similarity indicates a higher value as the face image is registered as an unauthorized player as indicated by the ratio sum, for example, and the similarity is a value higher than a predetermined threshold value. In this case, an example will be described in which it is determined that the image is a face image of an unauthorized player corresponding to the similarity. However, for example, when the similarity is expressed as a difference sum between the feature amounts of the captured face image and the face image registered as an unauthorized player, the determination unit 85 determines that the similarity is greater than the threshold value. If it is small, the captured face image is regarded as a fraudulent player's face image, or in the case of an average ratio or the like, it is a predetermined value in the range of 0 to 1 and a value close to 1. If there are, they can be regarded as the same person.

  When the similarity is a value lower than a predetermined threshold, the determination unit 85 considers that the player is not an unauthorized player, but associates the similarity, the person ID, and the camera ID with the face image and stores a result less than the threshold. It supplies to the control part 87.

  The less than threshold result storage control unit 87 sets the similarity, person ID, and camera ID to which the similarity is supplied from the determination unit 85 as a value whose similarity is lower than a predetermined threshold. Store in table 88.

  The detection frequency analysis unit 89 monitors the less-threshold table 88, and when the less-threshold result storage control unit 87 updates the less-threshold table 88, that is, when a less-threshold result is newly recorded, the less-threshold table 88. And detecting the frequency at which the fraudulent player's face image with a similarity less than the newly updated threshold is selected as the face image with the highest similarity, and the analysis result is sent to the threshold changing unit 90. Supply. More specifically, the detection frequency analysis unit 89 results in less than the threshold value, but the face image having the highest similarity with the face image of the same unauthorized player is captured by the camera 3 having the same camera ID. Information about the imaging time and number of times of face images is supplied to the threshold value changing unit 90 as an analysis result.

  Based on the analysis result supplied from the detection frequency analysis unit 89, the threshold value changing unit 90 is the same as the face image registered as the same unauthorized player and the face image having the highest similarity that is less than the threshold value. It is determined whether or not the camera 3 with the camera ID has been imaged a predetermined number of times within a predetermined time. For example, when it is determined that the camera 3 having the same camera ID has imaged a predetermined number of times within a predetermined time, the threshold value changing unit 90 updates the threshold value in the threshold value table 86. In this case, the higher the similarity is, the more similar the face image registered in the unauthorized player registration database 84 and the face image captured by the camera 3 are. The similarity threshold corresponding to the player's face image is reduced. More specifically, the threshold value changing unit 90 subtracts a predetermined number of similarity threshold values corresponding to the target fraudulent player's face image, or multiplies a predetermined weighting factor smaller than one. And make it smaller.

  When the similarity is a difference sum or the like, the smaller the threshold value, the more similar the face image captured as a fraudulent player is to be registered. The unit 90 increases the threshold value. Further, when the similarity is an average ratio, the closer the similarity is to a predetermined value (for example, 1), the more similar the captured face image is to the face image registered as an unauthorized player. Therefore, in such a case, the threshold value changing unit 90 sets the threshold value to a value away from the predetermined value (for example, when the predetermined value is 1, the threshold value changing unit 90 sets the threshold value to a smaller value (a value closer to 0). ).

  Further, when the determination unit 85 determines that the imaging state is “defective”, the determination process indicates that the verification process is impossible and reconfirmation is required, and in this case, the verification process is not performed. Since the similarity is not calculated, no information is output to the less-threshold result storage control unit 87.

  Next, the configuration of an embodiment of the integrated monitoring server 7 will be described with reference to FIG.

  The position information acquisition unit 101 acquires a person ID, position information, and face image supplied from the person tracking server 5 and supplies them to the table management unit 104. The face image acquisition unit 102 extracts the face image, the person ID, and the imaging state from the determination result to which the face image, the person ID, and the imaging state supplied from the face recognition server 6 are attached, and extracts the table management unit 104. To supply. The determination result acquisition unit 103 extracts only the determination result from the determination results attached with the face image, the person ID, and the imaging state supplied from the face recognition server 6 and supplies the extracted result to the table management unit 104.

  The table management unit 104 includes the person ID, position information, and face image supplied from the position information acquisition unit 101, and the face image, person ID, and imaging supplied from the face image acquisition unit 102 and the determination result acquisition unit 103. The determination result to which the state is added is associated with a person ID common to both and managed by the integrated monitoring table 105. More specifically, in the integrated monitoring table 105, the determination result table 105a in which the determination result to which the face image and the imaging state are added is associated with the person ID, the position information, and the time information are associated with the person ID. And a position management table 105b (similar to the position management table 64 of the person tracking server 5). The face image is supplied from both the position information acquisition unit 101 and the face image acquisition unit 102, but any face image may be used. That is, the face image may be transmitted from either the person tracking server 5 or the face recognition server 6.

  The monitoring unit 106 monitors the integrated monitoring table 105. If there is information indicating that the player is an unauthorized player in the determination result item, the monitoring unit 106 generates notification information and supplies it to the output unit 107 including a display and a speaker. Thus, the staff of the game store is informed that an unauthorized player has visited the store. In addition, a player whose number of times of passing in front of the camera (the number of times that the unauthorized player determination process described later cannot be performed and the determination result is reported as reconfirmation) is a predetermined number or more can be intentionally verified from the camera 3. The monitoring unit 106 notifies that an unidentified person is coming to the store because there is a possibility that the player is acting as an unauthorized player.

  In addition, when the operation unit 109 including a keyboard and buttons is operated, the reproduction unit 108 performs corresponding position information and time information based on the player's person ID notified as an unauthorized player in the integrated monitoring table 105. Is output to the output unit 107 in accordance with changes in time.

  Next, the integrated monitoring table management process in the monitoring system of FIG. 1 will be described with reference to the flowchart of FIG.

  In step S1, the camera 3 determines whether or not a predetermined time has elapsed, and repeats the process until the predetermined time has elapsed. The predetermined time may be set to an arbitrary time according to the monitoring sampling rate, and may be, for example, about 250 milliseconds.

  If it is determined in step S <b> 1 that the predetermined time has elapsed, in step S <b> 2, the camera 3 captures an image of the installed range and supplies the captured image to the image processing unit 4. The image acquisition unit 21 of the image processing unit 4 acquires the supplied image and supplies it to the face image extraction unit 22.

  In step S <b> 3, the face image extraction unit 22 extracts a player's face image from the supplied image and supplies the extracted image to the transmission unit 23. More specifically, the face image extraction unit 22 extracts a face image from, for example, an arrangement of a characteristic part such as an eye or a nose, where the skin is exposed from the color of the captured image. To the transmitter 23. Further, the transmission unit 23 transmits the face image to the person tracking server 5.

  In step S <b> 21, the position information extraction unit 61 of the person tracking server 5 acquires the position information attached with the person ID supplied from the entrance stereo image sensor 1 or the island stereo image sensor 2 and supplies the position information to the association unit 63. To do.

  In step S <b> 22, the face image acquisition unit 62 acquires the face image transmitted by the process of step S <b> 4, for example, and supplies the acquired face image to the association unit 63.

  In step S <b> 23, the association unit 63 associates the position information with the person ID and the face image with each other and supplies the position information to the position information communication unit 65.

That is, the position information supplied by the entrance stereo image sensor 1 or the island stereo image sensor 2 is coordinates (X ₁ , Y ₁ , Z ₁ ) when the person ID = 1, for example, as shown in FIG. When the person ID = 2, the coordinates (X ₂ , Y ₂ , Z ₂ ), and when the person ID = 3, the coordinates (X ₃ , Y ₃ , Z ₃ ). On the other hand, in the image captured by the camera 3, as shown in FIG. 11, the positional relationship of the player's face image in the two-dimensional image (for example, the player of the face image C viewed from the camera 3). Is positioned behind the player of face image A, and the player of face image B is on the left side of the player of face image A). Therefore, the face image and the person ID are attached so that the positional relationship in the space imaged from the angle of the camera 3 matches the position information detected by the entrance stereo image sensor 1 or the island stereo image sensor 2. By associating with the position information, for example, as shown in FIG. 12, the player of face image B is at the position of person ID = 1, and the player of face image A is at the position of person ID = 2. However, the player of the face image C is associated with the position of the person ID = 3.

  In step S <b> 24, the position information communication unit 65 of the person tracking server 5 transmits the person ID corresponding to the supplied face image to the image processing unit 4.

Further, in step S25, the position information communication unit 65 causes the position management table 64 to store the position information of the player of the face image transmitted by controlling the association unit 63. More specifically, the associating unit 63 registers the position information with the person ID as the position management table 64. For example, the location management table 64 is registered as shown in FIG. In FIG. 13, the left side shows the person ID, the center shows the position information, and the right side shows the time information, and the position information at the time 10: 10: 10: 00 with the person ID = 1 is (x _a , y _a , z _a ), and the position information at time 11: 11: 11: 0 of the person ID = 2 is shown to be (x _b , y _b , z _b ), It is indicated that the position information at the time 12: 12: 12: 00 of the person ID = 3 is (x _c , y _c , z _c ). Here, it is assumed that the XY plane is the floor surface and the Z direction is the height. Note that AA: BB: CC: DD in the time information indicates AA hours BB minutes CC seconds DD / 100 seconds.

  In step S5, the transmission unit 23 of the image processing unit 4 adds the supplied person ID and the camera ID of the camera 3 that captured the face image to the face image, and transmits the face image to the face recognition server 6. To do.

In step S41, the face image acquisition unit 81 of the face recognition server 6 acquires a face image to which a person ID and a camera ID are added. In step S <b> 42, the face image acquisition unit 81 extracts any unprocessed one of the supplied face images and supplies the extracted unprocessed one to the imaging state determination unit 82.

  In step S43, the imaging state determination unit 82 determines the imaging state based on the supplied face image with the person ID and the camera ID. More specifically, the imaging state determination unit 82 recognizes, for example, the geometrical arrangement of the eyes, nose, mouth, ears, and the like, the brightness when imaged, and the like.

  In step S44, the imaging state determination unit 82 determines, based on the imaging state, whether or not the unauthorized player determination process for checking the state in which the face is captured and the facial image of the unauthorized player is possible. A person ID and a camera ID are supplied to the determination unit 85 together with a determination result such as “good” (can perform an unauthorized player determination process) or “bad” (cannot perform an unauthorized player determination process).

  If it is determined in step S44 that the unauthorized player determination process for checking against the face image of the unauthorized player is possible (when the determination result is “good”), the unauthorized player determination process is executed in step S45. The

  Here, the unauthorized player determination process will be described with reference to the flowchart of FIG.

  In step S <b> 81, the similarity calculation unit 83 calculates the degree of similarity between each unauthorized player's face image registered in the unauthorized player registration database 84 and the extracted face image. Here, the unauthorized player registration database 84 is, for example, as shown in FIG. In the unauthorized player registration database 84 of FIG. 15, the leftmost column has dictionary IDs recorded from the top as 001, 002, 003,..., And a face image column is provided on the right side of the dictionary ID. The image data of the face image registered as an unauthorized player corresponding to the dictionary ID is recorded, and the feature amount information is recorded on the right side of the face image. The feature amount is, for example, the ratio between the distance between eyes, the length from the chin to the forehead, and the length from the chin to the nose, and the predetermined feature amount is shown as a vector.

  In FIG. 15, the feature amounts are described as (15, 20,...), (10, 22,...), (13, 21,...) From the top. The face image of the “001” fraudulent player has a feature-to-eye distance of 15 as the first feature, and the length from the chin to the forehead and from the chin to the nose as the second element. It is shown that the ratio of the length is 20%, and the face image of the fraudulent player whose dictionary ID is “002” has an eye-to-eye distance of 10 as the first element of the feature amount, As a second element, it is indicated that the ratio of the length from the chin to the forehead and the length from the chin to the nose is 22%, and the face image of the unauthorized player whose dictionary ID is “003” As the first element of quantity, the distance between eyes is 13, and as the second element, the length from chin to forehead and from chin to nose It is shown that the length ratio is 21%.

  Based on the information in the unauthorized player registration database 84, the similarity calculation unit 83 determines the distance from the eyes to the eye, the length from the chin to the forehead for the face image supplied from the face image acquisition unit 81 as described above. And the various similar facial features such as the ratio of length from chin to nose, etc., and calculating the difference sum, average ratio, or ratio sum of each as the similarity, the most similar fraudulent game (A fraud player who is most similar to the face image acquired by the face image acquisition unit 81 among the face images registered in the fraudulent player registration database 84: here, the fraud player with the highest degree of similarity. ) Of the dictionary ID, similarity, person ID, camera ID, face image, and imaging state are supplied to the determination unit 85.

  In step S <b> 82, the determination unit 85 reads out a threshold value (similarity threshold value determined to be an unauthorized player) corresponding to the dictionary ID supplied from the similarity calculation unit 83 from the threshold table 86, and calculates the similarity level. It is determined whether or not the similarity is greater than or equal to the threshold compared with the similarity supplied from the unit 83. That is, here, the higher the similarity is, the more the fraudulent player's face image and the captured face image are considered to be similar, so the fraud that most closely resembles the captured face image. The threshold value corresponding to the dictionary ID of the player's face image is compared with the calculated similarity. Note that, as described above, the similarity is not necessarily the highest because the similarity between the captured face image and the face image of the fraudulent player who is most similar is not the highest. And the threshold value may differ from those in this example.

  The threshold value table 86 is, for example, as shown in FIG. 16, and is a threshold value table associated with the dictionary ID. More specifically, as shown in FIG. 16, the threshold value table 86 has dictionary IDs recorded in the right column and threshold values recorded on the left side. In FIG. 16, dictionary IDs 001 to 006 are described. In both cases, the threshold value is set to 300.

  In step S82, the determination unit 85 reads the threshold value of the dictionary ID (the similarity threshold value determined to be an unauthorized player) supplied from the similarity calculation unit 83 on the threshold value table 86, and the corresponding similarity level. In FIG. 16, in the case where the similarity supplied from the similarity calculation unit 83 is greater than 300, the determination unit 85 is extracted in step S83. It is set as a determination result that the face image is an unauthorized player's face image.

  If it is determined in step S82 that the similarity supplied from the similarity calculation unit 83 is not greater than or equal to the threshold, that is, if the similarity supplied from the similarity calculation unit 83 is less than 300, in step S84. The determination unit 85 supplies the similarity, dictionary ID, and camera ID that are less than the threshold to the less than threshold result storage control unit 87. When the less-threshold result storage control unit 87 acquires the less-threshold result including the similarity, the dictionary ID, and the camera ID that are less than the supplied threshold, the less-threshold result is stored in the less-threshold table 88. The below threshold table 88 is, for example, as shown in FIG. In FIG. 17, a camera ID column is provided in the leftmost column, and “05”, “01”, “03”, “02”, “06”, and “02” are shown from the top. Further, a dictionary ID column is provided next to the camera ID column, and the dictionary ID corresponding to the camera ID is “011”, “005”, “019”, “002”, “025”, and “005” is shown. Furthermore, a similarity column is provided on the right side of the dictionary ID column, and the similarity corresponding to the camera ID is “260”, “196”, “183”, “217”, “198” from the top. , And “196”. Further, a time column is provided next to the similarity column, and the time recorded corresponding to each camera ID is “10:01”, “10:10”, “10: 50 ”,“ 10:52 ”,“ 11:15 ”, and“ 11:20 ”.

  Therefore, for example, if the uppermost camera ID is less than the threshold value indicated by “05”, the face image most similar to the dictionary ID “011” is captured and the registered fraudulent player face image is obtained. The similarity is “260”, which indicates that the detection was made at “10:01” (10: 1). Similarly, for example, if the camera ID in the second row from the top is less than the threshold value indicated by “01”, a face image most similar to the dictionary ID “005” is captured and registered fraudulent player It is indicated that the similarity is “196” with respect to the face image of “10:10” (10:10) (10:10).

In step S85, the detection frequency analysis unit 89 refers to the less-threshold table 88, and the similarity supplied from the similarity calculation unit 83 is less than the threshold among the face images captured by the camera 3 with the same camera ID. The detection frequency of the face image with the same dictionary ID and the detected time are analyzed, and the analysis result is output to the threshold value changing unit 90 .

In step S86, based on the analysis result, the threshold value changing unit 90 determines whether face images with the same dictionary ID are recorded as a result less than the threshold value a predetermined number of times within a predetermined time. For example, the face image newly picked up by the camera 3 with the camera ID “02” is most similar to the face image registered as an unauthorized player with the dictionary ID “005”, but the similarity is “196”. If the result is less than the threshold value, the result of the step S84 records a result less than the threshold value indicated by the dotted line in the lowermost part of FIG. 17. At this time, the same camera as the result less than the threshold value is recorded. A face image captured by the camera 3 with the ID and having the same similarity but less than the threshold is not detected as shown in FIG. It is determined that the face image with ID is not recorded as a result less than the threshold value a predetermined number of times within a predetermined time, and the process proceeds to step S87.

  In step S87, the determination unit 85 sets as a determination result that the extracted face image is not an unauthorized player's face image.

  If it is determined in step S86 that face images of the same dictionary ID are recorded as a result of less than the threshold value a predetermined number of times within a predetermined time, the threshold value changing unit 90, in step S88, sets the corresponding threshold value. The threshold value corresponding to the dictionary ID recorded in the table 86 is updated, and the process returns to step S82.

That is, in this case, similarity and images captured, since it is assumed that the face image registered as the illegal player is set to a higher value are similar, threshold The changing unit 90 updates the threshold value so as to make it easier to recognize as an unauthorized player by reducing the threshold value. That is, the fact that a result less than the same threshold value is recorded a predetermined number of times or more is similar to the registered unauthorized player although the face image captured by the camera 3 is less than the threshold value, so that the unauthorized player is A state that is not considered indicates that it has been repeated. Face images are taken from various angles. Therefore, the fact that a face image taken a plurality of times by the same camera 3 within a predetermined time is considered to be similar to the same unauthorized player means that the same unauthorized player can be captured repeatedly. It can be considered that the nature is high. Therefore, in such a case, the threshold value changing unit 90 updates the threshold value so that the threshold value is decreased and the player is detected as an unauthorized player even with a lower degree of similarity.

That is, at 11:20, the face image captured by the camera 3 with the camera ID = “ 02 ” and the face image of the unauthorized player registration database 84 are represented by the dictionary ID “005” as shown in the upper part of FIG. The similarity is 196, the similarity is 140 for the dictionary ID “003”, and the similarity is 110 for the dictionary ID “001”. The dictionary ID of the unauthorized player with the dictionary ID “005” is selected, but is smaller than the threshold value “300”. Therefore, in the process of step S84, the table below the threshold value table 88 is displayed as shown at the bottom of FIG. Will be recorded. Therefore, if the predetermined number of times is 2, the face image that is captured with the camera ID “002” and determined to be most similar to the unauthorized player with the dictionary ID “005” is the unauthorized player. It is determined that it is not.

Further, in this state, the face image captured at 11:25 by the camera 3 with the camera ID “02” is similar to the dictionary ID “005” as shown in the lower left part of FIG. When the degree is “204”, the similarity to the dictionary ID “006” is “ 140 ”, and the similarity to the dictionary ID “003” is “105”, the dictionary ID “005” having the highest similarity is obtained. ”Is still selected, but is smaller than the threshold value“ 300 ”, so that it is recorded in the less than threshold value table 88 in the process of step S84.

  However, the image is captured at 11:25 by the camera 3 with the camera ID “02” and is captured by the camera 3 with the camera ID “02”, which is close to the dictionary ID “005” at the beginning. If the time from the set time to the present time) is 10 minutes, in step S86, face images of the same dictionary ID are recorded as a result less than the threshold value a predetermined number of times within a predetermined time. It will be determined.

  At this time, in step S88, the threshold value changing unit 90 updates the threshold value corresponding to the dictionary ID “005” recorded in the corresponding threshold value table 86, for example, “300” shown in FIG. 16 to “150”. To do. As a result, in the processing of the next step S82, the similarity degree of the newly captured face image with respect to the dictionary ID “005” is “204”, and therefore, as shown in the lower right part of FIG. Since it is larger than “150”, the player is detected as an unauthorized player in step S83.

  For example, the threshold value changing unit 90 may update the threshold value by decreasing the threshold value by multiplying the threshold value by a weight coefficient of 0 to 1, or update the threshold value by subtracting a predetermined value. Anyway. That is, in this case, the threshold value changing unit 90 may update the threshold value by multiplying “0.5” as a weighting coefficient (150 = 300 × 0.5), or “150” as a predetermined value. "May be subtracted from the threshold (150 = 300-150).

  Through the above processing, it is determined not only whether the face image included in the image supplied from the image processing unit 4 is a fraudulent player's face image, but also the captured face image is a fraudulent player. For example, even if there is a misperception that the degree of similarity does not exceed a predetermined threshold due to an angle or an imaging state, for example, the same dictionary ID is the highest degree of similarity within a predetermined time with the same camera 3. When a state that is regarded as high (a state in which it is determined that the same fraudulent player is most similar) is repeated, the threshold value can be lowered based on the frequency of imaging. It is possible to reduce false positives and accurately detect visits by unauthorized players. Further, when the face image captured with low similarity is not actually a face image of an unauthorized player, the same dictionary ID is regarded as having the highest similarity within a predetermined time by the same camera 3 Since it is unlikely that (the state in which it is determined that the same unauthorized player is most similar) is repeated, the face image of a player who is not an unauthorized player is the face image of an unauthorized player It is possible to reduce the possibility of being mistaken as.

  In the above description, the case where the captured face image and the face image of the player registered as an unauthorized player are similar is described as having a higher similarity value. In the case where the similarity is lower as the face image of the player registered as an unauthorized player is more similar (when the value is close to 0), the threshold value changing unit 90 includes the corresponding threshold value table 86. Update to increase the threshold corresponding to the recorded dictionary ID. When the similarity is closer to a predetermined value as the face image of the player registered as an unauthorized player is more similar, the threshold value changing unit 90 corresponds to the dictionary ID recorded in the corresponding threshold value table 86. The threshold value is updated so as to be a value away from the predetermined value. In addition, the predetermined number of times for updating the threshold may be two or more times, and it is needless to say that the predetermined time may be a time other than 10 minutes. Furthermore, an example has been described in which the threshold is updated only when a face image having the highest similarity to the face image of an unauthorized player with the same dictionary ID is captured by the same camera 3 within a predetermined time. However, any one of the plurality of cameras 3 repeatedly captures a face image calculated as a similarity that is the highest but smaller than the threshold value with a face image of an unauthorized player with the same dictionary ID within a predetermined time. In this case, the threshold value may be updated so as to be a value that is easily recognized as an unauthorized player.

  In the above description, the case where the threshold value is lowered according to the frequency with which the state in which it is determined that the same fraudulent player is most similar is repeated, but for example, the same fraudulent game The threshold is not changed according to the frequency of repetition of a state in which it is determined that the person is most similar, but the degree of similarity is multiplied by a weighting factor greater than 1, and is greater than the threshold. You may make it easy to detect as an unauthorized player by making it easy to take a value. As a matter of course, also in this example, when the similarity is defined so as to be smaller as the similarity value is smaller, the weighting coefficient multiplied by the similarity is a value smaller than 1, and as a result, the similarity is Since it becomes easy to take a value smaller than the threshold value, it becomes easy to be regarded as an unauthorized player.

  Furthermore, in the above, an example has been described in which when the similarity is smaller than the threshold, the result less than the threshold is always recorded in the less than threshold table 88. However, the similarity is equal to or higher than a predetermined value, and Only when the result is less than the threshold value, the result less than the threshold value may be recorded. By doing so, the captured face image is considered to be more similar to the unauthorized player. It is possible to count the number of detections only as the number of detections, so that the threshold is not reduced by repeating a state in which a face image that does not resemble an unauthorized player is captured. It becomes possible to suppress misidentification with higher accuracy.

  Now, the description returns to the flowchart of FIG.

  If it is determined in step S44 that determination is impossible, in step S46, the determination unit 85 sets reconfirmation as a determination result, and the process proceeds to step S47.

  In step S47, the face image acquisition unit 81 determines whether or not all of the supplied face images have been collated with the face image of the unauthorized player, that is, whether or not the unauthorized player determination process has been performed. If not, the process returns to step S42, and the subsequent processes are repeated. If it is determined in step S47 that all the illegal player determination processes have been performed, the process proceeds to step S48.

  In step S <b> 48, the determination unit 85 transmits the person ID, face image, imaging state, and determination result to the integrated monitoring server 7. If the determination result is an unauthorized player, the determination unit 85 also transmits to the integrated monitoring server 7 the registered fraudulent player's face image and similarity information with the highest similarity.

  On the other hand, in step S61, the face image acquisition unit 102 of the integrated monitoring server 7 includes the person ID, the face image, the imaging state, and the determination result supplied from the face recognition server 6, the face image to which the person ID is added, And the imaging state information is acquired and supplied to the table management unit 104. Further, the determination result acquisition unit 103 acquires a determination result from the person ID, the face image, the imaging state, and the determination result supplied from the face recognition server 6 and supplies the determination result to the table management unit 104.

  That is, as described above, when the unauthorized player determination process is not completed, the image processing unit 4 supplies the face image and the person ID to the face recognition server 6, and the face recognition server 6 is based on the face image. Since the ID, the face image, the imaging state, and the determination result are supplied, the face image acquisition unit 102 includes the person ID, the face image, the imaging state, and the determination result supplied from the face recognition server 6. The face image to which the person ID is added and the information on the imaging state are acquired, and the determination result acquisition unit 103 acquires the determination result.

  In step S62, the table management unit 104 stores the position information to which the person ID supplied from the position information acquisition unit 101 in the process of step S61 and the time information are stored in the integrated monitoring table 105 as the position management table 105b. . The table management unit 104 also includes the face image with the person ID supplied from the face image acquisition unit 102, information on the imaging state, and the determination result of the unauthorized player determination process supplied from the determination result acquisition unit 103. Based on this information, the face image, the imaging state, and the determination result are recorded in the integrated monitoring table 105 as the determination result table 105a in association with the person ID (both in the determination result table 105a and the position management table 105b, the first time In the case of, it becomes a registration process, and after that, it is accumulated.)

  Here, the position management table 105b is the same as the position management table of FIG. The determination result table 105a is a table as shown in FIG. 19, for example. That is, a person ID, a face image, an imaging state, and a determination result are shown from the left in the drawing, and a face image corresponding to the person ID = 1 is shown in the top row. In addition, the imaging state is “good”, and the determination result is written as “normal player”, indicating that the player is a normal player (not an unauthorized player).

  In the second row from the top, a face image corresponding to the person ID = 2 is shown. In addition, the imaging state is “good”, and the determination result is written as “illegal player”, indicating that the player is an unauthorized player.

  Furthermore, in the third row from the top, a face image corresponding to the person ID = 3 is shown. In addition, the imaging state is “bad (face orientation failure)”, the determination result is written as “reconfirmation 2”, it is unknown whether the player is an unauthorized player, and reconfirmation is reported as the determination result It is shown that the number of times (the number of times that the illegal player determination cannot be made) is two.

  In the fourth row from the top, there is no face image corresponding to the person ID = 4, indicating that no face image has been captured. In addition, the imaging state is “no face image”, the determination result is written as “reconfirmation 3”, it is unknown whether or not the player is an unauthorized player, and the number of times reconfirmation is reported as the determination result (incorrect It is shown that the number of times that the player determination is not possible is 3 times.

  If the determination result is an unauthorized player, information on the similarity and the face image of the unauthorized player with the highest similarity is also sent, so for example, in the imaging state column of FIG. Such information may be registered, or a new separate field may be provided in the integrated monitoring table 105 for registration.

  Through the above processing, the player of the face image in the image captured by the camera 3 is associated with the person ID, and the position information, the face image, and the determination result are collected in the integrated monitoring table 105. It is possible to manage the illegal player determination process based on the player's face image in a lump. Note that since the information in the determination result table 105a and the position management table 105b in the integrated monitoring table 105 is managed for each person ID, the face image, imaging state, and determination result managed in the determination result table 105a The position information and time information managed by the position management table 105b can be managed as one table (integrated monitoring table 105) by the person ID.

  In addition, about the process of step S66, you may make it update a face image, only when the imaging state of the face image in which an unauthorized player determination process cannot be performed is better than the existing state. As a result, only the face image in the best state can be recorded in the integrated monitoring table 105, and when the unconfirmed player visit information described later is presented, the unauthorized player determination process is impossible. However, since the face image in the best state can be presented, it becomes possible to facilitate the search for unidentified players by the staff.

  Next, with reference to the flowchart of FIG. 20, the monitoring process of the unauthorized player by the integrated monitoring server 7 or the unconfirmed player that cannot perform the unauthorized player determination process will be described.

  In step S201, the monitoring unit 106 refers to the integrated monitoring table 105 and pays attention to the determination result column to determine whether there is an unauthorized player. For example, when there is an unauthorized player, that is, If the unauthorized player determination result is an unauthorized player, the process proceeds to step S202.

  In step S <b> 202, the monitoring unit 106 supplies a series of information corresponding to the person ID determined as an unauthorized player from the integrated monitoring table 105 to the output unit 107. The output unit 107 displays, for example, illegal player visit information as shown in FIG. 21 based on a series of information corresponding to the person ID supplied from the monitoring unit 106, and notifies the illegal player visit.

  In FIG. 21, a person ID display field 201 is provided at the upper left in the figure. In this case, “person ID = AA” is displayed, and an unauthorized player is assigned a person ID = “AA” in the integrated monitoring table 105. It is shown that the person is registered as "." Below the person ID display field 201, a face image display field 202 and a registered image display field 203 are provided, each displaying a face image. The face image display field 202 displays a face image corresponding to the person ID = AA, and the registered image display field 203 is registered in advance with the highest similarity to the player displayed in the face image display field 202. A face image of a fraudulent player is displayed.

  Below the face image display field 202, a condition display field 204 is provided. In this case, “Face detection condition: good” and “Face similarity: 204” are displayed in the condition display column 204, the imaging state is “good”, and the similarity calculation result Is shown to be 204.

  In the right column of FIG. 21, a current position display field 205 is provided, and based on the latest position information corresponding to the person ID = AA, the player's game store currently determined to be an unauthorized player. The position is shown as point 205a, for example.

  Below the condition display field 204, an action trajectory display button 206 is provided. When an action trajectory of a player determined as an unauthorized player is to be reproduced, it is operated (pressed) by the operation unit 109 or the like. Then, the action trajectory reproduction process described later is executed.

  Below the current position display field 205, a confirmation button 207 is provided. When it is desired to end the display of illegal player visit information, the display of the illegal player visit information is ended by being operated.

  As a result of the above processing, when a determination result indicating that an unauthorized player has been detected is registered in the integrated monitoring table 105, the unauthorized store visitor information is displayed from the output unit 107, so that the staff of the game store can Therefore, it is possible to recognize that an unauthorized player has visited the store, and it is also possible to quickly find an unauthorized player based on the face image and information on the current position.

  If a speaker or the like is provided in the output unit 107, a sound indicating a warning may be generated. In this way, it is possible to more quickly recognize the visit of an unauthorized player. It becomes.

  Now, the description returns to the flowchart of FIG.

  In step S203, the monitoring unit 106 refers to the integrated monitoring table 105 and pays attention to the determination result column. The number of times the determination result is unconfirmed (reconfirmation) and reconfirmation is reported as the determination result, that is, , It is determined whether or not there is a player whose number of times passed in front of the camera 3 is greater than or equal to a predetermined number of times. The process proceeds to step S204.

  In step S <b> 204, the monitoring unit 106 supplies the output unit 107 with a series of information corresponding to the player ID of the player who cannot perform the unauthorized player determination process, from the integrated monitoring table 105. The output unit 107 displays, for example, unconfirmed player visit information as shown in FIG. 22 by a series of information corresponding to the person ID supplied from the monitoring unit 106, and notifies the unconfirmed player visit.

  In FIG. 22, a person ID display field 221 is provided at the upper right in the figure. In this case, “person ID = BB” is displayed, and an unidentified person is person ID = “BB” in the integrated monitoring table 105. It is shown that the person is registered as. Below the person ID display field 221, a face image display field 222 is provided. In the face image display field 222, an image corresponding to the person ID = BB that cannot be subjected to the unauthorized player determination process is displayed. Note that FIG. 22 shows an example in which an image that cannot be recognized as a face, for example, facing backward or facing sideways is displayed. By displaying in this way, even if the unauthorized player determination process cannot be performed, the clerk can recognize features such as the appearance, clothes, complexion, or hairstyle, so the possibility of an unauthorized player can be visually confirmed. It becomes possible to search for a high player.

  In the right column of FIG. 22, a current position display field 223 is provided. Based on the latest position information corresponding to the person ID = BB, a player's game store that cannot currently perform an unauthorized player determination process is provided. The position is shown as point 223a, for example.

  An action trajectory display button 224 is provided below the face image display field 222, and when an action trajectory of a player who cannot perform unauthorized player determination processing is to be reproduced, the operation section 109 or the like is operated (pressed down). Then, an action trajectory reproduction process described later is executed.

  Below the current position display field 223, a confirmation button 225 is provided. When it is desired to end the display of unconfirmed player visit information, the display of the unconfirmed player visit information ends.

  As a result of the above processing, the number of times that the integrated monitoring table 105 has passed in front of the camera 3 is a predetermined number or more (the number of times that reconfirmation has been reported as a determination result is a predetermined number or more), and the state where the unauthorized player determination process is disabled Thus, by displaying unidentified player visit information from the output unit 107, it is possible for a clerk at a game store to be aware of the existence of a monitoring system and to act as an unauthorized player who is acting so that a face image cannot be accurately captured. It is possible to recognize that a highly-skilled player has visited the store, and in addition, an image (image that can not be determined as an unauthorized player but can recognize the appearance, clothes, complexion, or hairstyle) and the current location Based on the information, it becomes possible to quickly find a player who is not capable of performing the unauthorized player determination process and is likely to be an unauthorized player.

  If the output unit 107 is provided with a speaker or the like, it may be possible to generate a sound indicating a warning different from the case where an unauthorized player is detected. It becomes possible to find a player who cannot perform the player determination process and is highly likely to be an unauthorized player.

  If there is no unauthorized player in step S201, the process in step S202 is skipped, and the number of times that the camera has passed in front of the camera is greater than or equal to the predetermined number in step S203 (the number of times reconfirmation is reported as a determination result is If there is no player of a predetermined number or more), the process of step S204 is skipped, the process returns to step S201, and the subsequent processes are repeated.

  Next, action trajectory processing executed when the above-described action trajectory buttons 206 and 224 are operated will be described with reference to the flowchart of FIG.

  In step S221, the reproducing unit 108 refers to the position management table 105b of the integrated monitoring table 105, and accumulates based on the person ID displayed in the person ID display field 201 or 221 of FIG. Read position information and time information.

  In step S222, the reproducing unit 108 plots the position information on the coordinates in the amusement store according to the time information, and supplies the result to the output unit 107 for display.

  In step S223, the reproducing unit 108 determines whether or not the next position information exists. For example, when it is determined that the next position information exists, the process returns to step S221, and the subsequent processes are performed. Is repeated and all the position information is plotted, it is determined in step S223 that the next position information does not exist, and in step S224, “return” (for example, the return button 254 in FIG. 24) has been operated. If it is determined whether or not “return” has been operated, the process ends.

  For example, the action trajectory is displayed as shown in FIG. That is, in FIG. 24, a plan view 251 inside the game store is shown, and points 251a indicating the player's position are plotted in sequence, and after passing, a trajectory 252 is displayed as indicated by the dotted line in the figure. The Further, in FIG. 24, a game table display field 253 indicating a game table that has stopped on the path of the trajectory 252 is also shown. By doing in this way, among the game tables provided on the island facility, In this case, it can be recognized that an unauthorized player or an unidentified player has stopped at the 151st level, and it can be seen that the 151th level has been stopped.

  As a result, if a fraud is detected at a game machine that stops, an unauthorized player who is detected by analyzing an image captured in the vicinity of the fraudulent game table can be obtained by providing a separate surveillance camera. Can be used as proof that some sort of fraud has occurred, and it is possible to contribute to the clearance of fraudulent players. In addition, in the case of an unconfirmed player, it can be used as a material for determining whether or not the player is an unauthorized player based on the behavior trajectory described above.

  According to the above, in the monitoring system, it is possible to reduce the false detection of an unauthorized player and improve the detection rate.

  Incidentally, the series of monitoring processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.

  FIG. 25 shows a configuration example of a general-purpose personal computer. This personal computer includes a CPU (Central Processing Unit) 301. An input / output interface 305 is connected to the CPU 301 via a bus 304. A ROM (Read Only Memory) 302 and a RAM (Random Access Memory) 303 are connected to the bus 304.

  The input / output interface 305 includes an input unit 306 including an input device such as a keyboard and a mouse for a user to input an operation command, an output unit 307 for outputting a processing operation screen and an image of a processing result to a display device, a program and various data. A storage unit 308 including a hard disk drive for storing data, a local area network (LAN) adapter, and the like are connected to a communication unit 309 that performs communication processing via a network represented by the Internet. Also, a magnetic disk 321 (including a flexible disk), an optical disk 322 (including a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc)), and a magneto-optical disk 323 (including an MD (Mini Disc)) Alternatively, a drive 310 for reading / writing data from / to a recording medium such as the semiconductor memory 324 is connected.

  The CPU 301 executes various processes according to a program stored in the ROM 302 or a program read from the magnetic disk 321 to the semiconductor memory 324 and installed in the storage unit 308 and loaded from the storage unit 308 to the RAM 303. The RAM 303 also appropriately stores data necessary for the CPU 301 to execute various processes.

  In this specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series in the order described, but of course, it is not necessarily performed in time series. Or the process performed separately is included.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

It is a figure which shows the structure of one Embodiment of the monitoring system to which this invention is applied. It is a figure which shows the structure of one Embodiment of the entrance stereo image sensor of FIG. It is a figure which shows the structure of one Embodiment of the island stereo image sensor of FIG. It is a figure explaining the example of arrangement | positioning of the entrance stereo image sensor of FIG. 1, an island stereo image sensor, and a camera. It is a figure which shows the structure of one Embodiment of the image processing unit of FIG. It is a figure which shows the structure of one Embodiment of the person tracking server of FIG. It is a figure which shows the structure of one Embodiment of the face recognition server of FIG. It is a figure which shows the structure of one Embodiment of the integrated monitoring server of FIG. It is a flowchart explaining the management process of an integrated monitoring table. It is a figure explaining the positional information detected by a stereo image sensor. It is a figure explaining the image imaged with an entrance camera or an island camera. It is a figure explaining the process which matches position information and a face image. It is a figure explaining a position management table. It is a flowchart explaining an unauthorized player determination process. It is a figure explaining an unauthorized player registration database. It is a figure explaining a threshold value table. It is a figure explaining a table less than a threshold value. It is a figure explaining the effect by the update of the threshold value in an unauthorized player determination process. It is a figure explaining the determination result table. It is a flowchart explaining a warning process. It is a figure which shows the example of a display of the unauthorized player visit information which shows that the unauthorized player visited the store. It is a figure which shows the example of a display of unidentified player visit information which shows that an unidentified player visited a store. It is a flowchart explaining action locus reproduction processing. It is a figure explaining action locus reproduction processing. It is a figure explaining a medium.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,1-1 thru | or 1-3 Entrance stereo image sensor 2,2-1 thru | or 2-17 Island stereo image sensor 3,3-1 thru | or 3-15 Camera 4 Image processing unit 5 Person tracking server 6 Face recognition server 7 Integration Monitoring server

Claims (7)

Storage means for storing unauthorized registrant face images;
An acquisition means for acquiring an arrival person's face image;
Similarity calculation means for calculating the similarity between the face image of the arrival person and the face image of the unauthorized registrant stored in the storage means;
Determining means for determining that an unauthorized registrant whose calculated similarity with the arrival has arrived when the similarity calculated by the similarity calculating means is higher than a predetermined threshold;
In the determination result of the determination means, when the similarity is lower than the threshold , the similarity is stored in association with the face image of the unauthorized registrant whose similarity with the arrival is calculated. When,
A frequency detecting means for detecting a frequency at which a face image of an unauthorized registrant whose similarity to the arrival person is lower than the threshold is stored in the storage means ;
Of fraud registration's face images stored by the storing means, the frequency detected by the frequency detecting means and a threshold updating means for low update the threshold for many incorrect registration's face image than a predetermined number of times Monitoring device. The acquisition means acquires a camera ID for identifying a camera that captured the arrival person's face image together with the arrival person's face image;
When the similarity is lower than the threshold value in the determination result of the determination means, the storage means uses the similarity as the face image of the unauthorized registrant and the camera ID for which the similarity with the arrival has been calculated. And store it in association with
The frequency detection means obtains a face image captured by a camera having the same camera ID from among face images of unauthorized registrants whose similarity to the arrival person stored in the storage means is lower than the threshold value. The monitoring apparatus according to claim 1, wherein the monitoring frequency is detected. The threshold update unit, the frequency detected by the frequency detecting means, when more than a predetermined number of times within a predetermined time, the monitoring device according to claim 1 for updating lower the threshold. The threshold update unit, the frequency detected by the frequency detecting means, when more than a predetermined number of times, the monitoring device according to claim 1 for updating lower weighting the threshold. The threshold update unit, the frequency detected by the frequency detecting means, when more than a predetermined number of times, the monitoring device according to claim 1 for updating from the threshold by a predetermined value lower. An accumulation step for accumulating unauthorized registrant face images;
An acquisition step of acquiring a face image of the arrival person;
A similarity calculation step of calculating the similarity between the face image of the arrival person and the face image of the unauthorized registrant accumulated by the processing of the accumulation step;
A determination step for determining that an unauthorized registrant whose calculated similarity with the arrival has arrived when the similarity calculated by the processing of the similarity calculation step is higher than a predetermined threshold;
In the determination result determined by the processing of the determination step, when the similarity is lower than the threshold , the similarity is associated with a face image of an unauthorized registrant whose similarity with the arrival has been calculated. Memory step for storing
A frequency detection step of detecting a frequency at which a face image of an unauthorized registrant whose similarity to the arrival person is lower than the threshold is stored in the processing of the storage step;
Of fraud registration's face image stored by the processing of the storage step, the threshold updating the frequency detected by the processing of the frequency detecting step updates lowered the threshold for many incorrect registration's face image than a predetermined number of times A monitoring method including steps and. An accumulation step for accumulating unauthorized registrant face images;
An acquisition step of acquiring a face image of the arrival person;
A similarity calculation step of calculating the similarity between the face image of the arrival person and the face image of the unauthorized registrant accumulated by the processing of the accumulation step;
A determination step for determining that an unauthorized registrant whose calculated similarity with the arrival has arrived when the similarity calculated by the processing of the similarity calculation step is higher than a predetermined threshold;
In the determination result determined by the processing of the determination step, when the similarity is lower than the threshold , the similarity is associated with a face image of an unauthorized registrant whose similarity with the arrival has been calculated. Memory step for storing
A frequency detection step of detecting a frequency at which a face image of an unauthorized registrant whose similarity to the arrival person is lower than the threshold is stored in the processing of the storage step;
Of fraud registration's face image stored by the processing of the storage step, the threshold updating the frequency detected by the processing of the frequency detecting step updates lowered the threshold for many incorrect registration's face image than a predetermined number of times A program that causes a computer to execute processing including steps.

Images