JP2007190076A - Monitoring support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a monitoring support system which can lessen the work burden of face comparison on employees and operators. <P>SOLUTION: A best shot face image data extraction section 21 extracts the best shot face image data of the same person in different places on the basis of a plurality of images captured by a monitoring camera 11. An offline face comparison section 24 compares the best shot face image data of an arbitrary person in different places with the face image data of a fraudster extracted by a fraudster face image data extraction section 23 and outputs the best shot face image data of a person as a candidate for face image data of the fraudster if the face image data exceed a prescribed coincidence rate. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a video monitoring system including a monitoring camera, a video storage device, and the like, and relates to a monitoring support system that supports the discovery of a registered person using face images and supports behavior monitoring. .

  Conventionally, as shown in Patent Document 1, for example, a player verification check system has been proposed in which verification verification of a player who has visited a game hall can be performed without human intervention. In such a system, the face data of an unauthorized person is registered in advance, collated with the face data of a player who has visited the game hall, and the collation result is output (alarmed) online. .

JP 2001-178958 A

  In general, the correct response rate of face verification (authentication) in such a player verification system is not 100%, but there are two errors: missed registrant (missing) and error (misinformation) mistaken non-registered registrant. To do. In the setting to reduce the error that misses the registrant, the error that mistakes the non-registrant from the registrant increases at the same time, while in the setting that reduces the error that mistakes the non-registrant to the registrant, the error that misses the registrant at the same time. Get higher. In other words, these two errors are in a trade-off relationship.

  In particular, when alarming online, if there are many misreports, the workload of employees and operators who are system users increases, so it is necessary to operate the system to minimize misreports as much as possible. However, in such an operation, there are a number of cases in which registrants who are registrants are overlooked, and there is a problem that the expected effect as a player verification confirmation system cannot be obtained.

  The present invention has been made to solve the above-mentioned problems, and it is possible to check an unauthorized person's visit time offline using a recorded image and to find an unauthorized person's return visit online. The purpose is to provide a monitoring support system that provides a function to support the work to be performed and can reduce the work load of employees and operators.

  The monitoring support system according to the present invention includes a best shot face image data extraction unit that extracts the best shot face image data of the same person at different places and the acquisition time based on a plurality of images obtained at different places. The best shot face image data extraction unit extracts the best shot face image data for each different location of the target person and the face image data of the unauthorized person, and these face image data exceeds a predetermined match rate. In this case, the best shot face image data of the person is output as a candidate for the unauthorized person's face image data.

  The monitoring support system according to the present invention compares the best shot face image data of different places with the face image data of the unauthorized person, and when the face image data exceeds a predetermined matching rate, the best shot face image Since the data is output as a candidate for fraudulent face image data, it is possible to reduce the workload of employees and operators in the face matching process.

Embodiment 1 FIG.
1 is a block diagram showing a monitoring support system according to Embodiment 1 of the present invention.
In the following embodiments, a game store will be described as an example. However, the present invention is not limited to a game store and may be of any size, such as a supermarket, a convenience store, a department store, etc. It can be applied to various stores and facilities where humans come and go.

  In FIG. 1, the monitoring support system includes a video monitoring system 1 and an image analysis device 2. The video surveillance system 1 is a conventional video security system including a surveillance camera 11 and a video storage device 12. The surveillance camera 11 is an imaging device for capturing images of different places in a store, and includes, for example, the following cameras.

  That is, a store entrance monitoring camera 111 that captures customers coming from the store entrance, an inter-passage monitoring camera 112 that captures customers who are playing or moving, and a exit switch monitoring that captures customers coming to the exit (coin) exchange The camera 113 includes a prize exchange counter monitoring camera 114 that photographs a customer who comes to the prize exchange counter immediately before leaving the store. The installation location and number of cameras can be arbitrarily laid out, and are not limited to those in this embodiment. It is assumed that each camera is assigned a number in order so that each individual can be identified. The type of camera is not particularly limited, such as an analog camera (NTSC camera, etc.), a digital camera (IEEE 1394 camera, USB camera, etc.), a network camera (Web camera, IP camera, etc.).

  The video storage device 12 is a so-called multi-channel recorder, has a built-in hard disk, and stores video signals such as analog video output and digital video output of the store entrance monitoring camera 111 to the gift exchange counter monitoring camera 114 together with shooting time information. It is a device that records and saves inside. In addition, recorded data in the video storage device 12 can take out video of an arbitrary channel at an arbitrary time based on an external command.

  The image analysis apparatus 2 is an apparatus that analyzes the video acquired by the video monitoring system 1 and outputs the analysis result. The best shot face image data extraction unit 21, the customer list generation unit 22, the unauthorized person face image data extraction. Unit 23, offline face matching unit 24, and input / output unit 25.

  The best shot face image data extraction unit 21 is a functional unit that performs best shot face image data extraction processing described in, for example, Japanese Patent Application Laid-Open No. 2005-227957. Here, since image processing for extracting the best shot face image data is known, it will be briefly described. When a person enters a frame in each camera of the monitoring camera 11, the face area of the person is detected and extracted as face image data. Once a face is detected, tracking is performed on a person-by-person basis. Tracking ends when a person frames out of the camera. At that stage, the face direction, brightness, size, presence / absence of eye detection, image quality, etc. are comprehensively selected from the multi-frame face image data tracked so far. Thus, the best shot face image data most suitable for face collation is automatically determined. Since tracking is performed on a person-by-person basis, the best shot face image data for each person can be obtained even when a plurality of persons are in the frame at the same time. The determination of the person's frame-in and frame-out may be made based on the detection state of the face, or for example, based on the detection state of the person by the person tracking method described in Japanese Patent Laid-Open No. 2003-169319. Good.

  The visitor list generation unit 22 is a functional unit that holds the list of the best shot face image data obtained by the best shot face image data extraction unit 21 after adding the number of the monitoring camera 11 and the shooting time. In this store visitor list generation unit 22, every time new best shot face image data is obtained by the best shot face image data extraction unit 21, it is added to the store visitor list and is updated every moment. That is, the store visitor list generation unit 22 holds face image data at a plurality of locations for each person.

The unauthorized person face image data extraction unit 2 is a functional unit that identifies an unauthorized person based on an image obtained by the monitoring camera 11 or an image stored in the video storage device 12 and outputs the unauthorized person face image data.
Generally, in a game store, each game machine is connected to a hall computer (not shown) which is a host computer of the store to monitor the entry and exit of a ball (coin). If there are an abnormally large number of balls (coins) with respect to the effect setting (for example, the jackpot probability) of each game table, there is a high possibility that the game table is cheating. In addition, an opening / closing signal of a gaming machine is transmitted to the hall computer. If the gaming machine is opened / closed without the permission of the employee, there is a high possibility that the gaming machine is cheating. In such a case, based on the information of the hall computer, the video of the monitoring camera 11 and the video storage device 12 is confirmed to identify an unauthorized person.

  That is, when the information indicating that the fraudulent person face image data extraction unit 23 is likely to be fraudulent is output from the hall computer, the camera number of the surveillance camera 11 photographing the generation source and the generation time thereof Based on the above, the face image data of the unauthorized person is extracted from the video stored in the video storage device 12. In other words, the face detection process is executed from the video whose time coincides with the place where the fraud has been performed, the face area of the person is extracted as face image data, and this is output as the face image of the fraudster. The difference from the best shot face image data extraction unit 21 is not to target multiple frames but to process one arbitrary frame.

  The offline face collation unit 24 extracts the fraudulent face image data by the fraudster face image data extraction unit 23 and then the best shot face image data group generated by the customer list generation unit 22 and the fraudster face image data. And calculating the facial image shooting time of the unauthorized person visiting the store, leaving the store, playing, and moving.

  Here, face collation (authentication) calculates the similarity of two pieces of face image data. If the similarity exceeds a threshold set by the system, it is determined that the persons are different from each other. Refers to the function to perform. As this processing, for example, “High-speed face authentication system based on optimal selection of Rectangle Filter”, published at Image Sensing Symposium 2004, Shotaro Miwa, Kazuo Hajima, Hiroshi Kage, Manabu Hashimoto, Michael Jones, Jay Thornton, SSII2004 The method described in. Can be used. The face matching method is not limited to such a method, and various known methods can be used.

  The input / output unit 25 includes a data input device such as a keyboard and a data output device such as a monitor, and has a function of displaying the output video of the monitoring camera 11 and the video of the video storage device 12. At that time, the images of each camera can be selected and displayed individually on the screen of the monitor, or the images of a plurality of cameras can be displayed simultaneously. Also, the best shot face image data obtained by the best shot face image data extraction unit 21 and the unauthorized person face image data obtained by the unauthorized person face image data extraction unit 23 can be displayed. Further, the position of these face image data can be displayed superimposed on the extraction source image. Furthermore, the past face image data list of the store visitor list generation unit 22 can be displayed.

Next, the operation of the monitoring support system according to the first embodiment will be described.
First, the respective images of the store entrance monitoring camera 111 to the prize exchange counter monitoring camera 114 are input to the best shot face image data extraction unit 21, and the best shot face image data is extracted for each person. The extracted best shot face image data is stored as a list in the customer list generation unit 22 with a camera number and shooting time added. Each video acquired by the store entrance monitoring camera 111 to the prize exchange counter monitoring camera 114 is stored in the video storage device 12.

  Further, the user designates the offline face matching timing from the input / output unit 25. The timing is, for example, once a day or every hour. Accordingly, the unauthorized person face image data extraction unit 23 extracts unauthorized person face image data at a timing designated by the user. For example, when there is information indicating fraud at an arbitrary location and time, a past stored video that matches the location and time is taken out from the video storage device 12 and the face image data present in this captured image Cut out. That is, the camera number and the shooting time are designated, the video is reproduced while being taken out, face detection processing is executed in an arbitrary scene, and a human face area is extracted as face image data. Then, this face image data is output to the offline face collation unit 24 as face image data of an unauthorized person. The unauthorized person face image data extraction unit 23 automatically extracts an unauthorized person's face image from the stored video of the video storage device 12. If the unauthorized person's face image data cannot be automatically extracted, the unauthorized person's face image data extraction unit 23 extracts it with assistance from the user via the input / output unit 25. You may do it.

  In the offline face collation unit 24, after the fraudster face image data extraction unit 23 extracts the fraudster face image data, the best shot face image data group generated by the customer list generation unit 22 and the fraudster face image data are extracted. Are compared, and the visit time of the fraudster, the time of leaving the store, the game, and the face image shooting time during movement are calculated.

  When face matching is performed on N best shot face image data groups and one unauthorized person face image data, N similarities are obtained as a result. By setting the system threshold value as T1 in advance via the input / output unit 25, A best shot face image data exceeding T1 out of N similarities is extracted. The extracted face image data is sent to the input / output unit 25 together with the unauthorized person face image data, and displayed on the monitor as a candidate for the unauthorized person face image.

  By confirming the A best shot face image data and the unauthorized person face image data that are likely to be the unauthorized person on the monitor of the input / output unit 25 at the same time, the employee or the operator can The best shot face image data of the unauthorized person can be selected. That is, since the correct answer rate of face matching (authentication) is not 100%, the A best shot face image data may include face image data other than the unauthorized persons. However, such an incorrect result can be ignored by the employee or operator confirming the result visually.

  In the first embodiment, the unauthorized person face image data extracting unit 23 extracts the unauthorized person face image data using the video stored in the video storage device 12. The face image data may be cut out in real time based on the video to be recorded. That is, when information indicating the possibility of fraud is output from the hall computer, the face image data of the fraudster is cut out using the video from the monitoring camera 11 at that time. In this case, since the offline face matching unit 24 performs face matching processing at a predetermined timing designated by the user, the unauthorized person face image data extracted by the unauthorized person face image data extracting unit 23 is the unauthorized person face image. The data extraction unit 23 temporarily holds the data.

  As described above, according to the monitoring support system of the first embodiment, the best shot face image data and the best shot face image data of the same person at different locations are obtained based on the plurality of images obtained at different locations. Best shot face image data extraction unit for extracting time, unauthorized person face image data extraction unit for extracting unauthorized person face image data, and different locations of target persons extracted by best shot face image data extraction unit If the best shot face image data of the person and the face image data of the unauthorized person are collated, and the best shot face image data of the person and the face image data of the unauthorized person exceed a predetermined match rate, the best shot face image data of the person Since it has an offline face matching unit that outputs shot face image data as a candidate for fraudulent face image data, for example, the best shot face image of the fraudster By referring to the camera number and shooting time that accompanies the data, it is possible to obtain behavior records such as the time of unauthorized persons entering the store, the time of leaving the store, the game, and the time of shooting the face image while moving. The scope of examining the recorded video at the time of discovery is limited, and the work burden on employees and operators can be reduced. Furthermore, since the collation result is output off-line with the assistance of employees and operators, oversight can be reduced.

  In addition, according to the monitoring support system of the first embodiment, it includes a video storage device that stores images similar to the plurality of images input to the best shot face image data extraction unit, and the unauthorized person face image data extraction unit includes: Since the face image data of the unauthorized person is extracted from the video of the video storage device based on the signal indicating the predetermined fraud, the unauthorized person face image data extracting unit Extraction can be performed more reliably.

Embodiment 2. FIG.
In the second embodiment, an unauthorized person list is prepared in advance, and an online face collation unit that performs collation processing each time the best shot face image data is extracted is added.

FIG. 2 is a configuration diagram of the monitoring support system according to the second embodiment.
The illustrated monitoring support system includes a video monitoring system 1 and an image analysis device 2a. Since the video monitoring system 1 is the same as the video monitoring system 1 of the first embodiment, description thereof is omitted here.

  The image analysis apparatus 2a includes a best shot face image data extraction unit 21, a visitor list generation unit 22, an unauthorized person face image data extraction unit 23, an offline face collation unit 24, an input / output unit 25, an unauthorized person list generation unit 26, an online A face matching unit 27 is provided. Here, since the configurations of the best shot face image data extraction unit 21 to the input / output unit 25 are the same as those in the first embodiment, description thereof is omitted here.

  The fraudster list generation unit 26 is a functional unit that adds a camera number and a shooting time to the face image data obtained by the fraudster face image data extraction unit 23 and holds it as an unauthorized person list. In this unauthorized person list generation unit 26, every time new unauthorized person face image data is obtained, it is added to the unauthorized person list and updated every moment.

  Further, as a result of the offline face matching in the offline face matching unit 24, when the best shot face image data of the unauthorized person exists in the customer list, the best shot face image data of the customer list generating part 22 is used as the unauthorized person face. The image data may be added to the unauthorized person list, or the unauthorized person face image data may be updated with the best shot face image data.

  The online face collation unit 27 refers to the fraudster list stored in the fraudster list generation unit 26 every time the best shot face image data extraction unit 21 extracts the best shot face image data, and has a predetermined matching rate. Is obtained, it is determined that an unauthorized person appears in the video from which the best shot face image data is extracted, and the determination result is output to the input / output unit 25.

Next, the operation of the second embodiment will be described.
The unauthorized person list generating unit 26 records and updates the unauthorized person face image data extracted by the unauthorized person face image data extracting unit 23 or the best shot face image data determined to be an unauthorized person by the offline face matching unit 24.

  In addition, the store entrance monitoring camera 111 to the gift exchange counter monitoring camera 114 of the monitoring camera 11 acquire videos at different places such as when visiting the store, leaving the store, playing, and moving, and the best shot face image data extraction unit 21 extracts the best shot face image corresponding to each place.

When the best shot face image data is input from the best shot face image data extraction unit 21, the online face collation unit 27 reads the fraudster list from the fraudster list generation unit 26 and performs collation with the best shot face image data.
When face matching is performed on M fraudulent person face image data groups and one best shot face image data, as a result, M similarities are obtained. By setting the system threshold value as T2 in advance via the input / output unit 25, out of M similarities, B unauthorized person face image data exceeding T2 are extracted. If at least one fraudulent face image data is extracted (B ≧ 1), it is determined that the customer extracted as the best shot face image data is likely to be a fraudster, and the determination result is input as a fraudster candidate. Output to the output unit 25. That is, it is determined that an unauthorized person has visited the store again, and the input / output unit 25 issues an alarm. The offline face matching process is the same as that in the first embodiment, and a description thereof is omitted here.

  As described above, according to the monitoring support system of the second embodiment, when the best shot face image data is obtained by the best shot face image data extraction unit, the best shot face image data and the unauthorized person provided in advance are obtained. The online face matching accuracy is improved by providing an online face matching unit that outputs the person of the best shot face image data as a fraud candidate when a predetermined matching rate is obtained by matching the list. Can be missed and false alarms can be reduced.

  In addition, according to the monitoring support system of the second embodiment, the fraudster list is generated based on the face image data extracted by the fraudster face image data extraction unit. Human face image data can be added as appropriate, and the accuracy of online face matching can be further improved.

  Further, according to the monitoring support system of the second embodiment, the fraudster list is generated based on the best shot face image data extracted as the fraudster face image data candidates by the offline face collation unit. The fraudster face image data in the fraudster list can be updated as appropriate, and the accuracy of online face collation can be further improved in this respect.

  In the second embodiment, as the unauthorized person list, the unauthorized person face image data extracted by the unauthorized person face image data extracting unit 23 or the best shot face image determined by the offline face matching unit 24 as an unauthorized person. Although the data is used, for example, face image data of a person who has never visited the store but has been identified as an unauthorized person, such as unauthorized person data at another store, may be included. If comprised in this way, the unauthorized person who has never visited a store can also be discovered, and it can aim at the improvement of reliability as a monitoring assistance system.

  In the first and second embodiments, the case where the monitoring support system is applied to the amusement store has been described. However, when applied to other supermarkets, convenience stores, etc., the unauthorized person face image data extracting unit 23 is used. For example, a signal from a shoplifting sensor can be used as a signal for detecting an unauthorized person.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the monitoring assistance system by Embodiment 1 of this invention. It is a block diagram which shows the monitoring assistance system by Embodiment 2 of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 Surveillance camera, 21 Best shot face image data extraction part, 23 Unjust person face image data extraction part, 24 Offline face matching part, 26 Unjust person list generation part, 27 Online face matching part

Claims (6)

Based on a plurality of images obtained for different places, a best shot face image data extraction unit that extracts the best shot face image data of the same person for each different place and the acquisition time of the best shot face image data;
An unauthorized person face image data extraction unit for extracting unauthorized person face image data;
The best shot face image data of the target person extracted by the best shot face image data extraction unit is collated with the face image data of the unauthorized person, and the best shot face image data of the person and the unauthorized person are collated. A monitoring support system comprising an offline face matching unit that outputs the best shot face image data of the person as a candidate for the unauthorized person's face image data when the person's face image data exceeds a predetermined matching rate. A video storage device that stores images similar to the plurality of images input to the best shot face image data extraction unit;
2. The monitoring according to claim 1, wherein the fraudulent face image data extraction unit extracts fraudulent face image data from the video of the video storage device based on a signal indicating a predetermined fraud. Support system.   When the best shot face image data is obtained by the best shot face image data extraction unit, the best shot face image data is compared with a preliminarily provided fraudster list, and a predetermined match rate is obtained. The monitoring support system according to claim 1, further comprising an online face collation unit that outputs a person of the best shot face image data as a fraud candidate.   4. The monitoring support system according to claim 3, wherein the fraudster list is generated based on the face image data extracted by the fraudster face image data extraction unit.   5. The monitoring support system according to claim 3, wherein the fraudster list is generated based on best shot face image data extracted as a fraudster face image data candidate by an off-line face collation unit.   The monitoring support system according to any one of claims 3 to 5, wherein the unauthorized person list is generated based on face image data of an unauthorized person acquired from the outside.

Images