JP2018151840A - System, method and program for collation - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide means for easily confirming whether or not detection has been properly done as an identical person.SOLUTION: A collation system has a detector that collates detection face data that is a face image of a detection-object person or a feature quantity thereof with input face data that is an inputted face image or a feature quantity thereof so as to detect input face data of an identical person to the detection-object person, a counter that counts for each detection-object person a detection frequency of input face data of an identical person to the detection-object person of the detector, and a display controller that displays a person-juxtaposing screen of a detection-object person whose detection frequency is plural in the number of times. The person-juxtaposing screen is a screen on which a plurality of person images E of the input face data the detector has detected as an identical person to the detection-object person are arranged.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a verification system, a verification method, and a verification program.

  A system has been proposed in which a person is photographed at a store, a facility, or the like, and various services are performed using the captured image data.

  In the store visitor management system of Patent Document 1, the number of store visitors is tabulated and managed based on the captured face image of the store visitor. Further, based on the captured face image data of the visitor, it is determined whether or not the visitor is a notification-required visitor, and when the notification-required visitor visits the store, the fact is notified.

  In the face image management device of Patent Document 2, a player's face image is photographed at the store entrance, in-store and store exit, and a suspicious person flag is registered based on the photographed data. For example, the suspicious person flag is registered when the store entrance / exit is repeated 3 times or more per hour, the store is repeated 10 times or more per day, or the toilet is entered / exited 5 times or more per hour.

  In the personal image providing system of Patent Document 3, a person playing in an amusement facility is photographed, and an image in which he / she appears with a natural expression is retrieved and obtained from images in which an unspecified number of subjects are captured. Can do.

JP 2008-287658 A Japanese Patent No. 5824806 Japanese Patent No. 3992909

  In order to detect a suspicious person, for example, there is a desire to detect a person who repeatedly enters a store or the like in a short time. Such detection can be realized by face matching using a photographed person image. However, when repeated visits of the same person are detected by face matching, other persons may be mistakenly determined to be the same person, so that the store clerk visually recognizes the detected face image and they are the same person. It is preferable to confirm whether or not there is.

  The store visitor management system of Patent Document 1 is a system that counts the number of store visitors, and determines whether or not the imaged store visitor has visited the store during the counting period. Add to. That is, it is determined whether or not the imaged store visitor is the same as the person who visited the store in the past, but there is no means for the system user to confirm whether or not the determination is appropriate.

  In the face image management device of Patent Document 2, when there are multiple visits or the like based on the shooting data of the player, the face image management device is registered as a suspicious person. However, there is no means for the user of the system to check whether or not the photographing data determined by the face image management apparatus as having visited the same person a plurality of times is from the same person.

  In the personal image providing system of Patent Document 3, an image searched based on facial features is displayed in order to select one's own photograph. However, the images listed are displayed in thumbnail format, which is a reduced version of the original image so that you can see what each image is as a whole. It is difficult for a user of the system to check whether or not the same person is the same person.

  As described above, the conventional apparatus or the like is not provided with means for easily confirming whether or not the detection of the same person is appropriate for the user of the system. In order to confirm whether or not the same person has been detected multiple times, it is necessary to display and confirm past detection histories one by one, which is very laborious.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide means for easily confirming whether or not the detection of the same person is appropriate.

[Configuration 1]
The collation system of the present invention is
The detected face data that is the face image of the detection target person or its feature value is compared with the input face data that is the input face image or its feature quantity, and the input face data of the same person as the detection target person is detected. A detector to perform,
A counting unit that counts the number of detections of the input face data of the same person as the detection target person by the detection unit;
A display control unit for displaying a person parallel screen for the detection target person whose detection count is a plurality of times,
The person parallel screen is a screen in which a plurality of person images related to the input face data detected by the detection unit as the same person as the detection target person are arranged side by side.

[Configuration 14]
The collation method of the present invention is:
The detected face data that is the face image of the detection target person or its feature value is compared with the input face data that is the input face image or its feature quantity, and the input face data of the same person as the detection target person is detected. Detecting step to
A counting step of counting the number of detections of the input face data of the same person as the detection target person by the detection unit for each detection target person;
A display step of displaying a person parallel screen for the detection target person whose detection count is a plurality of times,
The person parallel screen is a screen in which a plurality of person images related to the input face data detected by the detection unit as the same person as the detection target person are arranged side by side.

[Configuration 15]
The collation program of the present invention is:
The detected face data that is the face image of the detection target person or its feature value is compared with the input face data that is the input face image or its feature quantity, and the input face data of the same person as the detection target person is detected. Detection process to
A counting process for counting the number of detections of the input face data of the same person as the detection target person by the detection unit for each detection target person;
A verification program for causing a computer to execute a display process for displaying a person parallel screen for the detection target person whose detection count is a plurality of times,
The person parallel screen is a screen in which a plurality of person images related to the input face data detected by the detection unit as the same person as the detection target person are arranged side by side.

  According to the above-described configuration, the person parallel screen is displayed when the number of detections is a plurality of times, and the person parallel screen includes a plurality of person images that are images related to the input face data detected as the same person as the detection target person. Therefore, it can be easily confirmed whether or not the detection of the face data as the same person is appropriate.

[Configuration 2]
Moreover, the collation system of this invention may be comprised so that the said display control part may display the said person parallel screen, when the said frequency | count of detection reaches a predetermined detection threshold value.

  According to the above-described configuration, the person parallel screen is displayed when the number of detections reaches a predetermined detection threshold value, so that it is further easier to confirm whether or not the detection of the same person is appropriate.

[Configuration 3]
In the verification system of the present invention, the counting unit may be configured to count the number of times of detection within a predetermined period.

  According to the above configuration, since the counting unit counts the number of detections within a predetermined period, it can be confirmed whether or not the detection of the same person within the predetermined period is appropriate.

[Configuration 4]
Moreover, the collation system of this invention may be comprised so that the person's face regarding the said input face data by which the said person image is detected as the same person as the said detection object person may be included.

  According to the configuration described above, since the person image includes the face of the person related to the input face data detected as the same person as the person to be detected, it is easier to compare the images, and whether or not the detection as the same person was appropriate. This can be confirmed more easily.

[Configuration 5]
Moreover, the collation system of this invention may be comprised so that the upper body of the person regarding the said input face data in which the said person image was detected as the same person as the said detection object person is contained.

  According to the above configuration, since the person image includes the upper body of the person related to the input face data detected as the same person as the person to be detected, it is possible to check the clothes and body shape of the person, and the detection as the same person can be performed. It becomes easier to confirm whether it was appropriate.

[Configuration 6]
Moreover, the collation system of this invention may be comprised so that the said person image may be an image containing the whole body of the person regarding the said input face data detected as the same person as the said detection object person.

  According to the above-described configuration, since the person image includes the whole body of the person related to the input face data detected as the same person as the detection target person, the person's whole body clothes, body shape and height can be confirmed, and the same person It is further easier to confirm whether or not the detection is appropriate.

[Configuration 7]
In the collation system of the present invention, the plurality of person images arranged side by side on the person parallel screen have substantially the same size as the person related to the input face data detected as the same person as the person to be detected. It may be configured to be displayed.

  According to the above-described configuration, since the person related to the input face data detected as the same person as the person to be detected is displayed in substantially the same size, the comparison of images is further facilitated, and the detection as the same person is appropriate. It can be more easily confirmed whether or not there has been.

[Configuration 8]
In the collation system of the present invention, the plurality of person images arranged side by side on the person parallel screen have substantially the same size of the face of the person related to the input face data detected as the same person as the person to be detected. The person image may be arranged so that the face of the person related to the input face data detected as the same person as the detection target person on the person parallel screen has substantially the same height. .

  According to the above-described configuration, since the face of the person related to the input face data detected as the same person as the person to be detected on the person parallel screen is displayed with substantially the same size and height, it becomes easier to compare the images. It can be further easily confirmed whether or not the detection as the same person is appropriate.

[Configuration 9]
In the collation system of the present invention, the person parallel screen is a face image related to the input face data detected as the same person as the detection target person, and the face image with the earliest shooting time and the latest shooting time are It may be configured to be a screen on which at least one face image is arranged.

  The configuration described above has the following advantages. When the face image with the earliest shooting time is arranged, it is possible to grasp how collation is performed compared to other person images. When the face image with the latest shooting time is arranged, the face image is the latest face image related to the person to be detected, which is useful when actually searching for the person. That is, the convenience of the collation system can be improved by the above-described configuration.

[Configuration 10]
The collation system of the present invention further includes a notification unit that performs notification when the number of detection times reaches a predetermined notification threshold within a predetermined period, and the display control unit is configured to display the person when the notification is performed. You may be comprised so that a parallel screen may be displayed.

  According to the above configuration, when the number of detections reaches a predetermined notification threshold within a predetermined period, a person parallel screen is displayed when the notification is performed. Can check the person parallel screen, and the convenience of the collation system can be improved.

[Configuration 11]
Moreover, the collation system of this invention may be comprised so that it may further have an input reception part which receives the input of whether the warning of a suspicious person detection is issued when the said alerting | reporting part alert | reports.

  According to the above-described configuration, the input reception unit issues an alarm for detecting a suspicious person after receiving an input as to whether or not to issue an alarm.

[Configuration 12]
Further, the collation system of the present invention is configured to include an exclusion receiving unit that receives an input indicating that the input face data related to the person image arranged on the person parallel screen is excluded from being counted by the counting unit. May be.

  According to the above-described configuration, the exclusion receiving unit receives an input indicating that the input face data related to the person image is excluded from the target of counting, and thus an inappropriate person image (for example, clearly another person image from another person image) Can be excluded from the object of counting.

[Configuration 13]
The collation system of the present invention further includes a display unit that is controlled by the display control unit to display the person parallel screen, and the display unit is configured to be a display screen of a portable terminal device. Also good.

  According to the above-described configuration, since the face of the detection target person is displayed with substantially the same size and height on the person parallel screen, it is easier to compare the images, and whether or not the detection as the same person is appropriate. This can be confirmed more easily.

  ADVANTAGE OF THE INVENTION According to this invention, the means which can confirm easily whether the detection as the same person was appropriate can be provided.

Diagram showing the configuration of the verification system The figure which shows the example of person parallel image Explanatory drawing of person image generation method by image generation unit Flow chart showing processing procedure of collation method Flow chart showing processing procedure of collation method

(First embodiment)
Hereinafter, a collation system, a collation method, and a collation program according to the present embodiment will be described with reference to the drawings. These systems are applied to, for example, stores such as amusement stores, retail stores, banks, and facilities such as city halls and libraries, and can be used to detect suspicious persons who repeatedly enter these in a short time. . And a collation system, a collation method, and a collation program display a person parallel screen as a means which can check easily whether the detection as the same person was suitable about the above-mentioned detection.

  In this embodiment, a case where a matching system is installed in a store and a person who repeatedly enters is detected will be described as an example. Specifically, a person entering the store is photographed, and the photographed data is sequentially stored. At the same time, the face image of the photographed person and the stored photographing data are collated, and repeated entrance of the same person is detected. When a plurality of entrances are detected, the attendant is notified of this, and a person parallel screen is displayed on the office monitor or the attendant's smartphone.

  As shown in FIG. 1, the verification system 1 includes a camera 10, a controller 20, and a display input device 30.

  The camera 10 is a camera that captures still images or moving images. As shown in FIG. 1, the camera 10 is placed at the entrance A of the store and photographs a person B who enters the store through the entrance A. The captured image (captured image) is transmitted to the controller 20.

  The controller 20 controls the overall operation of the verification system 1 and performs various processes. The controller 20 includes a control unit 21, a storage unit 22, and a communication unit 23.

  The control unit 21 includes an input unit 21a, a detection unit 21b, a counting unit 21c, an image generation unit 21d, a display control unit 21e, an input reception unit 21f, an alarm control unit 21g, and an exclusion reception unit 21h. The control unit 21 includes, for example, a software program for realizing various processes, a CPU that executes the software program, and various hardware controlled by the CPU. Software programs and data necessary for the operation of the control unit 21 are stored in the storage unit 22, but the storage destination is not particularly limited, and is stored in a storage device such as a disk or a flash memory provided separately. There may be a mode in which data is stored in a server or a storage device that is connected to be communicable.

  The input unit 21a performs processing on the face image input to the collation system 1 or input face data that is a feature amount thereof. In the present embodiment, the input unit 21a extracts, from the captured image captured by the camera 10 (that is, data input to the verification system 1 by the camera 10), the captured face image of the person B and its feature amount by image processing. And stored in the storage unit 22. The feature amount is an amount obtained by quantifying the feature of a person's face in the face image. Hereinafter, the “face image and its feature amount” may be referred to as “face data”.

  Specifically, when the store is opened due to opening time, shooting by the camera 10 and processing by the input unit 21a are started. The face data of the person B generated by the input unit 21a is sequentially stored in the storage unit 22 as detected face data of the person to be detected, and is subject to collation by the detection unit 21b as input face data each time.

When the image captured by the camera 10 is a moving image or when a plurality of still images are shot at short time intervals, the input unit 21a generates one input face data for one entry of the person B. It is preferable to be configured to do so. For example, if it takes about 5 seconds for the person B to pass through the shooting location, a short time interval is set to 10 seconds with a margin, and even if the person B is detected a plurality of times, 1 is detected within 10 seconds. Detected once. This time interval may be set automatically by analyzing the moving speed of the person, or may be set arbitrarily by the staff.
In addition, when selecting one of a plurality of input face data candidates, the selection may be made based on the quality of the face image such as the resolution, sharpness, and face orientation of the detected face. May be selected at a timing such as the first, last, or middle of the interval, may be selected at a position in the captured screen, or may be selected in combination.

  The detection unit 21b collates the detected face data that is the face image of the detection target person or its feature quantity with the input face image or the input face data that is the feature quantity thereof, and inputs the same person as the detection target person. Detect data.

  The counting unit 21c counts, for each detection target person, the number of detections of input face data of the same person as the detection target person by the detection unit 21b from a predetermined start time to the current time in a predetermined period. The predetermined period can be arbitrarily set by the user of the verification system 1. For example, if the start time is set to the store opening time (for example, 10:00), it is possible to detect repeated entrances from the opening to the present time. For example, if the start time is set 60 minutes before the present time, it is possible to detect a person who has repeatedly entered the store for 60 minutes.

  The image generation unit 21d generates a person image and a person parallel image from the captured image captured by the camera 10. That is, the image generation unit 21d generates an image to be displayed on the person parallel screen from the captured image. The person image is an image related to input face data detected as the same person as the person to be detected, and the person image includes input face data detected by the detection unit as being the same person as the person to be detected after the start time. Is included. The person parallel image is an image related to input face data detected as the same person as the person to be detected, and is an image in which a plurality of the above-described person images are arranged side by side.

  The display control unit 21e displays a person parallel screen for a detection target person whose detection count is a plurality of times. The person parallel screen is a screen on which a plurality of person images related to input face data detected by the detection unit 21b as the same person as the detection target person are arranged side by side. Specifically, the display control unit 21e controls the display input device 30 to display the person parallel image generated by the image generation unit 21d on the display input device 30. The display control unit 21e transmits a plurality of person images to the display input device 30 without generating the person parallel images, and the display control for displaying them in parallel or the display for performing the same display as the person parallel images. Control is also possible.

  The input reception unit 21f receives an input as to whether or not to issue a warning for detecting a suspicious person when the notification unit (display input device 30) performs notification. For example, the input receiving unit 21f controls the display input device 30 to display an input screen as to whether or not to issue a suspicious person detection alarm when the display input device 30 notifies an attendant or the like. When a staff member or the like performs an input indicating that “alarm is to be issued” to the display input device 30, the alarm control unit 21g performs control to generate an alarm.

  The alarm control unit 21g issues an alarm for detecting a suspicious person when the input receiving unit 21f receives an input indicating that “alert an alarm”. For example, the alarm control unit 21g controls the display input device 30, a wireless communication device for an attendant, and issues an alarm for detecting a suspicious person to the attendant.

  The exclusion receiving unit 21h receives an input indicating that the input face data related to the person image arranged on the person parallel screen is excluded from the objects to be counted by the counting unit 21c. For example, the exclusion receiving unit 21h waits for an input from the display input device 30 when the display input device 30 displays the person parallel screen. When an attendant or the like operates the display input device 30 to select one (or a plurality of) person images and performs an exclusion operation, the exclusion receiving unit 21h counts input face data related to those person images. The control which excludes from the object of the count by the part 21c is performed. Then, the counting unit 21c excludes the corresponding input face data and counts the number of detections.

  The storage unit 22 includes a storage device such as an HDD (Hard Disk Drive) or a nonvolatile RAM (Random Access Memory), and stores detected face data, input face data, captured images, person images, person parallel images, and the like.

  The communication unit 23 is an interface unit for performing data communication with each component of the verification system 1 via a wireless LAN, a wired LAN, or the like.

  The display input device 30 is, for example, a desktop personal computer, a touch panel type liquid crystal monitor, a smartphone, or the like, and is a device that displays a screen and receives an input from an attendant or the like. That is, the display input device 30 includes a display screen of a portable terminal device. The display input device 30 functions as a display unit that displays the person parallel screen under the control of the display control unit 21e. Further, the display input device 30 functions as a notification unit that performs notification when the number of detections reaches a predetermined notification threshold within a predetermined period. Notification to the staff by the display input device 30 can be performed by, for example, sound, voice, display on a screen, vibration, or the like. The detection threshold and the notification threshold are arbitrarily set by the user of the verification system 1.

  Hereinafter, operation | movement of the collation system 1 is demonstrated along a specific example.

  It is assumed that 35 people enter the store from the opening time 10:00 to the current time 12:10. During this time, the camera 10 captures these 35 persons, the input unit 21a creates face data X1 to X35, and the storage unit 22 stores these face data X1 to X35.

  Assume that person B1 enters the store at time 12:10. Similarly, the camera 10 captures the person B1, the input unit 21a creates the face data X36, and the storage unit 22 stores the face data X36.

  Here, the detection unit 21b collates the face data X36 of the person B1 who recently entered the store with the face data X1 to X36 stored in the storage unit, and detects the face data of the same person as the person B1. For example, assume that five face data X16, X20, X28, X32, and X36 are detected (the face data X36 is naturally detected because it is the face data of the person B1).

  Assume that the verification system 1 is set to detect repeated entry within 60 minutes. Then, the “start time” related to the operation of the counting unit 21c is 11:10 60 minutes before 12:10. The detection unit 21b counts the number of detections after the start time 11:10 for the five detected face data. It is assumed that the generation time of X16 (that is, the shooting time) is 11:13, X20 is 11:21, and similarly the generation time after 11:10. Then, the counting unit 21c counts the number of detections related to the person B1 who recently entered the store as “5”.

  Here, from face data X1 to X35, those whose generation time (shooting time) is within 60 minutes (face data generated within a predetermined period) are extracted, and detection processing is performed on them. Is also possible.

  It is assumed that the collation system 1 is set to display a person parallel screen and notify when a store is repeatedly detected five times. That is, in this specific example, the detection threshold value and the notification threshold value are “5”. Since the number of detections has reached “5”, the image generation unit 21 d generates a person parallel image in order to display the person parallel image on the display input device 30. Then, the generated person parallel image is transmitted from the controller 20 to the display input device 30, and the person parallel screen is displayed on the screen of the display input device 30. At the same time, the display input device 30 notifies the store staff and the like of the repeated entry detection.

  When the detection threshold is set to “2”, the person parallel screen is displayed on the display input device 30 when the number of detections is “2”, that is, when the number of detections is multiple. For example, when the person B1 has entered the store once in the past (at this time, the face data of the person B1 is sent to the storage unit as detection target data and becomes detected face data), and the next entry, The number of detections is “2” at the store and the current store, and the person parallel screen is displayed on the display input device 30. When the number of detections reaches “5” and reaches the notification threshold, notification is performed.

  When the person parallel screen is displayed on the screen of the display input device 30 and the above notification is made, the input receiving unit 21f displays an input screen as to whether or not to issue a suspicious person detection alarm, and a suspicious person detection alarm. It waits for the input of whether to issue. When a staff member or the like performs an input indicating that “alarm is to be issued” to the display input device 30, the alarm control unit 21g performs control to generate an alarm. For example, the warning control unit 21g displays a warning screen indicating that a suspicious person is detected on the display input device 30 of the clerk, or transmits a notice of suspicious person detection to a clerk or the like using a wireless communication device for the clerk.

  When a person parallel screen is displayed on the display input device 30, if an attendant or the like operates the display input device 30 to select one (or a plurality of) person images and performs an exclusion operation, an exclusion receiving unit 21h Performs control for excluding the input face data related to those human images from the objects of counting by the counting unit 21c. If it does so, the counting part 21c will exclude the input face data concerned, will count the frequency | count of detection, and the above-mentioned operation | movement of the collation system 1 will be performed based on the result. At this time, with respect to the input face data related to the selected person image, the detection unit 21b may be set so that input face data having a high similarity to the input face data is not detected as the detection target person.

  As described above, in the collation system 1, the detection unit 21b detects the face data X36 (detected face data) of the person B1 (detection target person) who recently entered the store and the stored face data X1 to X36 (input face data). ) And the input face data of the same person as the detection target person is detected. Then, the counting unit 21c counts the number of detections of input face data of the same person as the detection target person after a predetermined start time (11:10) by the detecting unit 21b. Then, the display control unit 21e causes the display input device 30 (display unit) to display a person parallel screen for the person to be detected when the number of detection times becomes plural.

  Next, a person parallel image (person parallel screen) generated by the image generation unit 21d and displayed by the display input device 30 will be described. FIG. 2 shows an example of the person parallel image C.

  On the left side of the person parallel image C, face images D1 and D2 are arranged side by side. From the center to the right side of the person parallel image C, five person images E1 to E5 are arranged side by side. Each of these images is an image related to the person B1 who recently entered the store, and more specifically, is an image of a person related to the input face data detected as the same person as the detection target person.

  In the present embodiment, the face image D1 is the face image with the earliest shooting time among the face images of the person B1, that is, the image of the face data X16. The face image D2 is the face image with the latest shooting time among the face images of the person B1, that is, the face data X36 of the person B1 that has been shot recently.

  Each of the five person images E1 to E5 is an image including five face data X16, X20, X28, X32, and X36 detected by the detection unit 21b as being the same person as the person B1. In the person parallel image C, the person images E1 to E5 are arranged in order from the left in order of the generation time (that is, the photographing time) of the detected face data.

  The five person images E1 to E5 include the face of the person B1, and they are displayed with substantially the same size. In the person parallel image C, the person images E1 to E5 are arranged so that the face of the person B1 has substantially the same height.

  In the present embodiment, the five person images E1 to E5 include the whole body of the person B1, that is, the upper half of the person B1. In the five person images E1 to E5, the person B1 is displayed with substantially the same size.

  When the camera 10 captures the person B1, the size of the face and the whole body of the person B1 in the captured image of the camera 10 changes according to the distance between the camera 10 and the person B1. When the distance between the camera 10 and the person B1 is large (that is, when the person B1 is far), the size of the face / whole body in the captured image is small, and when the distance is small (when close), the size of the face / whole body in the captured image is large. It gets bigger. In the person parallel image C, if the displayed face / body size / position of the person B1 is different among the plurality of person images E, it is difficult to confirm whether the person B1 belongs to the same person. Therefore, in the present embodiment, when the image generation unit 21d generates a person image and a person parallel image, the size and position of the face / whole body of the person B1 displayed are adjusted.

  A method in which the image generation unit 21d generates the person image E from the captured image F of the camera 10 will be described with reference to FIG. Although three persons are shown in the photographed image F on the left side in the figure, a case where a person image E of the foremost person B1 is generated will be described. First, the image generation unit 21d recognizes the face of the person B1 by image processing and sets the face detection frame G and the face center H. Next, the image generation unit 21d acquires the length in the vertical direction of the face detection frame G, and sets the position moved from the face center H downward in the vertical direction as the extraction center K. Then, the image generation unit 21d enlarges or reduces the captured image F so that the vertical length described above becomes a predetermined number of pixels (for example, 50 pixels). Then, the image generation unit 21d performs trimming of the captured image F with a cutout frame J (for example, 240 pixels × 320 pixels) having a predetermined size with the cutout center K as a reference, and further in the person parallel image. The image is trimmed to a predetermined width (for example, 85 pixels) according to the display width, and a human image E on the right side in the figure is generated.

  As described above, the person image E is generated based on the face center H (or the face detection frame G) of the person B1 detected in the captured image F and the size of the face detection frame G. The face / whole body of the person B1 can be aligned to a predetermined size, and the position of the face / whole body in the person image E can be made appropriate.

  Here, in the captured image F, when the position of the person B1 is biased upward, the cutout frame J protrudes above the captured image F. Then, there is a case where the upper side is cut off like the person image E3 in FIG. On the other hand, when the position of the person B1 is biased downward in the photographed image F, there may be a case where the lower side is cut like the person image E5 in FIG. Even in such a case, it is easy to check whether the person images E1 to E5 are the same person by arranging the face of the person B1 at substantially the same height.

  In the above description, the example of performing the face recognition and generating the person image E for the person B1 in the foreground in the photographed image F has been described, but from the same photographed image F, standing behind the person B1. It is also possible to perform face recognition of a male and a female and generation of a person image E.

  The collation method performed by the collation system 1 demonstrated above is demonstrated referring the flowchart of FIG.

  In the input step of step S <b> 101, the person B entering the store from the entrance A is photographed by the camera 10. The captured image (captured image) is transmitted to the controller 20. Then, the face data X of the person B is created by the input unit 21a. The created face data X1 is stored in the storage unit 22.

  In the detection step of step S102, the detection unit 21b collates the detected face data of the person to be detected or the feature amount thereof with the input face image or the input face data of the feature amount, and detects the detection target. Input face data of the same person as the person is detected. In the present embodiment, the face data (X1 to X36) stored in the storage unit 22 and the face data X36 of the person B1 who recently entered the store are collated, and the person who has entered in the past and the person B1 who has recently entered are the same. Whether it is a person is detected.

  In the counting step of step S103, the counting unit 21c counts the number of detections of input face data of the same person as the person to be detected after a predetermined start time in the detecting step.

  In step S104, it is determined whether the number of detections has reached a detection threshold. If the number of detections has not reached the detection threshold, the process returns to step S101. If the number of detections has reached the detection threshold, the process proceeds to step S105.

  In the image generation step of step S105, a person image and a person parallel image are generated by the image generation unit 21d. The person image is an image of a person to be detected, and the person image includes input face data detected by the detection unit as being the same person as the person to be detected after the start time. The person parallel image is an image about a detection target person, and is an image in which a plurality of the above-described person images are arranged side by side. Then, the process proceeds to step S106.

  In the display step of step S106, the display control unit 21e displays a person parallel image of the detection target person transmitted from the controller 20 on the display input device 30 (display of a person parallel screen). Then, the process ends.

(Second Embodiment)
In the collation system 1 according to the second embodiment, the storage of the detected face data in the storage unit 22 and the count of the number of detections by the counting unit 21c are performed in a manner different from the first embodiment. In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and the description may be omitted (the same applies to other embodiments).

  In the first embodiment described above, when the input unit 21a generates the face data X1 of the person B1 from the captured image of the camera 10, the face data is sent to the storage unit 22 and stored as detected face data of the detection target person. The data are sequentially stored in the unit 22. Then, the detection unit 21b sequentially collates the face data X1 and the detected face data, and the detected face data determined to be the same person as the person B1 is extracted. Based on the number of detected face data extracted, the counting unit 21c counts the number of detections.

  In the present embodiment, when the input unit 21a generates the face data X1 of the person B from the captured image of the camera 10, the detection unit 21b first collates the face data X1 and the detected face data. When the detected face data determined to be the same person as the person B1 is extracted, the counting unit 21c adds “1” to the number of detections related to the person B1. When the detection face data of the same person as the person B1 is not extracted, the counting unit 21c sets the number of detections related to the person B1 to “1” (that is, the first detection), and at this time, the face data X1 is the detection target person. The detected face data is stored in the storage unit 22. That is, in the present embodiment, the detected face data of the detection target person is stored in the storage unit 22 in a state where there is no overlap for the same person. The storage unit 22 stores a captured image of the camera 10 for generating a person image and a person parallel image.

  A collation method performed in the collation system 1 of the present embodiment will be described with reference to the flowchart of FIG.

  In the input step of step S <b> 201, the person B entering the store from the entrance A is photographed by the camera 10. The captured image (captured image) is transmitted to the controller 20. Then, the face data X1 of the person B is created by the input unit 21a. The created face data X1 is sent to the detection unit 21b.

  In the detection step of step S202, the detection unit 21b collates the detected face data of the detection target person or the feature amount thereof with the input face image or the input face data of the feature amount. That is, the detected face data stored in the storage unit 22 and the face data X1 (input face data) of the person B generated by the input unit 21a are collated, and the person to be detected, that is, the person who has entered in the past, It is detected if the selected person B is the same person.

  When the input face data of the same person as the detection target person is detected, that is, when the detection target person, that is, the person who entered in the past and the person B who recently entered are the same person (step S203: Yes), go to step S205 move on. When the input face data of the same person as the person to be detected is not detected, that is, when the person B1 has entered for the first time (step S203: Yes), the process proceeds to step S204.

  In the registration step of step S204, the face data X1 of the person B generated by the input unit 21a is registered in the storage unit 22 as detected face data of the detection target person. Then, the process proceeds to step S205.

  In the counting step of step S205, the counting unit 21c counts the number of detections of input face data of the same person as the person to be detected after the predetermined start time in the detecting step. Specifically, “+1” is added to the number of detections related to the person B who has recently entered the store.

  If step S203 is Yes, that is, input face data of the same person as the person to be detected is detected, person B1 has entered the store at least once in the past. Therefore, the number of times of detection should be 1 or more, and the number of times of detection increases by 1 due to the recent store entry / detection.

  When step S203 is No, that is, when input face data of the same person as the person to be detected is not detected, the person B1 is admitted for the first time. Therefore, the number of times of detection is “0”, and the number of times of detection becomes “1” by the recent store entry / detection.

  In step S206, it is determined whether the number of detections has reached a detection threshold. When the number of detections reaches the detection threshold (step S206: Yes), the process proceeds to step S207. If the number of detections has not reached the detection threshold (step S206: No), the process returns to step S201.

  In the image generation step of step S207, a person image and a person parallel image are generated by the image generation unit 21d as in step S105 of the first embodiment. The generated person parallel image is transmitted to the display input device 30. Then, the process proceeds to step S208.

  In the display step of step S208, the display control unit 21e displays a person parallel image of the detection target person transmitted from the controller 20 on the display input device 30 (display of a person parallel screen). Then, the process ends.

(Other embodiments)
(1) In the above-described embodiment, the case has been described in which the verification system 1 is applied to a closed space having an entrance, such as a store or a facility, and a person repeatedly entering from the entrance is detected. The verification system 1 can also be applied to a corner of an urban area or a region such as a store / facility, where a person entering the area is photographed and detected. The verification system 1 can also be applied to a person passing through a road, a passage, a sidewalk, or the like.

(2) In the above-described embodiment, a case has been described in which a person is photographed by the camera 10 to acquire input face data. The aspect of the collation system 1 is not limited to this, and it is also possible to apply to captured images such as moving images and still images that have already been saved. It is also possible to capture a captured image into the collation system 1 via a network or a storage medium and perform the same processing as in the above-described embodiment.

  In the above-described embodiment, an example in which the number of detections from a predetermined start time to the present time is described has been described. However, it is also possible to perform processing from an arbitrary first time to an arbitrary second time as a predetermined period. It is. In particular, when collation / detection processing is performed on a captured image, it is possible to set an arbitrary period as a predetermined period and perform processing on a captured image whose photographing time is included in the period.

  In the above-described embodiment, the person parallel screen is displayed when the number of detections reaches a predetermined detection threshold. By modifying this and detecting the entire captured image (for example, the above-described captured image), the person parallel screen may be displayed for the detection target person whose detection count has reached a predetermined detection threshold. Good. When there are a large number of person images arranged in parallel and cannot fit on the screen of the display input device 30, the images may be displayed over a plurality of pages or displayed in a form that can be browsed by scrolling. List display is also possible.

(3) In the above-described embodiment, the case where the person image includes the whole body of the person has been described as an example. As the person image, an image displaying the upper body of the person (above the waist) may be used. Alternatively, an image that displays a human bust (above the chest) may be used.

(4) In the above-described embodiment, the example in which the image generation step is performed when the number of detections reaches the detection threshold has been described, but the generation timing of the person image or the person parallel image is not limited to this. For example, a person image or a person parallel image may be generated every time the captured image F is generated by the camera 10 or each time a person is detected in the captured image F.

(5) When the person parallel screen is displayed on the display input device 30, an input regarding the subsequent processing may be accepted. For example, as options for the subsequent processing, (a) the display of the person is not performed after that, and (b) the person parallel screen is displayed every time the person's entrance is detected, the display input device 30 is displayed. It may be displayed and selected by a store clerk. Particularly, (b) is suitable for the following operation. For a person who has been detected and notified for a plurality of times for the first time, it may be dangerous to deal with immediately, so that the person can be registered in the verification system 1 and the person parallel screen displayed on the next visit to notify Like that.

  Each configuration illustrated in the above-described embodiment is functionally schematic and does not necessarily need to be physically configured as illustrated. That is, the form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed / integrated in an arbitrary unit according to various loads or usage conditions. Can be configured.

  As described above, the collation system according to the present invention can provide a means for easily confirming whether or not the detection as the same person is appropriate.

1: collation system 10: camera 20: controller 21: control unit 21a: input unit 21b: detection unit 21c: counting unit 21d: image generation unit 21e: display control unit 21f: input reception unit 21g: alarm control unit 21h: exclusion reception Unit 22: Storage unit 23: Communication unit 30: Display input device (display unit, notification unit)
A: Entrance B, B1: Person C: Person parallel images D1, D2: Face images E, E1 to E5: Person image F: Captured image G: Face detection frame H: Face center J: Cut frame K: Cut center

Claims (15)

The detected face data that is the face image of the detection target person or its feature value is compared with the input face data that is the input face image or its feature quantity, and the input face data of the same person as the detection target person is detected. A detector to perform,
A counting unit that counts the number of detections of the input face data of the same person as the detection target person by the detection unit;
A display control unit for displaying a person parallel screen for the detection target person whose detection count is a plurality of times,
The person parallel screen is a collation system in which a plurality of person images related to the input face data detected by the detection unit as the same person as the detection target person are arranged side by side.   The collation system according to claim 1, wherein the display control unit displays the person parallel screen when the number of detection times reaches a predetermined detection threshold.   The collation system according to claim 1, wherein the counting unit counts the number of times of detection within a predetermined period.   The collation system according to any one of claims 1 to 3, wherein the person image is an image including a face of a person related to the input face data detected as the same person as the person to be detected.   The collation system according to any one of claims 1 to 4, wherein the person image is an image including an upper body of a person related to the input face data detected as the same person as the person to be detected.   The collation system according to claim 1, wherein the person image is an image including a whole body of a person related to the input face data detected as the same person as the person to be detected.   7. The person related to the input face data detected as the same person as the person to be detected is displayed in substantially the same size in the plurality of person images arranged side by side on the person parallel screen. The collation system according to any one of the items.   In the plurality of person images arranged side by side on the person parallel screen, a person's face related to the input face data detected as the same person as the person to be detected is displayed in substantially the same size, and the person parallel screen The collation system according to any one of claims 1 to 7, wherein the person image is arranged so that a face of the person related to the input face data detected as the same person as the person to be detected has substantially the same height. .   The person parallel screen is a face image related to the input face data detected as the same person as the detection target person, and at least one of a face image with the earliest shooting time and a face image with the latest shooting time is arranged. The collation system according to claim 1, wherein the collation system is a displayed screen.   2. The information processing apparatus according to claim 1, further comprising a notification unit that performs notification when the number of detection times reaches a predetermined notification threshold within a predetermined period, wherein the display control unit displays the person parallel screen when the notification is performed. The collation system of any one of 1-9.   The collation system according to claim 10, further comprising an input receiving unit that receives an input as to whether or not to issue a warning of suspicious person detection when the notification unit performs notification.   12. The apparatus according to claim 1, further comprising: an exclusion receiving unit that receives an input indicating that the input face data related to the person image arranged on the person parallel screen is excluded from a target of counting by the counting unit. Collation system.   The display unit according to claim 1, further comprising a display unit that is controlled by the display control unit to display the person parallel screen, and the display unit is a display screen of a portable terminal device. Matching system. The detected face data that is the face image of the detection target person or its feature value is compared with the input face data that is the input face image or its feature quantity, and the input face data of the same person as the detection target person is detected. Detecting step to
A counting step of counting the number of detections of the input face data of the same person as the detection target person by the detection unit for each detection target person;
A display step of displaying a person parallel screen for the detection target person whose detection count is a plurality of times,
The person parallel screen is a collation method in which a plurality of person images related to the input face data detected by the detection unit as the same person as the detection target person are arranged side by side. The detected face data that is the face image of the detection target person or its feature value is compared with the input face data that is the input face image or its feature quantity, and the input face data of the same person as the detection target person is detected. Detection process to
A counting process for counting the number of detections of the input face data of the same person as the detection target person by the detection unit for each detection target person;
A verification program for causing a computer to execute a display process for displaying a person parallel screen for the detection target person whose detection count is a plurality of times,
The person parallel screen is a collation program in which a plurality of person images related to the input face data detected by the detection unit as the same person as the detection target person are arranged side by side.

Images