JP5631043B2 - Visitor notification system - Google Patents

Description

  The present invention is a store visitor notification system that uses face verification to compare a face image of a store visitor with a registered database and notify the registered store visitor when the image is registered.

  In a general store visitor notification system, an imaging camera is arranged toward the flow line of a store visitor, and a face image of the store visitor who has entered the entire imaging area of the image capture camera is taken per unit time. Then, the face image of the visitor is detected from the captured image, and when the face width in the captured image is equal to or larger than a predetermined size, a plurality of face images are acquired for the same person and supplied to the subsequent face collation process. Yes.

  For example, if 10 face images are required for the same person due to the performance of the face matching system, if the shooting speed of the imaging camera is set to 5 frames per second, the face image of the visitor is displayed on the back face for more than 2 seconds. Must be supplied to the verification process.

  However, customers usually walk around while looking around, and the face turns in any direction, up, down, left, or right during walking, and the brightness of the face is affected by the lighting present on the walking path. Changes every moment. Therefore, as described above, it is difficult to acquire a face image of the same person for 2 seconds or more under the same illumination condition within the entire imaging region range, and supply it to the subsequent face matching process. Had fallen.

  As a countermeasure, install an advertising display that draws the store's attention on the store's imaging path, stop the store visitor, and shoot the store on the shooting path from the side of the display for the same lighting. There is a face detection system that attempts to acquire more face images facing the front (in the direction of the imaging camera) under conditions (see Patent Document 1).

  In addition, when there is an openable door such as an automatic door on the customer's imaging passage, pay attention to the characteristics of the person that the face of the visitor faces forward when the door is opened. There is a biometric matching system that acquires a face image to acquire an image suitable for subsequent matching determination (see Patent Document 2).

  Furthermore, the acquired image from the imaging camera is divided into a plurality of areas, search conditions are set for each area, and processing that does not meet the conditions is omitted to improve the efficiency of face search processing, and face detection accuracy There is a face detection device that also improves (see Patent Document 3).

JP 2008-243095 A JP 2002-140699 A JP 2008-234578 A

  In the face detection system described in Patent Document 1, the customer's attention may be drawn at the start of the installation of the advertising display, and the visitor may stop. There is a problem that does not continue. Patent Document 1 also mentions a measure for preventing further tiredness, but it is difficult to continue the effect.

  The biometric matching system described in Patent Document 2 is effective for an open / close sliding door such as an automatic door. However, when the purpose of a visitor is clear, for example, in a glass-covered store, the front is opened at the time of opening the door. It is possible to face the direction of the target place such as reception. Furthermore, when the customer visits the door continuously in a continuous state, the customer never stops in front of the door and looks before it.

  The face detection device described in Patent Literature 3 selects various search conditions such as a face size condition that determines the size of a face to be detected, a face direction condition that determines the face direction to be detected, and an illumination condition. Thus, a suitable image is discriminated from the images of the imaging camera. However, the more severe the search conditions, the smaller the number of images that can be acquired as a result, and the more the conditions, the greater the processing load.

  In addition to these, (1) a method for collating by registering facial feature amounts calculated from a plurality of face orientations in advance, and (2) a method for collating by complementing variation in face orientation during walking. (3) A method of taking images of the same store visitor using a plurality of imaging cameras is conceivable. However, in (1) and (2), a lot of processing time is required to create new image data from a plurality of face images. Further, in (3), it is necessary to determine the identity of the store visitor image for each camera, and a lot of processing time is required. Therefore, the application to a use that requires real-time processing to notify the store visitor is not possible. Have difficulty.

  As described above, in the store visitor notification system in which the object to be face-matched is moving and the face-matching result is required to be notified on the spot, the above-described conventional example has almost the same environment. Thus, it is difficult to acquire the required number of face images facing the imaging camera in a short time, and it is also difficult to execute the complement processing of the acquired face images in a short time.

  Therefore, in view of the above-described problems, an object of the present invention is to provide a store visitor notification system having high face matching accuracy by acquiring many face images under the same environment.

The store visitor notification system according to the present invention is a store visitor notification system that compares a face image captured by an image capture camera with a database and notifies a registered store visitor when the image is captured by the image capture camera. An image acquisition unit, a face detection region setting unit that sets a plurality of predetermined regions of the imaging region as a face detection region, a face detection unit that detects a face image from the face detection region, and a face image detected by the face detection unit In such a case, the image acquisition unit includes an image acquisition speed control unit that increases a speed at which the captured image is acquired. In addition, the store visitor notification system of the present invention further includes a face detection area weighting unit that weights each of a plurality of predetermined areas set as the face detection area by the face detection area setting unit, and the image acquisition speed control unit includes the weighting Accordingly, the image acquisition unit increases the speed at which the captured image is acquired.

The store visitor notification system according to the present invention includes a face detection area setting unit that sets a plurality of predetermined areas of the imaging area as a face detection area, and an image acquisition unit that captures an image when a face image is detected by the face detection unit. By providing the image acquisition speed control unit that increases the speed of acquiring the image, it is possible to acquire many face images without significantly increasing the processing load, and the accuracy of face matching is improved. In addition, the store visitor notification system of the present invention further includes a face detection area weighting unit that weights each of a plurality of predetermined areas set as the face detection area by the face detection area setting unit, and the image acquisition speed control unit includes the weighting Accordingly, by increasing the speed at which the image acquisition unit acquires the captured image, many face images can be acquired without significantly increasing the processing load, and the accuracy of face matching is improved.

It is a lineblock diagram of a store visitor notification system concerning a premise technology. 1 is a configuration diagram of a store visitor notification system according to Embodiment 1. FIG. It is the figure which showed a mode that the imaging camera 1 installed toward the store entrance imaged a store visitor. It is a figure which shows the imaging area of an imaging camera. It is a figure which shows the relationship between the effective pixel number of an imaging camera, and an imaging surface size. It is a figure which shows Japanese head dimension data. It is a figure which shows the imaging area of an imaging camera. It is a block diagram of the store visitor notification system concerning Embodiment 2.

(Embodiment 1)
<Prerequisite technology>
The structure of the store visitor notification system according to the prerequisite technology of the present invention will be described with reference to FIG. As shown in FIG. 1, the store visitor notification system according to the base technology includes an imaging camera 1, an image acquisition unit 2, a face detection unit 4, a face tracking unit 6, a face image cutout normalization unit 7, and a feature point extraction unit 8. Prepare.

  The image acquisition unit 2 acquires a captured image from the imaging camera 1. The face detection unit 4 detects whether or not there is a face in the captured image. The face tracking unit 6 predicts the movement of the face image detected by the face detection unit, and assigns an identification number to each face image. The face image cutout normalization unit 7 extracts a predetermined number of face images assigned with identification numbers, corrects the face image inclination and size using these as reference points, and aligns the face images (normalizes them). ). The feature point extraction unit 8 extracts feature points from the normalized face image.

  The store visitor notification system includes a collation unit 9, a temporary collation unit 10, a notification unit 11, a registered face image database (DB) 12, and a temporary registration face image database (DB) 13. The feature points extracted by the feature point extraction unit 8 are provisionally registered for each identification number in the temporary registration face image DB 13. Further, the operator selects image data that the operator wants to register from the temporary registered face image DB 13, adds individual information (name, affiliation, etc.) to the registered image and registers it in the registered face image DB 12.

  The collation unit 9 collates the feature points extracted by the feature point extraction unit 8 with the registered face image DB 12. The collation unit 9 digitizes the image similarity and determines that the images match when the image exceeds a preset threshold. If they match, the notification unit 11 notifies the outside that they match. If they do not match, the extracted feature points are collated with the temporarily registered face image DB 13 in the temporary collation unit 10.

  Similar to the collation unit 9, the temporary collation unit 10 determines match / mismatch based on a preset threshold value. If they match, the image of the feature point is added to the temporarily registered face image DB. If they do not match, the image of the feature point is temporarily registered in the temporarily registered face image DB.

<Configuration>
The configuration of the store visitor notification system according to the present embodiment will be described with reference to FIG. In FIG. 2, the same components as those in the store visitor notification system according to the prerequisite technology shown in FIG. As shown in FIG. 2, the store visitor notification system according to the present embodiment includes a face detection region setting unit 3 that sets a face detection region in a captured image, in addition to the configuration of the store visitor notification system according to the prerequisite technology, And an image acquisition speed control unit 5 that controls the acquisition speed.

  That is, the store visitor notification system according to the present embodiment includes an image acquisition unit 2 that acquires a captured image of the imaging camera 1, and a face detection region setting unit that sets one or a plurality of predetermined regions of the imaging region as a face detection region. 3, a face detection unit 4 that detects a face image from the face detection region, and an image acquisition speed control that increases the speed at which the image acquisition unit 2 acquires a captured image when the face detection unit 4 detects a face image. Therefore, many face images can be acquired without greatly increasing the processing load, and the accuracy of face matching is improved.

  The face detection area setting unit 3 sets a face detection area in a part or a plurality of parts of the image output from the image acquisition unit 2. The image acquisition speed control unit 5 controls the image acquisition unit 2 that acquires a captured image from the imaging camera 1 to increase the number of image acquisitions per unit time when a face image is detected in the face detection region. The image acquisition interval of the image acquisition unit 2 may be narrowed, or the image distribution interval from the imaging camera 1 to the image acquisition unit 2 may be narrowed.

  Since the configuration other than this is the same as that of the store visitor notification system according to the base technology shown in FIG.

<Imaging camera settings>
FIG. 3 is a diagram showing a state in which the imaging camera 1 installed toward the entrance of the store photographs a visitor, with the upper half being a plan view and the lower half being a perspective view.

  The effective pixel width of the image pickup device of the image pickup camera 1 is 640 pixels (VGA), and the aspect ratio of the image pickup device is 3: 4 (VGA). If the pixel width required for face detection is 64 pixels and the human face width is 0.16 m, the imaging surface width H at the store entrance is 1.6 m and the imaging surface height V is 1.2 m ( FIG. 4). FIG. 5 shows the image pickup surface size at the store entrance when the number of image pickup elements is other than 64 pixels assigned to the face width. When the number of effective pixels of the image sensor is 1280 × 1024 (SXGA), the image pickup surface width H at the store entrance is 3.2 m, and the image pickup surface height V is 2.56 m. When the number of effective pixels of the image sensor is 1600 × 1200 (UXGA), the image pickup surface width H at the store entrance is 4.0 m, and the image pickup surface height V is 3.0 m. When the effective pixel number of the image pickup device is 1920 × 1080 (HDTV), the image pickup surface width H at the store entrance is 4.8 m, and the image pickup surface height V is 2.7 m. In the following description, the effective number of pixels of the image sensor is 640 × 480 (VGA).

  Based on general Japanese body data, the height of the entire imaging area at the entrance is determined and the face is taken from the entrance to the depth direction so that the visitor's face can be photographed accurately at the entrance. The installation height of the imaging camera 1 is determined by calculating whether it can be performed.

  FIG. 6 shows Japanese height and head dimension data. This data was quoted from “National Institute of Advanced Industrial Science and Technology, Makiko Kawauchi and Masaaki Mochimaru, Japanese Head Dimensions Database 2001, H16PRO-212”. According to this data, the average height of adolescent males is about 1.70 (m), the average total head height is about 0.23 (m), and the height of most Japanese is 1.88 (m) or less. It is. Therefore, the upper end of the imaging surface at the store entrance is set to 1.88 (m) from the floor. Since the imaging surface height V is 1.2 m as described above, the height h (m) from the floor to the lower end of the imaging surface may be set to 0.68 (m) (upper diagram in FIG. 7).

7, the height of the imaging camera 1 is H (m), the height from the entrance to the lower end of the imaging surface is h (m), and the planar distance from the entrance to the imaging camera 1 is D. (M) Let M (m) be the height to the visitor's chin. Then, assuming that the imaging camera 1 is installed in front of the entrance, the depression angle maximum value θ _max of the imaging camera 1 is tan ⁻¹ {(H−h) / D}, and the face imageable region L (m) in the depth direction. Becomes D × (M−h) / (H−h).

  The camera height H = 2.3 (m) and the plane distance D = 13 (m) from the entrance to the camera. From FIG. If 70 (m) and 0.23 (m), the height M = 1.47 (m) up to the visitor's chin, so the face imageable area L = 6.34 (m), and the imaging area It can be seen that the depth direction is limited to this range.

  The image acquisition unit 2 acquires images of the entire imaging area of the imaging camera 1 installed in this way. The entire imaging area includes an area suitable for subsequent face collation, but the illumination environment is There are also areas that are dark or where the illumination is shining locally on only part of the face. If the effective pixel width is about 640 pixels, if the imaging area is carefully selected when the imaging camera 1 is installed, the number of areas inappropriate for face matching as described above is not included so much. As the number of pixels increases to 1280 pixels and 1920 pixels, the proportion of the area inappropriate for face matching in the entire imaging area increases.

  Therefore, in the store visitor notification system according to the present embodiment, the face detection area setting unit 3 that sets the face detection area in a part or a plurality of parts in the entire imaging area is provided, so that the same environment can be used for the subsequent face collation processing. Supply the images acquired below. Furthermore, if the imaging area is limited to the face detection area, the face search area is also limited, so that the load of the face search process per acquired image can be greatly reduced.

  While setting the face detection area has such advantages, there is also a problem that the area in the depth direction where the face image can be acquired is limited by limiting the imaging area. As shown in FIG. 3, since the face imageable area is limited to the set detection area, if the number of acquired images per unit time is the same as before, the number of acquired face images decreases. End up.

  By providing the face image acquisition speed control unit 5 as a countermeasure, the number of image acquisitions per unit time is increased when a face image exists in the area set by the face detection area setting unit 3. The image acquisition interval of the image acquisition unit 2 may be narrowed, or the image distribution interval from the imaging camera 1 to the image acquisition unit 2 may be narrowed. When the face image does not exist in the face detection area, the face image acquisition speed control unit 5 returns again to the normal image acquisition speed.

  The store visitor notification system according to the present embodiment is suitable for face matching in the subsequent stage by adding a face detection area setting acquisition unit 3 and a face image acquisition speed control unit 5 to a store visitor notification system using conventional face verification. The face image acquisition rate can be improved easily and inexpensively.

<Effect>
The store visitor notification system according to the present embodiment has the following effects. That is, the store visitor notification system according to the present embodiment is a store visitor notification system that compares a face image captured by an image capture camera with a database and notifies a registered store visitor when the images match. An image acquisition unit 2 that acquires a captured image of the camera 1, a face detection region setting unit 3 that sets one or a plurality of predetermined regions of the imaging region as a face detection region, and face detection that detects a face image from the face detection region And an image acquisition speed control unit 5 that increases the speed at which the image acquisition unit 2 acquires a captured image when a face image is detected by the face detection unit 4. When face images are no longer detected from the face detection area, it is possible to acquire many face images without significantly increasing the processing load by returning the captured image acquisition speed to the normal speed, improving the accuracy of face matching. To do.

  Further, the image acquisition speed control unit 5 controls the number of images acquired per unit time of the image acquisition unit 2 to increase the speed at which the image acquisition unit 2 acquires a captured image. Thereby, many face images can be acquired without significantly increasing the processing load, and the accuracy of face matching is improved.

  Alternatively, the image acquisition speed control unit controls the image delivery speed to the image acquisition unit 2 of the imaging camera 1 to increase the speed at which the image acquisition unit 2 acquires the captured image. Thereby, many face images can be acquired without significantly increasing the processing load, and the accuracy of face matching is improved.

(Embodiment 2)
FIG. 8 is a diagram showing the configuration of the store visitor notification system according to the second embodiment. The store visitor notification system according to the second embodiment is provided with a face detection area weighting unit 14 in addition to the configuration of the store visitor notification system according to the first embodiment shown in FIG.

  The face detection area setting unit 3 sets face detection areas in a plurality of parts of the entire imaging area of the imaging camera 1. The face detection area weighting unit 14 weights each of the plurality of face detection areas. The image acquisition speed control unit 5 increases or decreases the number of image acquisitions per unit time of the image acquisition unit 2 according to weighting.

  For example, if the first area and the second area are set as the face detection area, when a face appears in the first area, the number of images acquired from the imaging camera 1 is set to n times the normal setting, and the second area When a face appears, the weight is determined by dividing the area so that the number of images acquired from the imaging camera 1 is m times the normal setting.

  Alternatively, when a face appears in any one of the first area and the second area, the number of images acquired from the imaging camera 1 is set to n times the normal setting, and both the first area and the second area are When a face appears in the area, the weighting determination method is changed depending on the number of setting areas where the face is detected, such as increasing the number of images acquired from the imaging camera 1 to m times the normal setting. Also good.

  Although an example in which two face detection areas are set has been described, weighting is similarly determined and the number of acquired images is controlled when three or more face detection areas are set.

<Effect>
The store visitor notification system according to the present embodiment has the following effects. That is, in the store visitor notification system of the present embodiment, the face detection area setting unit 3 sets a plurality of predetermined areas as face detection areas, and includes a face detection area weighting unit 14 that weights each of the plurality of predetermined areas. In addition, the image acquisition speed control unit 5 increases the speed at which the image acquisition unit 2 acquires a captured image according to the weighting. Thereby, many face images can be acquired without significantly increasing the processing load, and the accuracy of face matching is improved.

  The present invention is installed at the entrance of a store or the like, and the face of the visitor is collated with a registered face image database of the visitor registered in advance. The present invention is applicable to a store visitor notification system that requires real-time capability to notify the store with accompanying information, and supports store management at a low cost.

  DESCRIPTION OF SYMBOLS 1 Imaging camera, 2 Image acquisition part, 3 Face detection area setting part, 4 Face detection part, 5 Image acquisition speed control part, 6 Face tracking part, 7 Face image extraction normalization part, 8 Feature point extraction part, 9 Collation part DESCRIPTION OF SYMBOLS 10 Temporary collation part, 11 Notification part, 12 Registration face image database, 13 Temporary registration face image database, 14 Face detection area weighting part.

Claims (3)

A store visitor notification system that compares a face image captured by an imaging camera with a database, and notifies a registered visitor when they match,
An image acquisition unit for acquiring a captured image of the imaging camera;
A face detection area setting unit that sets a plurality of predetermined areas of the imaging area as face detection areas;
A face detection unit for detecting a face image from the face detection region;
An image acquisition speed control unit that increases a speed at which the image acquisition unit acquires the captured image when a face image is detected by the face detection unit ;
A face detection area weighting unit that weights each of the plurality of predetermined areas,
The image acquisition speed control unit increases a speed at which the image acquisition unit acquires the captured image according to the weighting.
Visitor notification system.   The image acquisition speed control unit increases the speed at which the image acquisition unit acquires the captured image by controlling the number of images acquired per unit time of the image acquisition unit. The store visitor notification system described in 1.   The image acquisition speed control unit increases a speed at which the image acquisition unit acquires the captured image by controlling an image distribution speed of the imaging camera to the image acquisition unit. 1. The store visitor notification system according to 1.

Images