JP2009009231A - Security management system and security management method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a security management system and a security management method, which improves of convenience of a user, and ensures security in a specific area. <P>SOLUTION: Face data of a person having authority to enter a store as a security area are registered, an image including at least a face of a person approaching a commodity inside the store is photographed, the face of the person included in the photographed image and the already registered face data of the person are collated, takeout of the commodity by the person is permitted when the face of the person included in the photographed image accords with the already registered face data of the person, it is detected that the person permitted with the takeout of the commodity takes out the commodity, and information showing that the person has taken out the commodity is stored in a storage device, based on a detection result thereof. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to, for example, a security management system and a security management method for managing entrance / exit in merchandise stores, restaurants, etc., selling goods, and providing various services.

  Conventionally, many entry / exit management systems manage security based on information stored in a storage medium possessed by a user. For example, a card reader and an optical sensor are installed in a specific facility where free access is restricted, and a combination of the information of the IC card read by the card reader and the detection result of the optical sensor A system for managing entry / exit has been proposed (for example, Patent Document 1).

However, in the technique of Patent Document 1, it is considered that security is weak against an action of entering and leaving using a card illegally obtained by another person. In the technique of Patent Document 1, the behavior of visitors in an area where security is ensured is monitored, or the person who is determined not to behave properly at the time of leaving is refused to leave. Can not do it. Therefore, there is a demand for a system that maintains security in a state where a certain security area is unmanned without placing a burden on the user.
JP 2002-304652 A

  An object of one aspect of the present invention is to provide a security management system and a security management method capable of improving convenience for users and ensuring security in a specific area with an automated system.

  A security management system according to an aspect of the present invention includes a storage unit that stores face data of a person who has the authority to purchase a product, and an image captured by an imaging unit that captures an image including at least a face of a person approaching the product. A collation unit that collates the face of the person included and the face data of the person stored in the storage unit, and the face of the person included in the image photographed by the photographing unit is stored in the storage unit by the collation unit. A permission means for permitting access to the product by the person, and a detection means for detecting that the person permitted to access the product by the permission means has taken out the product when the face data of the person matches. And control means for storing in the storage means information indicating that the person has taken out the product based on the detection result of the detection means.

  According to one aspect of the present invention, a security management system photographs a first storage unit that stores face data of a person authorized to enter a security area, and an image including at least the face of the person entering the security area. A first collating unit that collates a human face included in an image photographed by the first photographing unit and a human face data stored in the second storage unit, and the first collating unit. When the face of the person included in the image photographed by the first photographing means matches the face data of the person stored in the first storage means, the person enters the security area. The admission permission means to be permitted, and the face data and history data of the person permitted to enter the security area by the admission permission means are stored as the visitor data. Second storage means, and a person's face included in the image taken by the second photographing means for photographing an image including at least a face of a person approaching the product arranged in the security area, and the second Second collating means for collating the face data of the visitors stored in the storage means, and the face of the person included in the image photographed by the second photographing means by the second collating means is the first collating means. If the face data of the person stored in the storage means 2 matches, the access permission means for permitting access to the product by the person, and the person permitted to access the product by the access permission means Detection means for detecting that the product has been taken out, and information indicating that the person has taken out the product based on a detection result by the detection means, as the history data of the person, as the second data And a control means for storing the 憶 means.

  A security management method according to one aspect of the present invention is a method used in a security management system, storing face data of a person having authority to purchase a product in a storage unit, and at least a person approaching the product A face of a person included in an image obtained by photographing an image including a face is collated with the face data of the person stored in the storage means, and the face of the person included in the photographed image by this comparison is stored in the storage means. If it matches the stored face data of the person, the person is allowed to access the product, and the person who is permitted to access the product detects that the product has been taken out. Information indicating that the person has taken out the product is stored in the storage means.

  A security management method according to one aspect of the present invention is a method used in a security management system, storing face data of a person authorized to enter a security area in a first storage means, A face of a person included in an image obtained by photographing an image including at least a face of a person entering the inside is collated with the face data of the person stored in the first storage means, and the photographed image is obtained by this collation. If the face of the person included matches the face data of the person stored in the first storage means, the person is allowed to enter the security area and is allowed to enter the security area. The person's face data and history data are stored in the second storage means as visitor data, and the merchandise arranged in the security area is stored. The face of the person included in the image obtained by photographing the image including at least the face of the person approaching the person is matched with the face data of the visitor stored in the second storage means, and the image obtained by this comparison is obtained. When the face of the person included matches the face data of the person stored in the second storage means, the person is permitted to access the product, and the person who is permitted to access the product by this access permission Is detected, and information indicating that the person has taken out the product is stored in the second storage means as history data of the person based on the detection result.

  According to one embodiment of the present invention, it is possible to provide a security management system and a security management method that can improve convenience for the user and ensure security in a specific area by an automated system.

Embodiments of the present invention will be described below with reference to the drawings.
The security management system described in the present embodiment is a system for performing security management of a merchandise store or a restaurant, for example, a system for realizing labor saving or unmanned operation of a merchandise store or a restaurant. In addition, the security management system can be applied to security management in small stores provided in facilities such as railway station premises, department stores, large-scale shopping centers, theme parks, amusement parks, and the like. . In the following description, a security management system for the purpose of unattended in a certain store is assumed.

FIG. 1 is a diagram illustrating an example of a store to which a security management system according to an embodiment of the present invention is applied.
As shown in FIG. 1, a door 1, a gate device 2 (2a, 2b), a display device 3 (3a, 3b), a camera C1, C2, etc. are installed near the entrance / exit of a store subject to security management. Yes. The door 1 is installed at the entrance of a store that is subject to security management. The door 1 is constituted by an automatic door, for example, and is opened when a person approaches. The gate device 2 is installed inside the store from the door 1. The gate device 2 is a device for restricting visitors or exits. The gate device 2 forms a passage for entry or exit with two housings (2a, 2b). For example, when a person who does not have the authority to enter from the door 1 enters, the gate (described later) of the gate device 2 is closed to prevent the person from entering.

The camera C1 is installed so that an image including the face of a person entering the store can be taken. That is, the face of the person who is going to enter the store through the passage formed by the gate device 2 is photographed by the camera C1. The camera C2 is installed so that an image including the face of a person who is going to enter from the store can be taken. That is, the face of the person who is going to leave the passage formed by the gate device 2 is photographed by the camera C2.
The display device 3 (3a, 3b) is installed on the entrance side (store inside) and the exit side (door 1 side) of the gate device 2, respectively. The display device 3a displays various guidance for a person who is about to enter the store. The display device 3b displays various guidance for a person who intends to leave the store.

  At the entrance side where these devices are installed, the face of the person who wants to enter the store, and the person registered by the information read from the IC card presented by the person (person who can enter the store) Whether or not the person enters the store is controlled based on the determination result. On the exit side, whether the person is a registered person (a person who can enter the store) or the possession of the product based on the information read from the IC card presented by the person's face and the person who is leaving the store. It is determined whether there is no leaving or the like, and the person's exit is controlled based on the determination result. These store entry process (entrance process) and store exit process (store exit process) will be described in detail later.

  Further, as shown in FIG. 1, a plurality of merchandise shelves are installed in a store that is subject to security management. Each product shelf is provided with a camera for capturing an image including a person's face and a camera for capturing the state of the product. For example, the product shelf A (corner A) includes a camera C3 for photographing a person's face around the product shelf A and a plurality of cameras Ca for photographing the state of each product displayed on the product shelf A. And are installed. Similarly, the product shelf B (corner B) has cameras C4 and C5 for photographing the faces of persons around the product shelf B and a plurality of products for photographing the state of each product displayed on the product shelf B. And a camera C6 for photographing a person's face around the product shelf C, and the state of each product displayed on the product shelf C are photographed on the product shelf C (corner C). For this purpose, a plurality of cameras Cc are installed.

  Based on images taken by each camera installed in the vicinity of each product shelf, the system recognizes people approaching each product shelf and detects products taken out (taken out) from each product shelf. It has come to do. For example, when the person A takes out the product a from the product shelf A, in the system, information indicating that the person A has taken out the product a from the product shelf A (or the person A takes one product from the product shelf A. (Information indicating that it has been taken out) is recorded. Such a product monitoring process will be described in detail later.

  Further, as shown in FIG. 1, a door 4 for entering a service facility is provided in a store that is subject to security management. Here, as a service facility, for example, a toilet, a restroom, a baby room, and the like are assumed. However, only a specific person (for example, a member who has registered as a member in advance) can use the service facility. For this reason, the security management system unlocks the door 4 only when a person having authority to use the service facility approaches.

  A camera C9 is provided in the vicinity of the door 4. The camera C9 is installed so as to take an image including the face of a person who is going to enter the service facility from the door 4. Further, a card reader for reading information from the IC card may be provided in the vicinity of the door 4. Therefore, the security management system determines whether or not the person approaching the door 4 is authorized to enter the service facility by authentication using a face image or IC card, and the determination result. The opening / closing of the door 4 is controlled based on the above. The entrance process to such a service facility will be described in detail later.

  Further, as shown in FIG. 1, a settlement area (register) is provided in a store that is subject to security management. In the checkout area, the price of goods acquired from each product shelf or the usage fee of the service facility is settled. In the checkout area, cameras C7 and C8 for taking images including the face of a person and a terminal device (checkout terminal) 5 for checkout processing are provided. The camera C7 is installed so as to capture an image including the face of a person approaching the checkout area. The camera C8 is installed so as to capture an image including the face of the person who is executing the checkout process at the cash register terminal 5.

  The system identifies a person (settler) from the images taken by the cameras C7 and C8 in the checkout area, and all the products that the person has taken out of each product shelf are included in the products that the person intends to purchase. Judge whether or not. If it is determined by this determination that all the products taken out from each product shelf are included in the products that the person intends to purchase, the cashier terminal 5 uses the credit card function of the IC card possessed by the person or the electronic money. Checkout is performed by function. A person who has completed the check-out of all the products at the check-out area can be left. Such a settlement process will be described in detail later.

Furthermore, cameras C10, C11, C12, etc. are provided in the store to be subjected to security management as shown in FIG. These cameras are used for person monitoring by video. For example, the cameras C10 and C11 are cameras for monitoring a door such as a back door or a staff passage that is not used by a general user. The cameras C11 and C12 are cameras for monitoring a person who uses the stairs or escalator when the stairs and escalator are installed.
Each of the cameras C1 to C12, Ca, Cb, and Cc may be any camera that can capture an image or product including a person's face. For this reason, when the system is installed in an existing store, each camera may use a camera already installed in the store, or may newly install a camera.

Next, a hardware configuration for realizing the security management system will be described.
FIG. 2 is a block diagram showing a hardware configuration example for realizing the security management system.
As shown in FIG. 2, the security management system includes a server device 11, a face information registration system 12, an in-store system 13, a monitoring center (monitoring server) 14, and the like.

  The server device 11 is connected to the face information registration system 12 and the in-store system 13. The server device 11 is a device for controlling the entire system. For example, registrant information including face data registered by the face information registration system 12 is stored. A plurality of in-store systems 13 can be connected to the server device. In other words, the security management system has a configuration capable of realizing security management in a plurality of stores.

  The face information registration system 12 is a system for inputting personal information including face data for using each store where the in-store system is provided. In this face information registration system 12, an image including at least the face of the person and information on an IC card possessed by the person are registered as personal information for using each store. The IC card is a recording medium used for a specific application. For example, the IC card stores the personal information of the owner and has a credit card function and an electronic money function. Note that a mobile terminal device such as a mobile phone may be used instead of the IC card.

  The in-store system 13 is a system for realizing security management in each store. The in-store system 13 is a system installed in each store. The in-store system 13 has functions for performing store entry processing, product monitoring processing, service facility entrance processing, checkout processing, store exit processing, and the like as described above. Further, the in-store system 13 has a function of reading out information on the registrant through communication with the server device 11 and transferring information on a person determined as a person requiring attention in the store to the server device 11. is doing. Further, the in-store system 13 has a function of communicating with a monitoring center (monitoring server). With this function, the in-store system 13 transfers an image captured by each camera to the monitoring center (monitoring server) 14 or notifies the monitoring center (monitoring server) 14 of an alarm when a suspicious person is detected. This function is realized.

  The monitoring center (monitoring server) 14 monitors the situation in each store based on information supplied from the in-store system 13. For example, the monitoring center 14 displays images taken by the cameras supplied from the in-store systems 13 on a plurality of display devices, and monitors the inside of the stores by the observer. Further, when the monitoring center 14 receives an alarm from the in-store system 13, the monitoring center 14 notifies the monitor of information indicating the store that is the source of the alarm, and the monitor or another security guard is the source of the alarm. It has come to support the express to the store.

Next, a configuration example of the face information registration system 12 will be described.
The face information registration system 12 includes a control device 21, a face registration device 22, a camera 23, a card reader 24, and the like as shown in FIG. The control device 21 controls the face information registration system 12 as a whole. The control device 21 is configured by an electronic computer having a CPU, RAM, ROM, nonvolatile memory, and the like, for example. The control device 21 realizes various functions by the CPU executing a control program stored in a ROM or a non-volatile memory.

  The face registration device 22 performs processing for registering the face image of the person photographed by the camera 23 as the face image of the user (registrant) of the security management system. The face registration device 22 is constituted by, for example, an electronic computer having a CPU, RAM, ROM, nonvolatile memory, and the like. In the control device 21, the CPU executes a control program stored in the ROM or the non-volatile memory so that the face image is detected from the image captured by the camera and the facial feature data is extracted from the face image. A function of registering face images or facial feature data in the server device 11 via the control device 21 is realized. The control device 21 and the face registration device 22 may be configured by a single computer.

The card reader 24 reads information stored in the IC card. The card reader 24 reads information stored in an IC card held by a person photographed by the camera 23. The card reader 24 is connected to the control device 21 and transfers information read from the IC card to the control device 21.
With the configuration as described above, the control device 21 integrates the face image or facial feature data supplied from the face registration device 22 with the IC card information transferred from the card reader 24, and integrates the information. Is registered in the server device 11 as registrant information.

Next, a configuration example of the in-store system 13 will be described.
As shown in FIG. 2, the in-store system 13 includes an in-store server 30, a controller 31, a storage device 32, a face matching device 33, a display device 3, a card reader 34, a camera control device 35, a cash register control device 36, an alarm. It is connected to a device 37, an air conditioning controller 39, and the like.
The in-store server 30 has a function of communicating with the server device 11 and is connected to the server device 11 via a communication line or the like. The in-store server 30 is composed of, for example, an electronic computer having a CPU, RAM, ROM, nonvolatile memory, and the like, and various functions are realized by the CPU executing programs stored in the ROM or nonvolatile memory. is doing.

  The controller 31 functions as a main control device that controls the entire in-store system 13. The controller 31 is composed of, for example, an electronic computer having a CPU, RAM, ROM, non-volatile memory, etc., and various functions are realized by the CPU executing programs stored in the ROM or non-volatile memory. Yes. The controller 31 has a function of controlling opening / closing of a door provided in the gate device 2 or the door 4. That is, the controller 31 has a function of controlling entry / exit, use of a service facility, and the like based on a face matching result described later. The controller 31 has a clock 31a for measuring the date and time. As a result, the controller 31 has a function of determining the date and time of the person who entered the store (entering the store) or determining the time during which the store stays in the store.

  The said memory | storage device 32 memorize | stores the information regarding a shopkeeper, etc., for example. For example, personal information (face data, IC card identification information, etc.) of the store entrant, the date and time of entrance, information about the product taken out by the store entrant, and the like are stored in the storage device 32 as history data.

The display device 3 (3a, 3b) displays guidance for a person who wants to enter the store or displays guidance for a person who wants to leave the store. Further, the display device 3 may incorporate a touch panel as a user interface. In this case, the display device 3 also functions as a user interface for receiving an operation instruction from the user.
In the store, in addition to the display device 3, a display device that displays guidance or the like for a person in the store may be provided. In this case, the store display device can provide various guidance to the store visitor at each location. For example, a display device is provided around each product shelf, and the in-store server 30 displays a description of the product placed on the product shelf on the display device or how to take out the product from the product shelf. You may do it. Further, a display device may be provided around the checkout area, and the in-store server 30 may display a checkout process guide on the display device.

  The card reader 34 reads information from an IC card possessed by a store clerk. For example, the card reader 34A provided in the entrance gate device 2 functions as a reading unit that reads desired information (for example, identification information of an IC card) from an IC card presented by a person entering the store. Information read by the card reader 34 </ b> A is supplied to the in-store server 30 via the controller 31. Further, the card reader 34B provided in the vicinity of the door 4 obtains desired information (for example, identification information of the IC card) from the IC card possessed by the person (entrant) who intends to enter the service facility. read.

  The camera control device 35 is a device that controls the cameras C2, C3,..., Ca, Cb, Cc. The camera control device 35 has an image interface function for acquiring an image taken by each camera, an adjustment function for adjusting the shooting position of each camera, an image processing function such as correction of an image taken by each camera, and the like. ing.

The camera control device 35 includes a face matching unit 41, a product detection unit 42, and the like.
The face collating unit 41 has a function of detecting a human face image from images taken by the cameras C2, C3,..., A function of extracting face feature data from the detected face image, and extracted face feature data and storage device. 32 has a function (face matching function) for matching the facial feature data of the store visitor stored in 32. A face image detection result or a face comparison result is supplied to the in-store server 30.
The product detection unit 42 detects the state of the product from images taken by the cameras C2, ..., C6, Ca, Cb, Cc. For example, the product detection unit 42 detects a product held by a person (entrance) from images taken by the cameras C2, ..., C6, or from images taken by the cameras Ca, Cb, Cc. The state of the product placed on the product shelf (type of lost product, number, etc.) is detected. For example, when a product that the store clerk has in hand cannot be detected, the product detection unit 42 detects how many products in which product shelf are lost.

  With the functions as described above, in the camera control device 35, the product specified by the person identified by the collation result by the face collation unit has taken out which product (or how many products have been taken out from which product shelf). Can be determined. As a result, the camera control device 35 or the in-store server 30 records information indicating the product taken out from the product shelf by each store visitor in the storage device 32 in association with each store visitor as history data. Is possible.

  The cash register control device 36 controls the cash register terminal 5. The cash register terminal 5 identifies the product presented by the accountant and calculates the price of the product. The calculated product price is settled by the credit card function or electronic money function of the IC card possessed by the settlement operator. The cash register terminal 5 may perform cash settlement. In the checkout process as described above, the cash register control device 36 specifies the checkout person from the result of identification of the face of the checkout by the camera control device 35, and the product information that the checkout person has taken out from the product shelf in the store. To get. Thereby, the cash register control device 36 determines whether or not all the products taken out from the merchandise shelves by the cashier are included in the merchandise presented by the cashier at the cashier terminal 5. That is, the cash register control apparatus checks an unsettled product based on the result of the product monitoring process.

The alarm device 37 issues an alarm based on a face matching result or the like. The alarm device 37 has a function of communicating with the monitoring center 14 and also has a function of notifying the monitoring center of an alarm.
The shelf opening / closing device 38 controls the opening / closing of electronic locks or doors provided in the product shelves A, B, C based on the control by the controller 31. That is, when the controller 31 permits the store clerk to take out the product from the product shelf, the shelf opening / closing device 38 unlocks the electronic lock of each product shelf or opens the door. .

  In addition, the said air-conditioning control controller 39 can be utilized as a controller for controlling the door 4, for example. The air conditioning controller 39 constantly adjusts the temperature in the store and controls power such as lighting. For this reason, the door 4 and the like in the store may be efficient when controlled by the air conditioning controller 39. In such a case, the air conditioning controller 39 may control the opening and closing of the door 4.

Next, face information registration processing in the face information registration system 12 will be described.
A person who wants to use the store as described above performs registration processing by the face information registration system 12. In this registration process, it is necessary to have a predetermined IC card, and face data of a person who satisfies the condition is registered in the server device 11 in association with identification information (card number) of the IC card. .

FIG. 3 is a flowchart for explaining an example of face data registration processing in the face information registration system 12.
First, a person who wishes to register (registered person) causes the card reader 24 to read an IC card possessed by the person. Then, the card reader 24 reads information for determining the card number and the card type from the IC card presented by the applicant for registration (step S11). Information read by the card reader 24 is supplied to the control device 21. Based on the information supplied from the card reader 24 (information read from the IC card), the control device 21 determines whether or not the IC card is a registerable IC card (step S12). If it is determined that the IC card is not registerable (NO in step S12), the control device 21 ends the registration process.

  When it is determined that the IC card can be registered by the above determination (step S12, YES), the control device 21 requests the face registration device 22 to acquire the registered person's face data. Then, the face registration device 22 acquires an image including the face of the person to be registered photographed by the camera 23. When an image is acquired from the camera 23, the face registration device 22 performs processing for detecting a human face image from the acquired image. When the face image is detected, the face registration device 22 extracts feature data from the face image of the person, generates face data based on the extracted feature data, and supplies the generated face data to the control device 21 (step). S13). Here, face collation for a person who is walking is assumed in face collation processing described later. For this reason, for example, feature vectors obtained from a plurality of face images are acquired as the face data.

  When the face data of the person to be registered is acquired from the face registration device 22, the control device 21 registers the face data and card information such as a card number read by the card reader 24 in the server device 11. As a result, the server device 11 stores, as registrant data, data associating the face data of the registered person with the card number of the IC card possessed by the registered person.

Next, the process (entrance process) at the time of entering the store in the in-store system 13 will be described.
A person who has completed the registration process as described above can enter the store and purchase a product in the store. In this case, the person must have a registered IC card as a condition for entering a store and purchasing a product. Accordingly, a person who wants to enter the store (store visitor) enters the store from the door 1 while holding the registered IC card.

FIG. 4 is a diagram illustrating a configuration example of devices installed in the vicinity of a store entrance.
As shown in FIG. 4, a door 1 is provided at the entrance of the store, and a gate device 2 is disposed on the door 1. The door 1 is an automatic door that is opened when a person approaches. Therefore, the store visitor passes through the passage formed by the gate device 2 through the door 1.

  In the vicinity of the gate device 2a, a camera C1 for photographing the face of a person entering the store is provided. In addition, a display device 3a is provided at a position where a person entering the store can easily see. A camera C2 for photographing the face of a person leaving the store is provided near the gate device 2b, and a display device 3b is provided at a position where a person who is leaving the store can easily see. Yes.

  Further, the gate device 2 is provided with a card reader 24, a door 61, a button 62, and the like. The card reader 24 reads information stored in an IC card possessed by a store clerk. The door 61 is controlled to be opened and closed by the controller 31. The button 62 is a button for a process for a companion where a registrant and another person enter the store together (a companion handling process described later). Instead of the button 62, a touch panel may be built in the display device 3a, and the same function as the button 62 may be realized by the touch panel. However, the button 62 can be omitted in the case of an operation mode that does not allow sharing.

  In the configuration as shown in FIG. 4, the store visitor passes the opened door 1 and causes the card reader 24 to read the IC card. Thereby, the face collation device 33 determines whether or not the face of the shopkeeper photographed by the camera C1 matches the face of the registrant specified by the information read from the IC card. Based on the determination result, the controller 31 controls the opening and closing of the door 61, and displays guidance for the store visitor on the display device 3a.

Next, the procedure of the store entry process will be described.
FIG. 5 is a flowchart for explaining an example of the store entry process.
When the door 1 is opened and the store visitor enters the passage formed by the gate device 2, the camera C1 starts capturing an image including the face of the store walker (step S21). . Images captured by the camera C1 are sequentially supplied to the face matching device 33 in units of frames. On the other hand, when entering the gate device 2, the store visitor presents the IC card he / she owns to the card reader 34A. Then, the card reader 34A reads the stored contents of the IC card presented by the store visitor (step S22). The card reader 34A supplies the information read from the IC card presented by the store visitor to the controller 31.

  When the information read from the IC card is supplied by the card reader 34A, the controller 31 first determines whether or not the IC card is a valid card based on the information read from the IC card (step) SS23). For example, the controller 31 determines whether or not the IC card is a specific type of IC card that can use the system (specific use IC card) and the expiration date is valid.

  If it is determined by this determination that the IC card is a valid card (step S23, YES), the controller 31 determines whether the owner of the IC card is a person who has registered face data or the like. (Step S24). That is, when the controller 31 determines that the IC card is valid, the card number included in each registrant data stored in the server device 11 via the in-store server 30 and the card read from the IC card Match numbers. As a result, when there is registrant data with a card number that matches the card number read from the IC card, the controller 31 determines that the owner of the IC card is a registrant.

  When there is registrant data including a card number that matches the card number read by the card reader 34A (step S24, YES), the controller 31 downloads the registrant data from the server device 11 to the storage device 32. Processing is performed (step S25). Thereby, the storage device 32 stores registrant data including face data of the presented IC card owner.

  When the face data is stored in the storage device 32, the face collation device 33 extracts the face data from the face image included in the image taken by the camera C1, and downloads the extracted face data and the storage device 32 to the face data. The face data corresponding to the card number of the IC card is collated (step S26). When it is determined that the face data extracted from the image photographed by the camera C1 by the collation matches the face data corresponding to the card number of the IC card (step S27, YES), the controller 31 together with the store visitor. It is determined whether or not there is a companion to enter the store (step S28).

  Here, co-payment means that a registered entrant enters an unregistered person together. If it is determined by the above determination that there is co-existence (step S28, YES), the controller 31 performs co-operation handling processing (step S29). As for the above-mentioned companion, various processing modes are assumed depending on the operation mode. For this reason, the accompanying process will be described later in detail.

  When it is determined that there is no sharing by the above determination (step S28, NO), the controller 31 displays a guidance indicating that the store is permitted on the display device 3a (step S30), and opens the gate on the store side. (Step S31). When the entrance-side gate is opened, the controller 31 stores the store visitor's data in the storage device 32 as history data (store entrance data). At this time, according to the time counted by the clock 31a, information such as the store entry date and time is also stored in the storage device 32 as history data of the store visitor (step S32).

  If it is determined in step S23 that the IC card presented by the customer is not an available IC card, the controller 31 displays on the display device 3a that the store cannot be entered because it is not an available IC card. Then, the door 61 on the store entrance side is closed. As a result, the store entry is rejected.

  Further, when it is determined in step S24 that the IC card presented by the store is not a registered IC card (that is, when the card number of the presented IC card does not match the registered card number), the controller 31 displays on the display device 3a that it is not registered because it is not registered, and closes the door 61 on the store side. As a result, the store entry is rejected.

  If it is determined in step S27 that the face data of the store clerk and the face data of the registrant data do not match (that is, if the face data of the registrant data is determined to be NG), the controller 31 determines that the registrant data and the face are the same. Since they do not match, the fact that entry is impossible is displayed on the display device 3a, and the door 61 on the entry side is closed. As a result, the store entry is rejected.

  In the store entry process as described above, it is checked whether or not the IC card presented by the store is the IC card of the registrant, and whether or not the face of the store matches the face of the registrant. Check it out. This makes it possible to refuse entry to a person who has an IC card that has not been registered, or a person who wants to enter with an IC card of another registered person. In addition, since face matching itself is performed without any special operation by the shopkeeper, a registered person can enter a security-managed shop simply by presenting his IC card. Is possible.

  In the store entry process described above, it is determined whether the IC card possessed by the store owner and the face of the store store match the registrant data. However, depending on the operating conditions, the IC card check (steps S22 to S25) may be omitted. In this case, as the processing of step S26, the face matching device 33 determines whether or not there is registrant data including face data that matches the face of the store clerk (that is, the face matching of 1-to-N matching is performed). Execute. If the controller 31 determines that there is registrant data including face data that matches the face of the store clerk based on the result of this face matching process, the controller 31 permits the store entrant to enter the store, When it is determined that there is no registrant data including face data that matches the face of the store visitor, the controller 31 may refuse to enter the store. Thereby, the registrant can enter the store smoothly without presenting the IC card when entering the store, and the convenience can be improved.

Next, the co-responding process in step S29 will be described.
For example, it is assumed that there are various forms of sharing with parents and children (with children), caregivers, friends, couples, and the like. In the store entry process as described above, in order to ensure security, an operation mode in which sharing is not permitted is conceivable. In this case, the controller 31 controls to prohibit the passage of the person determined to be accompanied.

  However, for example, a person who does not normally need to have an IC card, such as an infant or a person who needs care, or a person who does not meet the conditions for issuing an IC card may be requested to use the store. There is also sex. In other words, depending on the operation mode, there may be a case where it is preferable to allow companionship, or to allow companionship only to children or care recipients. In such a case, various processing forms are conceivable as processing for the accompanying person (processing for supporting the accompanying person). Hereinafter, an example of the processing mode according to the operation mode corresponding to the shared mode will be described.

  For example, it is assumed that it is an operation mode in which only the number of persons designated by the registrant is allowed when entering the store. In such an operation mode, as shown in FIG. 4, there is a processing mode in which a button 62 that allows a registrant to input the number of persons is provided in the gate device 2 and the registrant accepts the number of persons specified when entering the store. Conceivable.

  In such a processing mode, when the registrant enters the store, the number of accompanying persons is input by the button of the gate device 2. In response to the designation of the number of accompanying persons, the controller 31 opens the door 61 until the accompanying person passes within a predetermined time. Thereby, the registrant and the accompanying person can enter the store. However, even in this case, for processing at the time of the store or when the store is closed, the camera C1 captures an image including the face of each person accompanying the face, and the face image and the face extracted from the face image The data is stored in the storage device 32 as data of a companion person associated with the registrant data.

  In addition, an operation mode in which a registrant allows a specified number of persons to be accompanied in advance can be considered. In this case, in the registrant data, the number of people who may be accompanied with the registrant is registered in advance, and up to the number of people registered in advance, entering the store together with the registrant. There may also be a processing form of accepting.

  In such a processing mode, when the registrant for whom the number of companions is set enters the store, the controller 31 is a person who has been registered in advance as to a person who can be determined to be accompanied by the registrant. Only minutes allow entry. As a result, it is possible for a registered person and a number of persons who have been registered in advance to enter the store. However, even in this case, for processing at the time of the store or when the store is closed, the camera C1 captures an image including the face of each person accompanying the face, and the face image and the face extracted from the face image The data is stored in the storage device 32 as data of a companion person associated with the registrant data.

  In addition, an operation mode in which only a person whose face data is registered as a person to be accompanied with a registrant is allowed to be accompanied is also considered. For example, by registering face data of a registrant's child, spouse, or parent in the registrant data as the face data of the person accompanying the registrant, those accompanying persons are registered with the registrant. In the case of companionship, a processing form in which entry is permitted by collation using face data is conceivable.

  In such a processing mode, the controller registers the face data extracted from the image photographed by the camera C1 and the person who is determined to be accompanied with the registrant in advance in association with the registrant data. It is determined whether or not the registered person is allowed to be accompanied by collation with the face data of the accompanying person. In other words, when the face data of a person who is determined to be accompanied by a registrant matches the face data of the accompanying person registered in advance in association with the registrant data, Allow store. As a result, a person registered in advance as face data of a person to be accompanied can be entered together with the registrant.

Next, the product monitoring process in the in-store system 13 will be described.
In the in-store system 13, it is necessary for the system to monitor each product displayed on each product shelf for the purpose of unmanned or labor saving. For this reason, in the said in-store system, the action of the person around the product shelf is monitored by each camera that captures the periphery of each product shelf, and the state of the product itself in each product shelf is also based on the image captured by each camera. Monitor. In the product monitoring process, a person who has taken out a product from the product shelf and a product taken out from the product shelf by the person are detected, and the data is stored as history data. As a result, it is possible to determine whether or not each person is surely paying out the product taken out from the product shelf in the subsequent checkout process.

FIG. 6 is a flowchart for explaining an example of the product monitoring process as described above.
That is, each of the cameras C3 to C6 captures images around predetermined product shelves A, B, and C (step S51). For the images taken by these cameras C3 to C6, face detection processing is performed by the face matching unit 41 in the camera control device 35 (step S52). When a face is detected from an image captured by any camera by such face detection processing (step S52, YES), the camera control device 35 extracts face data from the detected face image, The extracted face data is collated with the face data of the store visitor (regular store visitor) stored in the storage device 32 (step S53).

  When the face data extracted from the image captured by the camera by this face matching does not match the face data of the shopkeeper (step S54, NO), the controller 31 determines that the person is a suspicious person, and A notification for urging the alarm to the alarm device 37 is output. Upon receiving this notification, the alarm device 37 notifies the monitoring center 14 of the alarm (step S61). This merchandise monitoring process is also a process of monitoring a person approaching the merchandise shelf in the store. Therefore, when a person who is not a regular store visitor is detected, that is, when a person who does not match the face data of the store visitor is found, the person is notified to the monitoring center 14 as a suspicious person. It is possible to manage the security.

  If the face data extracted from the image captured by the camera by the face matching matches the face data of the shopkeeper (step S55), the controller 31 determines that the person is a legitimate shopkeeper. To do. Accordingly, when it is determined that the person is a regular store visitor, the controller 31 unlocks the product shelf that the store visitor is approaching by the shelf opening / closing device 38 or the door of the product shelf. Is released (step S55). When the product shelf is opened, the person can take out the product from the product shelf.

  In this state, the camera control device 35 is an image captured by the camera that is capturing the person or the camera that is capturing each product on the product shelf until the shopkeeper leaves the product shelf. Thus, the removal of the product from the product shelf is monitored (step S56). That is, the camera control device 35 monitors the behavior of the store visitor by the face matching unit 41 based on the video of the camera that is shooting the store visitor, and determines whether or not the product is being taken out. Determine. Further, in the camera control device 35, the product detection unit 42 detects whether or not the product is taken out based on an image of a camera that captures the product on the product shelf. When it is detected that the store clerk has taken out the product (step S56, YES), the camera control device 35 performs a process for determining the product to be taken out to determine which product the store clerk has taken out ( Step S57).

  Here, when it is difficult to determine the product taken out from the video of the camera photographing the store visitor, the camera control device 35 selects the product taken out from the video of the camera photographing the product shelf. Determine. Furthermore, when it is difficult to discriminate which product from the image of the camera that is photographing the product shelf, the camera control device 35 detects the number of products taken out from the product shelf. Further, when the number of products taken out from the product shelf is also unclear, the camera control device 35 detects that at least the shopkeeper has taken out any product from the product shelf.

  When the product taken out by the taken-out product discrimination process can be discriminated, the camera control device 35 provides information indicating which product the merchant has taken out from which product shelf as the history data of the customer. It memorize | stores in the memory | storage device 32 (step S58). In addition, when only the number of taken-out products can be determined, the camera control device 35 stores information indicating how many products the customer has taken out from which product shelves as the history data of the customer. It memorize | stores in the apparatus 32 (step S58). Further, when the number of pick-up items cannot be determined, the camera control device 35 stores information indicating the product shelves from which the store has taken out goods in the storage device 32 as history data of the store. Store (step S58).

  Further, the camera control device 35 determines whether or not the shopkeeper has moved away (leaved) from the merchandise shelf based on the video of the camera shooting the shopkeeper (step S59). ). For example, in the camera control device 35, the store visitor determines whether the face of the store visitor can no longer be detected by the face matching unit 41 based on the video of the camera shooting the store visitor. It is determined whether or not the product shelf has been moved away (leaved).

  Thereby, when it is determined that the store visitor has left the product shelf (step S59, NO), the camera control device 35 requests the controller 31 to lock the product shelf. By responding to this request, the controller 31 locks the product shelf. If it is determined that the store visitor has not left (NO in step S60), the camera control device 35 repeatedly executes the processes in steps S56 to S59.

  In the merchandise monitoring process as described above, when the shopkeeper exists around a merchandise shelf, the camera control device 35 determines whether the shopkeeper has taken the merchandise shelf from the merchandise shelf based on the video taken by each camera. We are monitoring the removal of goods. As the monitoring result, the camera control device 35 stores information (product information) indicating the product taken out by the store visitor in the history data of the store visitor. As a result, the products in the store can be efficiently monitored, and the product information that has been taken out is stored in the history data, so that it is possible to prevent a product from being missed during the payment process.

Next, admission processing for the service facility will be described.
In the in-store system 13, entrance management of service facilities that can be entered only from within the store is also performed in order to promote the use of the store or improve the convenience of registrants of the system. Here, as the service facility, a toilet, a restroom, a baby room, a break room, a shower room, and the like are assumed. In the in-store system 13, if a person who is confirmed to be a registrant (member) by face verification or IC card authentication does not have an unsettled product, Processing to permit use (entrance) is performed.

  Furthermore, as a form of operation of the service facility as described above, the use of the service facility may be charged, or may be limited to a specific member who can use the service facility among registrants. . For example, in the case where the use of the service facility is charged, the in-store system 13 may pay for the use fee of the service facility in a check-out process to be described later, or credit the IC card when entering the service facility. You may make it settle by a function or an electronic money function. Further, in the case where the use of the service facility is limited to a specific member, the in-store system 13 provides the person of the service facility with respect to a person who is confirmed to be a specific member by face verification or user authentication using an IC card. It should be allowed to use.

FIG. 7 is a flowchart for explaining an example of the entrance process of the service facility.
In the example shown in FIG. 7, if a person identified by face matching and an IC card is a specific member, if the person does not have unsettled goods, the person is allowed to enter the service facility. The form to be assumed shall be assumed.
That is, when a person who normally enters the store uses the service facility, the person moves to the front of the door 4 that is the entrance of the service facility in the store. Then, the camera C9 that captures the periphery of the door 4 captures an image including the face of the person who is walking and approaches the door 4 (step S81). Images taken by the camera C1 are sequentially supplied to the camera control device 35 in units of frames. Thereby, in the camera control apparatus 35, the face collation process by the face collation unit 41 can be performed.

  On the other hand, when the person approaches the door 4, the person presents the IC card held by the person to the card reader 34 </ b> B provided in the vicinity of the door 4. Then, the card reader 34B reads the stored content of the IC card presented by the person (step S82). The card reader 34B supplies the information read from the IC card presented by the store visitor to the controller 31.

  When the information read from the IC card is supplied by the card reader 34B, the controller 31 first determines whether or not the IC card is a valid card based on the information read from the IC card (step). SS83). For example, the controller 31 determines whether or not the IC card is a specific type of IC card that can use the system (specific use IC card) and the expiration date is valid.

  If it is determined by this determination that the IC card is a valid card (step S83, YES), the controller 31 is a registered registrant of a store clerk including a card number that matches the card number read by the card reader 34B. Identify the data. Information indicating the specified store visitor or information indicating the face data included in the registered user registrant data is notified to the camera control device 35. Thereby, the camera control device 35 collates the face data extracted from the image photographed by the camera C9 by the face collation unit with the face data of the store visitor specified by the information on the IC card (step S84). .

  When it is determined that the face data extracted from the image captured by the camera C9 by this face matching and the face data of the store clerk corresponding to the card number of the IC card match (step S85, YES), the controller 31 It is determined whether or not the store visitor is a person (specific member) who has authority to use the service facility (step S86). If it is determined by this determination that the person is a specific member (step S86), the controller 31 determines whether or not the person possesses an unsettled product based on the history data of the person. (Step S87).

  If it is determined by this determination that the person does not have an unsettled product (step S87, NO), the controller 31 displays a guide for permitting the person to enter the service facility. Or it alert | reports with an audio | voice (step S88) and the door 4 for entrance is open | released (step S89). When the entrance door 4 is opened and the person enters the service facility, the controller 31 records that the person's history data has entered the service facility (step S90).

  If it is determined in step S83 that the IC card presented by the person is not an available IC card (NO in step S83), the controller 31 cannot use the service facility because it is not an available IC card. The fact is notified by voice or display (step S91), and the entrance door 4 is kept closed (step S92). As a result, entry of the person to the service facility is denied.

  If it is determined in step S85 that the face data of the person does not match the face data included in the registrant data of the store clerk (that is, if face matching is determined to be NG) (step S85, NO) The controller 31 notifies the user that the service facility cannot be used because the face data of the face image taken by the camera C9 does not match the face data of the store clerk by voice or display (step S91). The entrance door 4 is kept closed (step S92). As a result, entry of the person to the service facility is denied.

  If it is determined in step S87 that the person is not a specific member having authority to use the service facility (NO in step S87), the controller 31 is not registered as a member and cannot use the service facility. (Step S91), and the entrance door 4 is kept closed (step S92). As a result, entry of the person to the service facility is denied.

  If it is determined in step S87 that the person has an unsettled product (YES in step S87), the controller 31 returns to the service facility because there is an unsettled product for the person. Information indicating that entry is not allowed is displayed or notified by voice (step S91), and the entrance door 4 is kept closed (step S92). As a result, entry of the person to the service facility is denied.

  In the admission process of the service facility as described above, it is checked whether the IC card or the face image possessed by the person who desires the user belongs to a member who can use the service facility, It is checked whether or not a person has an unsettled product taken out in the store. As a result, members who can use the service facility and who do not have unsettled merchandise can be allowed to enter the service facility and provide the services made by the above members. Is possible. On the contrary, it is possible to refuse entry to the service facility for a person who is not a member who can use the service facility and a person who has an unsettled product.

  In the admission process to the service facility described above, in order to determine whether or not the person who wishes to enter the service facility is a member, a check based on the person's face verification and the IC card possessed by the person We are going to carry out with the check of. However, depending on the operating conditions, either the face collation check (S81, S84-S85) or the IC card check (steps S82-S83) may be omitted.

For example, when a service facility is a toilet or the like, there may be many users who do not like taking a face when entering. In such a case, a service facility is provided to the person by identifying the person only by checking with the IC card presented by the person and checking that the person is a member and does not have unsettled goods. It is possible to give permission to use.
In addition, when it is desired to save the trouble of presenting the IC card when entering the service facility, the check of the IC card is omitted and the face data extracted from the face image taken by the camera C9 and the input stored in the storage device 32 are stored. By identifying a person by face matching with the store's face data and checking that the person is a member and does not have unsettled goods, the person is permitted to use the service facility. Is possible.

Next, a product settlement process in the in-store system 13 will be described.
In the in-store system 13, in order to eliminate unmanned or labor-saving, it is necessary to settle the products purchased by the store at the checkout (register) by the store's own operation. In order to realize such a checkout process, the in-store system 13 photographs the face of the person approaching the checkout and the face of the person executing the checkout process by the cameras C7 and C8 that photograph the vicinity of each checkout. is doing. A person whose face is photographed by these cameras C7 and C8 is detected as a person who performs settlement (settlement person). The cashier presents the product to be purchased at the cash register terminal 5 to cause the cashier terminal 5 to read information on the product desired to be purchased. At this time, the in-store system 13 specifies a checkout person from the images taken by the cameras C7 and C8, and information on the product to be taken out included in the history data of the checkout person and purchase that the checkout person has read into the cash register terminal 5 Check the validity of the checkout process based on the desired product information. In this way, when the validity of the product to be settled is confirmed, the system 13 performs a payment process using an IC card possessed by the settlement person.

FIG. 8 is a flowchart for explaining an example of the settlement process.
That is, each of the cameras C7 and C8 takes an image around the checkout area (step S111). For the images taken by these cameras C7 and C8, face detection processing is performed by the face collation unit 41 in the camera control device 35. When a face of a person is detected from an image captured by any camera by such face detection processing, the camera control device 35 extracts face data from the detected face image, and the extracted face data Is compared with the face data of the store visitor (regular store visitor) stored in the storage device 32 (step S112).

  If the face data extracted from the images taken by the cameras C7 and C8 by this face matching does not match the face data of the store clerk (step S113, NO), the controller 31 cannot perform the settlement process for the person. (Step SS120). Further, when it is determined that the person seems to be a suspicious person, the controller 31 may notify the monitoring center 14 of an alarm by the alarm device 37.

  If the face data extracted from the image captured by the camera by the face matching matches the face data of the store clerk (step S113, YES), the controller 31 determines whether the person who entered the store properly ( It is determined that it is a checkout. When the checkout is determined in this way, the controller 31 reads information on the product that the checkout has taken out from each product shelf in the store from the history data of the checkout stored in the storage device 32 (removed product). Information) is read out (step S114).

  On the other hand, the account adjuster presents the product to be purchased to the cash register terminal 5 while the above processing is being executed. For example, the accountant causes the bar code attached to the product to be read by a bar code reader attached to the cash register terminal 5. As a result, the cashier terminal 5 obtains information on the product that the checkout person desires to purchase and supplies the information to the cashier control device 36 (step S115). Note that the procedure for causing the cashier terminal 5 to read information on the product desired to be purchased is not limited to the barcode attached to the product. For example, when the IC tag is attached to the product, The merchandise information stored in the IC tag may be read by the cashier terminal 5.

  When the information of the product (the product to be settled) that the checkout person read by the cashier terminal 5 and the product information taken out by the checkout in the store are obtained, the cashier control device 36 (or the controller 31). ) Determines whether or not the merchandise item that the accountant wishes to purchase includes all the products that the accountant has taken out from the merchandise shelves in the store (step S116). In this determination, when information indicating a product is included as the extracted product information, it may be determined whether or not the product is included in the product to be settled. In addition, when information indicating the product shelf and the number of products is included as the extracted product information, it is determined whether or not the products to be settled include the number of products specified in the extracted product information. Just do it. Furthermore, when only the information indicating the product shelves of the extracted products is included as the extracted product information, it may be determined whether or not the products on the product shelves are included in the products to be settled.

  When it is determined by the above determination that all the products taken out in the store by the account adjuster are included in the products to be adjusted (step S116, YES), the controller 31 is controlled by the cash register control device 36. The cashier process is executed for each product to be settled presented by the cashier by the cashier terminal 5 (step S118).

  The cash register terminal 5 settles the payment for the merchandise to be settled by the credit card function or electronic money function of the IC card possessed by the settlement person. In this case, the cash register terminal 5 reads the information of the credit card function or the electronic money function from the IC card of the account settlement person by the card reader / writer 51, and collects the price of the commodity to be settled. For example, when an IC card possessed by a checkout person has a prepaid electronic money function, the cashier terminal 5 uses the card reader / writer 51 to withdraw the product price from the IC card to checkout. Complete the process.

  Although it is possible to accept payment for the product price in cash, this system has one of the purposes of unmanned or labor-saving, so the credit card function of the IC card or electronic It is assumed that the product price can be settled only by the money function. Furthermore, in order to improve the security in this system, only the IC card presented by the person who entered the store at the time of entry may be used for the checkout process.

  When the settlement process is completed, the cash register control device 36 gives a flag indicating that the settlement has been completed for each product that has been settled in the history data of the settlement account stored in the storage device 32. . As a result, for each product for which the accountant has completed payment, it is recorded that the account has been settled. For example, an action such as the person entering the service facility while having the account already settled. Is possible. In addition, when payment for all products is completed, the person can leave the store.

  Further, when it is determined by the above determination that all the products taken out by the account adjuster are not included (step S116, NO), the cash register control device 36 issues an alarm by the alarm device 37, and the account adjuster's The information is registered in a list of persons who need attention in the storage device 32 (step S117). At this time, a warning to confirm the unsettled product is displayed on the display unit of the cash register terminal 5 or the like. However, here, even in such a case, it is assumed that the checkout process for the purchase desired product presented by the checkout person is performed. Accordingly, the cash register control device 36 proceeds to step S118 and executes a payment process for the commodity price to be settled.

  In the checkout process as described above, a checkout person is identified by face collation using face images photographed by the cameras C7 and C8 provided around the checkout area, and products that the checkout person wishes to purchase and the history of the checkout person. The legitimacy of the product to be settled is checked by checking the product taken out in the store included in the data.

  As a result, it is possible to realize unmanned or labor-saving of the checkout area (register). Further, it is possible to check whether or not the product taken out in the store has been settled, and security management for the product in the store can be realized. Furthermore, for the user, the product can be purchased by settlement using an IC card, and convenience is improved.

Next, processing at the time of leaving the store in the in-store system 13 (store closing processing) will be described.
A person who has settled all the merchandise taken out of the merchandise shelf by the settlement process as described above, or a person who has not purchased the merchandise in the store can leave the store. In other words, leaving the store is a condition that all products taken out from the product shelf have been settled.

FIG. 9 is a flowchart for explaining an example of the store closing process.
When a person who wants to leave the store (a person who has left the store) approaches the passage formed by the gate device 2, the camera C2 starts to capture an image including the face of the store clerk who is walking. (Step S141). Images taken by the camera C2 are sequentially supplied to the face matching device 33 in units of frames. Thereby, the face collation device 33 extracts the face data from the face image included in the image photographed by the camera C2, and the face data of each shopkeeper stored in the storage device 32 is extracted. Collation with data is performed (step S142).

  When it is determined that there is a store visitor's face data that matches the face data extracted from the image captured by the camera C2 by this comparison (step S143, YES), the controller 31 stores the store visitor's history data. It is determined whether or not there is information on the product read from the device 32 and taken out by the person (step S144). If it is determined that there is a product taken out by the store clerk (step S144, YES), the controller 31 judges whether or not all the products taken out by the store clerk have been settled ( Step S145).

  When it is determined that all of the extracted products have been settled by this determination (step S145, YES), or when it is determined that there is no product information extracted by the store clerk (NO at step S144), the above The controller 31 determines whether or not the store clerk is accompanied (step S146). Here, in order to maintain consistency between the time of entering the store and the time of leaving the store, it is assumed that the person who is accepted as a companion is only the person who was accompanied by the store at the time of entry. Therefore, in the above determination, it is determined whether or not there is a person who was accompanied at the time of entering the store as the history data of the store clerk. If it is determined that there is a companion (NO in step S146), the controller 31 performs a companion handling process including a store exit process for the companion person (step S147). It should be noted that the same processing is performed for the person who has left the store regardless of whether or not he / she is accompanied, and the person who is determined to be a person who is accompanied by the regular registrant is accompanied by the person who is accompanied by the person in step S147. It is assumed that response processing is performed.

  Also, here, at the time of leaving the store, the face image or face data of each accompanying person is acquired at the time of entering the store or in the store, and is stored in the storage device 32 in association with the history data of the store exit. It is assumed that Therefore, for each accompanying person, the face data extracted from the image taken by the camera C2 and the accompanying person's face data stored in the storage device 32 in association with the history data of the store clerk. It is determined whether or not the store can be closed based on whether or not. That is, the face data extracted from the face image of the accompanying person photographed by the camera C2 matches the face data of the accompanying person stored in the storage device 32 in association with the history data of the store clerk. Then, the person (a person who is accompanied by the person) is permitted to leave the store.

  Further, for those who have left the store where it has been determined that all of the products to be taken out have been settled, the controller 31 displays a guidance indicating that the store has been closed on the display device 3b (step S148) and is provided in the gate device 2. The door 61 on the exit side is opened (step S149). Further, when the store clerk leaves the store, the controller 31 writes that the store has been closed in the history data of the store clerk or deletes the history data of the store clerk (step S150). .

  When the face collation is NG in step S143 (step S143, NO), the controller 31 closes the door 61 on the exit side in the gate device 2 (step S151), and the person leaving the store leaves the store. Information indicating that the store is not available is displayed on the display device 3b (step S152). Further, the controller 31 notifies the monitoring center 14 of an alarm by the alarm device 37 (step S153). As a result, it is possible to notify the monitoring center 14 of an alarm in order to leave a person who is not determined to be a regular store visitor.

  If it is determined in step S145 that there is a product that has not been settled (step S145, NO), the controller 31 closes the door 61 on the exit side of the gate device 2 (step S151), and the store clerk. Displays information on the display device 3b that the store cannot be closed because there is an unsettled product among the products taken out in the store (step S152). Further, the controller 31 notifies the monitoring center 14 of an alarm by the alarm device 37 (step S153). In addition, when there is an unsettled product, there may be a case in which the customer is not settled due to an error at the time of the settlement process or because the person does not intend to leave the store or erroneously detect the camera C2. For this reason, when there is an unsettled product, the alarm device 37 does not notify the monitoring center 14 of an alarm, but displays a guidance for prompting the settlement of the unsettled product on the display device 3b. Also good.

  In the store exit process as described above, the face of the store clerk matches the face of the regular store entrant, and the settlement of all the products that the store exit has taken out from the store shelves in the store has been completed. In this case, the store is allowed to leave the store. As a result, it is possible to refuse to leave the store for a person who has not entered the store properly or a person who has not completed the settlement of the product taken out from the product shelf in the store. In addition, face matching itself is performed without any special operation by the store entrant, so people who have just entered the store and completed the checkout of all the products can go smoothly without taking any special action. It will be possible to leave the store, improving convenience.

  In addition, when an unauthorized store clerk forcibly exits, information regarding the illegitimate store exit may be notified to the external system via the server device 11 or the monitoring center 14. In this case, an unauthorized store clerk can be detected by another system, and an illegal deterrent effect can be expected.

Next, various operation examples that can be operated in the security management system will be described.
This security management system has functions of ensuring safety in the store and preventing or deterring acts that are annoying other users in the store.
For example, in this security management system, an alarm may be sounded when a store visitor stays in the store for a predetermined time or longer in order to prevent or suppress the sitting behavior in the store for a long time. For example, the controller 31 periodically checks the history data of the store visitors stored in the storage device 32 on the basis of the time counted by the clock 31a, and more than a predetermined time from the store entry date and time. For the shoppers who have passed, the alarm device 37 may notify the monitoring center 14 of an alarm.

  In this security management system, it is assumed that each camera constantly monitors the state in the store in order to prevent the suspicious object from being left behind and the facility from being destroyed. For example, an image of a predetermined location that is captured in a state where no person is present is stored in the storage device 32, and an image of the location is periodically captured in the absence of a person. As a result, it is possible to compare images without a person. According to such an image comparison, if there is a change in the previous and subsequent images, it can be determined that there is a possibility that a suspicious object has been left or the facility has been destroyed. Thus, by constantly monitoring the suspicious object leaving or the destruction of the facility, it is possible to prevent or suppress acts such as the suspicious object leaving or the facility destruction by the deterrence effect.

The figure which shows the structural example in the store where the security management system is applied. The block diagram which shows the hardware structural example for implement | achieving a security management system. The flowchart for demonstrating the example of the registration process of face data. The figure which shows the structural example of the apparatus installed in the entrance / exit vicinity of a store. The flowchart for demonstrating the example of a shop entry process. The flowchart for demonstrating the example of goods monitoring processing. The flowchart for demonstrating the example of the entrance process of a service facility. The flowchart for demonstrating the example of a payment process. The flowchart for demonstrating the example of a store leaving process.

Explanation of symbols

  C1-C13, Ca, Cb, Cc ... Camera, 2 (2a, 2b) ... Gate device, 3 (3a, 3b) ... Display device, 4 ... Door, 5 ... Register terminal, 11 ... Server device, 12 ... Face information Registration system, 13 ... In-store system, 14 ... Monitoring center, 21 ... Control device, 22 ... Face registration device, 23 ... Camera, 24 ... Card reader, 30 ... In-store server, 31 ... Controller, 31a ... Clock, 32 ... Storage device 33 ... Face matching device 34 ... Card reader 34A ... Card reader 34B ... Card reader 35 ... Camera control device 36 ... Register control device 37 ... Alarm device 38 ... Shelf open / close device 41 ... Face Verification unit, 42 ... commodity detection unit, 51 ... card reader / writer, 61 ... door, 62 ... button

Claims (10)

Storage means for storing face data of a person authorized to purchase a product;
A collation unit that collates a human face included in an image captured by an image capturing unit that captures an image including at least a face of a person approaching the product with face data of the person stored in the storage unit;
Permission means for permitting access to the product by the person when the face of the person included in the image photographed by the photographing means matches the face data of the person stored in the storage means by the collating means;
Detecting means for detecting that a person who has been permitted to access the product by the permission means has taken out the product;
Control means for storing in the storage means information indicating that the person has taken out the product based on a detection result by the detection means;
A security management system comprising: Further, the face of the person included in the image photographed by the second photographing means for photographing the image including at least the face of the person who settles the price for the product is collated with the face data of the person stored in the storage means. Second verification means for
Information stored in the storage means when the face of the person included in the image photographed by the second photographing means matches the face data stored in the storage means by the second collating means. Determining means for determining whether or not all the products taken out by the person are included in the products that the person settles based on
When it is determined by the determining means that the product settled by the person includes the product taken out by the person, a settlement means for performing a settlement process of the product settled by the person;
If the face of the person included in the image photographed by the second photographing means by the second collating means does not match the face data of the person stored in the storage means, or if the person pays off An output means for outputting an alarm when the product taken out by the person is not included;
The security management system according to claim 1, further comprising: Further, in the storage means, the face of the person included in the image taken by the second photographing means for photographing an image including at least the face of the person approaching the entrance to the service facility from the area where the product is arranged. A second collation means for collating the stored person's face data;
Information stored in the storage means when the face of the person included in the image photographed by the second photographing means matches the face data stored in the storage means by the second collating means. A determination means for determining whether the person has an unsettled product based on
Permission means for permitting the person to enter the service facility when the determination means determines that the person does not have unsettled goods;
When the face of the person included in the image photographed by the second photographing means by the second collating means does not match the face data of the person stored in the storage means, or when the person is identified by the judging means If it is determined that you have unsettled goods, the prohibition means to prohibit the person from entering the service facility;
The security management system according to claim 1, further comprising: In addition, a settlement means for performing a settlement process for the product taken out by the person,
Second control means for storing in the storage means information indicating that the product settled by the settlement means has been settled;
The face of the person included in the image photographed by the second photographing means for photographing the image including at least the face of the person approaching the entrance of the area where the product is arranged and the person stored in the storage means A second matching means for matching face data;
Information stored in the storage means when the face of the person included in the image photographed by the second photographing means matches the face data stored in the storage means by the second collating means. A determination means for determining whether the person has an unsettled product based on
If it is determined by this determination means that the person does not have unsettled goods, permission means for permitting participation from the area;
When the face of the person included in the image photographed by the second photographing means by the second collating means does not match the face data of the person stored in the storage means, or when the person is identified by the judging means If it is determined that you have unsettled goods, the prohibition means to prohibit the person from participating in the area,
The security management system according to claim 1, further comprising: First storage means for storing face data of a person authorized to enter the security area;
The person's face included in the image taken by the first photographing means for photographing an image including at least the face of the person entering the security area is collated with the person's face data stored in the second storage means. First collating means for
When the face of the person included in the image photographed by the first photographing means matches the face data of the person stored in the first storage means by the first collating means, the person is in the security area. Admission means for allowing admission to the inside,
Second storage means for storing face data and history data of a person permitted to enter the security area by the entrance permission means, as visitor data;
The face of the person included in the image photographed by the second photographing means for photographing the image including at least the face of the person approaching the product arranged in the security area and stored in the second storage means. A second matching means for matching the face data of the visitors;
When the face of the person included in the image photographed by the second photographing means matches the face data of the person stored in the second storage means by the second collating means, An access permission means for permitting access;
Detection means for detecting that a person who has been permitted to access the product by the access permission means has taken out the product;
Control means for storing, in the second storage means, information indicating that the person has taken out the product based on a detection result by the detection means, as history data of the person;
A security management system comprising: Further, the face of the person included in the image photographed by the third photographing means for photographing an image including at least the face of the person who settles the price for the product and the face data of the person stored in the second storage means A third collating means for collating
When the face of the person included in the image photographed by the third photographing means matches the face data of the person stored in the second storage means by the third collating means, the second storage means Determining means for determining whether or not all the products taken out by the person are included in the products settled by the person based on the information stored in
When it is determined by the determining means that the product settled by the person includes the product taken out by the person, a settlement means for performing a settlement process of the product settled by the person;
When the face of the person included in the image photographed by the third photographing means by the third collating means does not match the face data of the person stored in the second storage means, or when the person is settled An output means for outputting an alarm when the product to be taken out does not include the product taken out by the person,
The security management system according to claim 5, further comprising: Further, the face of the person included in the image taken by the third photographing means for photographing at least the face of the person approaching the entrance to the service facility from the security area and the second storage means are stored. A third collation means for collating the face data of the person
When the face of the person included in the image photographed by the third photographing means matches the face data of the person stored in the second storage means by the third collating means, the second storage means Determining means for determining whether or not the person possesses an unsettled product based on information stored in
Permission means for permitting the person to enter the service facility when the determination means determines that the person does not have unsettled goods;
When the face of the person included in the image photographed by the third photographing means by the third collating means does not match the person face data stored in the second storage means, or by the judging means A prohibition means for prohibiting the person from entering the service facility when it is determined that the person has unsettled goods;
The security management system according to claim 5, further comprising: In addition, a settlement means for performing a settlement process for the product taken out by the person,
Second control means for storing in the second storage means information indicating that the product settled by the settlement means has been settled;
The face of the person included in the image captured by the third image capturing unit that captures an image including at least the face of the person approaching the exit of the security area is collated with the face data of the person stored in the storage unit. A third verification means;
When the face of the person included in the image photographed by the third photographing means matches the face data of the person stored in the second storage means by the third collating means, the second storage means Determining means for determining whether or not the person possesses an unsettled product based on information stored in
If it is determined by this determination means that the person does not have unsettled goods, permission means for permitting participation from the security area;
When the face of the person included in the image photographed by the second photographing means by the third collating means does not match the face data of the person stored in the second storage means, or by the judging means A prohibition means for prohibiting the person from participating in the area when it is determined that the person has unsettled goods;
The security management system according to claim 5, further comprising: A security management method used in a security management system,
Store the face data of the person who has the authority to purchase products in the storage means,
Collating the face of a person included in an image obtained by photographing an image including at least the face of a person approaching the product with the face data of the person stored in the storage means;
When the face of the person included in the photographed image matches the face data of the person stored in the storage unit by this collation, the person is allowed to access the product,
Detecting that a person authorized to access the product has removed the product;
Storing in the storage means information indicating that the person has taken out the product based on the detection result;
A security management method characterized by the above. A security management method used in a security management system,
Store face data of a person authorized to enter the security area in the first storage means,
Collating the face of the person included in the image obtained by photographing the image including at least the face of the person entering the security area with the face data of the person stored in the first storage means;
If the face of the person included in the captured image matches the face data of the person stored in the first storage means by this collation, the person is allowed to enter the security area,
Storing the face data and history data of the person permitted to enter the security area in the second storage means as the visitor data;
The face of the person included in the image obtained by photographing the image including at least the face of the person approaching the product arranged in the security area is matched with the face data of the visitor stored in the second storage means And
If the face of the person included in the photographed image matches the face data of the person stored in the second storage means by this collation, the person is allowed to access the product,
This permission detects that a person authorized to access the product has taken out the product,
Based on the detection result, information indicating that the person has taken out the product is stored in the second storage means as history data of the person.
A security management method characterized by the above.

Images