JP4966896B2 - Behavior management device - Google Patents

Description

  The present invention relates to a behavior management apparatus that manages and records a behavior of a person in a store of a financial institution.

2. Description of the Related Art Conventionally, in a store such as a financial institution, an action management device is known in which a plurality of photographing devices such as a digital video recorder are provided in a store and an action of a person such as a customer is recorded as an image.
Further, as this type of behavior management device, there is a device that obtains position coordinates of a plurality of passers-by based on an image, and records and displays a pass track for each passer-by based on the position coordinates (see, for example, Patent Document 1). ).
JP 2005-346617 A

By the way, in the above-described behavior management device, in the former case, when searching for the behavior of a specific customer, a desired video must be searched from videos recorded for a long time. In addition, if an image captured by a digital video recorder or the like is to be stored for a long period of time, the amount of data becomes enormous, which makes it difficult to store the image for a long period of time.
On the other hand, in the latter case, it is impossible to identify each customer, and it is necessary to estimate and extract a predetermined behavior from all the trajectories. There is a problem that it is difficult to detect itself.

  The present invention has been made in view of the above circumstances. An action management apparatus capable of recording a person's action with a small amount of data in a store of a financial institution or the like and easily searching for an action related to a specific person. It is to provide.

In order to solve the above-mentioned problem, the invention described in claim 1 is an action management device that manages a person's action in a predetermined area, and includes an imaging unit that captures an image in the predetermined area, and the imaging unit. Person identifying means for extracting the facial features of a person from the captured image, and authenticating the person based on the facial features of the person extracted by the person identifying means, and whether the authenticated person is a salesclerk or a customer Authentication means for determining whether or not the user is authenticated, action detection means for detecting a predetermined action by the person authenticated by the authentication means, and a character for converting the predetermined action specified by the action detection means into character data A data conversion unit; and character data converted by the character data conversion unit are stored in association with identification information of a person authenticated by the authentication unit ; Storage means for storing layout information relating to a layout in the predetermined area , wherein the behavior detection means is a person determined to be a store clerk by the authentication means, and a person determined to be a customer by the authentication means And detecting the behavior of a person who has been determined to be a customer by at least the authentication means, based on the positional relationship with each other, the business content handled by the store clerk, and the layout information stored in the storage means To do.

  According to a second aspect of the present invention, in the first aspect of the invention, when the person specified by the person specifying unit cannot be authenticated by the authenticating unit, the identification for newly specifying the unauthenticated person An identification information providing unit (for example, an identification information adding unit 36 in the embodiment) for adding information, and the storage unit associates the identification information newly provided by the identification information providing unit with character data. It is memorized.

  According to a third aspect of the present invention, in the invention according to the second aspect, the storage means includes identification information newly given by the identification information giving means and a video of a person to whom the identification information is given. The information is stored in association with each other.

  The invention described in claim 4 is the invention according to claim 2 or 3, wherein the authentication means is specified by the person specifying means including identification information newly given by the identification information giving means. It is characterized by performing person authentication.

  According to a fifth aspect of the present invention, in the first aspect of the present invention, the video when the behavior detecting unit detects a predetermined behavior from the video captured by the imaging unit. An extraction unit (for example, a still image extraction unit 38 in the embodiment) for extracting as a still image is provided, and the storage unit stores the still image extracted by the extraction unit in association with the character data. To do.

According to a sixth aspect of the present invention, in the invention according to any one of the first to fifth aspects, the article detection means (for example, an embodiment) that identifies a predetermined article from the video imaged by the imaging means. The behavior detection means detects the behavior of the person authenticated by the authentication means based on the type of the article specified by the article detection means.

The invention described in claim 7 is the invention according to any one of claims 1 to 6 , wherein the storage means associates the video of the person authenticated by the authentication means with the identification information of the person. It is memorized.

The invention described in claim 8 is the invention described in any one of claims 1 to 7 , wherein keyword input means for receiving input of a predetermined keyword (for example, keyword input unit 41 in the embodiment); Searching the identification information stored in the storage means and the related information stored in association with the identification information for the identification information matching the keyword input by the keyword input means or the identification information including the related phrase Search means (for example, the search unit 39 in the embodiment) for outputting necessary information from the identification information searched by the search means and the related information.

According to the first aspect of the present invention, a person is specified by the person specifying means from the video imaged by the image pickup means, and the specified person is authenticated by the authentication means, and further specified by the authentication means. A predetermined action of the person is detected by the action detection means, and the predetermined action detected by the action detection means is converted into character data by the character data conversion means, whereby the identification information of the specified action is added to the identified person identification information. Character data can be associated and stored in the storage means. Therefore, the amount of data can be greatly reduced by storing the predetermined action as character data compared to the conventional method of storing all captured images, and for example, based on the identification information, There is an effect that an action can be easily searched.
In addition, layout information relating to the layout in a predetermined area stored in the storage means, for example, information such as a position where a window counter or a reception ticket issuing machine is arranged, and a video imaged by the imaging means, for example, authentication means Based on the image of the authenticated person being at the place of the reception ticket issuing machine, the action detection means can detect the action of the person.
Furthermore, by determining whether the person specified by the person specifying means is a clerk or a customer by the authentication means, the positional relationship between the clerk and the customer can be detected. For example, the customer is located near the clerk at the deposit counter. Since it can be estimated that the customer makes a deposit when moving, the behavior detecting means can detect the predetermined behavior based on the positional relationship between the clerk and the customer and the layout information in the predetermined area.

  According to the second aspect of the present invention, in addition to the effect of the first aspect, when the person specified by the person specifying means cannot be authenticated by the authentication means, that is, when the person specified by the first visit or the like is authenticated If the storage means does not have any information necessary for the storage means, the identification information providing means assigns identification information for newly identifying a person who cannot be authenticated, and character data conversion means for the identification information. The character data converted by the above can be stored in association with each other, so that the predetermined action of the person who cannot be authenticated can be stored in the storage means.

  According to the third aspect of the invention, in addition to the effect of the second aspect, the identification information newly given by the identification information giving means and the video of the person can be associated and stored in the storage means. Later, an unauthenticated person can be identified based on the video.

  According to the invention described in claim 4, in addition to the effect of claim 2 or 3, the authentication means authenticates the person specified by the person specifying means including the identification information newly given by the identification information giving means. Can be performed. That is, the authentication means can perform behavior management such as second, third, etc. visit confirmation not only for customers who can authenticate personal names but also for persons who cannot authenticate personal names.

  According to the fifth aspect of the present invention, in addition to the effect of any one of the first to fourth aspects, a still image is extracted from a moving image that is a video imaged by the imaging means and stored in association with character data. Therefore, the amount of data can be reduced by the amount stored as a still image as compared with the case where the moving image is stored in the storage means as it is. In addition, the action evidence image recorded as character data can be recorded with less data than moving image recording.

According to the invention described in claim 6 , in addition to the effect of any one of claims 1 to 5, by specifying a predetermined article from the captured image by the article detector, for example, at a deposit counter Since it can be presumed that the deposit is made when the bankbook or the seal is specified, the behavior detecting means can detect a predetermined behavior according to the type of the article.

According to the invention described in claim 7 , in addition to the effect of any one of claims 1 to 6 , by storing the video of the person authenticated by the authentication unit in association with the information related to the person, For example, when searching for an image of an authenticated person, the image of the person can be immediately specified based on the identification information of the person.

According to the invention described in claim 8 , in addition to the effect of any one of claims 1 to 7 , the keyword input means includes identification information that matches the keyword accepted by the input, or a phrase related to the keyword. Since the identification information can be searched by the search means, the related information associated with the identification information can be searched based on the keyword and can be called from, for example, the storage means, so that the related information of the desired customer can be easily called. There is an effect that can.

Next, an action management apparatus according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows a schematic configuration of the behavior management apparatus 10 in this embodiment. The behavior management device 10 is a device that is installed in a store such as a financial institution to monitor and record a person's behavior, and includes a plurality of video cameras 20, a control device 30, a keyword input unit 41, and a display unit 40. It is prepared for.

  The plurality of video cameras 20 are for capturing a person at each location in the store of the financial institution, and each image captured by the plurality of video cameras 20 is controlled together with an identifier of the captured video camera 20. Transmitted to the device 30. FIG. 2 shows an example of the layout and camera arrangement in the store. The video camera 20 includes a front door 50, an ATM corner 51, each ATM 52, a store lobby 53, a counter counter 54, a copy machine 55, a safe deposit box 56, And it is arrange | positioned in the location which can image | video images, such as the reception ticket issuing machine 57, respectively. Each video camera 20 can be identified by its identifier.

  The control device 30 is configured by a computer such as a server arranged in the counter counter 54 of the store. As shown in FIG. 1, a person specifying unit 31, an authentication unit 32, an action detection unit 33, a character data conversion unit 34, a storage unit 35, an identification information adding unit 36, an article detection unit 37, a still image extraction unit 38, and a search unit 39. Further, a display unit 40 such as a liquid crystal and a keyword input unit 41 including a keyboard and a mouse are connected to the control device 30.

  The person specifying unit 31 specifies a person from the screens shown in each video based on the video captured by each video camera 20, that is, extracts a human image. As a method for identifying a person based on an image or a method for extracting an image, for example, a person's face is detected from images captured by each video camera 20, and the detected human face features are extracted. Various methods such as performing person identification can be used. In other words, the person specifying unit 31 cuts out an image of a person from the images displayed by the video cameras 20, and further, for example, a front image of the face so that the authentication unit 32 described later can easily authenticate. Is to be cut out.

  The authentication unit 32 performs authentication of the person specified by the person specifying unit 31. For example, information on the facial features of the person extracted by the person specifying unit 31 is stored in the storage unit 35 or the like in advance. The person is searched from the person list, and the person having the corresponding facial feature is authenticated as the person. That is, who is authenticated. Further, the authentication unit 32 determines whether the authenticated person is a store clerk or a customer based on information stored in the storage unit 35 in advance. Here, the authenticated person has identification information for identifying the person, and information indicating that the person is a clerk or a customer is associated with the identification information and stored in the storage unit 35 in advance. ing.

  The behavior detecting unit 33 detects a predetermined behavior of the person specified by the person specifying unit 31, and stores store layout information (see FIG. 2) stored in the storage unit 35 in advance, and a video camera. The predetermined action of the person specified by the person specifying unit 31 is detected based on the video imaged by 20. In addition, the behavior detection unit 33 is based on the positional relationship between the person who is determined to be a store clerk by the authentication unit 32 and the person who is authenticated as the customer by the authentication unit 32, and the layout information inside the store. Thus, it is configured to detect a predetermined action performed by the person.

  The character data conversion unit 34 converts the predetermined behavior detected by the behavior detection unit 33 into character data, and transmits the converted character data of the predetermined behavior to the storage unit 35.

  The storage unit 35 stores the character data of the predetermined action converted by the character data conversion unit 34 in association with the identification information of the authenticated person. Further, the storage unit 35 stores the character data of the predetermined action converted by the character data conversion unit 34 in association with the identification information (hereinafter simply referred to as temporary identification information) given by the identification information addition unit 36. To do. Further, the storage unit 35 stores the image extracted by the below-described still image extraction unit 38 in association with the identification information and the provisional identification information.

  The identification information providing unit 36 provides temporary identification information to a person who cannot be recognized by the authentication unit 32 described above, that is, a customer who has not opened an account. Here, as the temporary identification information, for example, identification information such as “Unknown1”, “Unknown2”,... “UnknownN” is given.

The article detection unit 37 detects a predetermined article based on images captured by the video camera 20. Examples of the predetermined article include “passbook” and “seal”.
The still image extracting unit 38 extracts a still image of the person specified by the person specifying unit 31 from a video obtained by capturing the person when the action detecting unit 33 detects a predetermined action.

  The search unit 39 searches the information stored in the storage unit 35 for information that matches the predetermined keyword input via the keyword input unit 41 and information including at least a phrase related to the keyword. Necessary information is output to the display unit 40 from the received information.

The behavior management apparatus 10 according to this embodiment has the above-described configuration. Next, the operation of the behavior management apparatus 10 will be described. The operation of the behavior management apparatus 10 will be described as an example when the behavior management apparatus 10 is installed in a store of a financial institution.
First, when the action management apparatus 10 is turned on, imaging by the video camera 20 is started, and the captured images are sequentially transmitted to the person specifying unit 31.
The control device 30 determines whether or not a person is included in the video by the person specifying unit 31. As a result of the determination, if a person is included in the video, the person is specified and the information is transmitted to the authentication unit 32. If the person is not included, the video is continuously monitored as it is.

  When the person is specified, the authentication unit 32 authenticates the specified person. Specifically, the authentication unit 32 determines whether or not the person information stored in advance in the storage unit 35 includes information that matches the person information specified by the person specifying unit 31. As a result of the determination, if the storage unit 35 includes information on a person that matches the information on the person specified by the person specifying unit 31, the identification information associated with the person information in the storage unit 35 is specified. Then, the identification information is transmitted to the behavior detection unit 33 and the storage unit 35. The authentication unit 32 determines whether the authenticated person is a customer or a store clerk based on information indicating a store clerk or a customer associated with the identification information in the storage unit 35. When authenticating by the authentication unit 32, authentication with a person who has not been authenticated in the past who has been provided with temporary identification information in the previous authentication processing is also performed. For example, the authentication unit 32 stores in the storage unit 35 in the past. When the authentication unit 32 determines that the facial feature information of the person having the provisional identification information “Unknown 325” matches, the temporary identification information “Unknown 325” is transmitted to the behavior detection unit 33 and the storage unit 35.

  On the other hand, if the authentication unit 32 determines that the information of the person that matches the information of the person specified by the person specifying unit 31 is not stored in the storage unit 35, the customer who has not been authenticated by the identification information adding unit 36 For the customer, temporary identification information for the customer that could not be authenticated by the authenticating unit 32 is newly given. Then, the temporary identification information for the customer that could not be authenticated is transmitted to the behavior detection unit 33 and the storage unit 35. When the temporary identification information is received from the identification information providing unit 36, the storage unit 35 stores the temporary identification information as new registration information.

  In parallel with the processing by the person identification unit 31 and the authentication unit 32, the article detection unit 37 detects a predetermined article registered in advance from the video, and detects the information on the detected article as a behavior. To the unit 33. Further, the still image extraction unit 38 extracts a still image including a person when the action detection unit 33 detects a predetermined action from a video (moving image) obtained by capturing the person.

  Next, the customer image associated with the identification information received from the authentication unit 32 or the temporary identification information received from the identification information adding unit 36, the layout stored in the storage unit 35, and the camera that captured the image Based on the arrangement, the behavior detection unit 33 detects a predetermined behavior of the customer. For example, when the customer enters the store from the front entrance 50, “Visit from the front entrance” is detected, and when the customer is imaged by the video camera 20 disposed in the ATM corner 51, direct action is taken. "Operating ATM of No. N" is detected. Alternatively, “deposit”, “withdrawal”, “transfer”, “passbook entry”, “exchange”, and the like are detected as predetermined actions that can be inferred. Further, when a customer is detected from an image of a camera that captures the vicinity of a reception ticket issuing machine (not shown), “reception” is detected as a predetermined action.

  Further, the behavior detection unit 33 detects the positional relationship between the customer and the store clerk from the captured image, and detects a predetermined behavior based on the positional relationship. For example, when it is detected that the customer has moved close to a store clerk who is in charge of a predetermined business, “transaction at N-th window” or the like is detected as a direct action. Alternatively, when the business content handled by the store clerk is account opening or cancellation, "account opening" or "cancellation" is detected as a predetermined action that can be analogized.

  Further, the behavior detection unit 33 detects a predetermined behavior of the customer based on the type of the article detected by the article detection unit 37. For example, when it is detected that a customer has taken out a card or passbook at the ATM corner 51, the direct actions are “take out the card from the wallet”, “insert the card into the ATM”, “ Detecting “take out from bags”. Further, “deposit”, “withdrawal”, “passbook entry” and the like are detected as predetermined actions that can be inferred. Then, when it is detected that the customer is at a window for performing a predetermined job and a seal is further detected, “take out the seal from the bags” or the like is detected as a direct action. Or since it is limited to the operation | work which uses a seal | sticker from the predetermined | prescribed business, ie, the action which uses a seal | sticker, this limited action is detected as a customer's predetermined | prescribed action.

  Next, the character data conversion unit 34 converts the predetermined behavior detected by the behavior detection unit 33 into character data, and the storage unit 35 extracts the predetermined behavior converted into character data and the still image extraction unit 38. Each still image is stored as related information in association with identification information.

  Here, an example of information stored in the storage unit 35 described above is shown in the following table. Note that “jpg” in the image in the table indicates an example of a file extension. In this table, for convenience, data such as facial features for specifying a person is omitted. The date / time information is also stored in association with each stored information, but is omitted in this table. Furthermore, in this table, only one piece of stored information related to one action is shown for one piece of identification information or one piece of temporary identification information, but this is not restrictive. From the time of visiting the store to the time of leaving the store, the behavior to be recorded is recorded by the amount detected and confirmed by the behavior detecting unit 33.

  On the other hand, when a person in charge of the store or the like inputs a keyword (for example, “01”, “payment”, or a customer name) for searching related information in the storage unit 35 via the keyword input unit 41. From the related information stored in the storage unit 35, the search unit 39 searches for the identification information that matches the keyword and the identification information including the phrase related to the keyword. Will be displayed on the display unit 40.

  In the above-described embodiment, a case has been described in which customer behavior is stored in association with identification information, but store clerk behavior may be stored in the same manner. Moreover, although the example which provides the action management apparatus 10 in the store of financial institutions, such as a bank, was demonstrated, it is not restricted to this structure, For example, you may make it provide in places other than a store other than a financial institution, or a store.

  Therefore, according to the above-described embodiment, the person specifying unit 31 specifies a person from the image inside the store captured by the video camera 20, authenticates the person specified by the authentication unit 32, and performs the action detecting unit. 33, the predetermined action of the person identified by the authentication unit 32 is detected by 33, and the person identified by converting the predetermined action detected by the action detection unit 33 by the character data conversion unit 34 into character data. The information can be stored in the storage unit 35 in association with the information. As a result, the amount of data can be greatly reduced by the amount that can be stored as character data compared to the conventional method of storing all captured images. Based on the above, it is possible to easily search for the action of a specific person.

  When the person specified by the person specifying unit 31 cannot be authenticated by the authenticating unit 32, that is, when the specified person is specified by the person specifying unit 31 for the first time, the identification information is registered in the storage unit 35. If there is not, provisional identification information for identifying a person who cannot be authenticated is assigned by the identification information adding unit 36, and the character data converted by the character data conversion unit 34 is associated with the temporary identification information. Since it can be memorized, a predetermined action of a person who cannot be authenticated can be memorized.

  Then, by extracting a still image from the moving image picked up by the video camera 20 and storing it in association with the character data, the amount of data is reduced by the amount stored as a still image compared with the case of storing a moving image. it can.

  In addition, layout information regarding the layout inside the store, for example, information such as the position where the counter counter 54 and the reception ticketing machine 57 are arranged, and images captured by the video camera 20, for example, the location of the reception ticketing machine 57 The action detection unit 33 can detect a person's action based on the video such as being in the room.

  Further, the authentication unit 32 determines whether the person specified by the person specifying unit 31 is a store clerk or a customer. When it moves to the vicinity of the store clerk, it is presumed that the customer deposits, so that the behavior detection unit 33 can detect the behavior according to the position of the customer.

  Then, by specifying a predetermined article from the captured image by the article detection unit 37, for example, when a deposit passbook or a seal is specified at a deposit window, it is estimated that a deposit is made. The behavior detection unit 33 can detect the behavior corresponding to the type of the behavior.

  Furthermore, by storing the video of the person specified by the person specifying unit 31 in association with the information related to the person, for example, when searching based on the person identification information and the provisional identification information, the identification information and It is possible to immediately specify the video of the person associated with the temporary identification information.

  In addition, since the keyword input unit 41 can search the identification information and provisional identification information that matches the keyword accepted by the keyword input, or the identification information and provisional identification information including a phrase related to the keyword, by the search unit 39. Since the related information associated with the identification information and the temporary identification information can be retrieved based on the keyword and called from the storage unit 35, the related information of the desired customer can be easily called.

  In the above-described embodiment, the case where the still image is stored in association with the identification information has been described. However, the present invention is not limited to the above-described embodiment, and for example, a moving image showing a customer corresponds to the identification information. It may be added and stored. With this configuration, the information given by the identification information giving unit 36 and the given person's video can be stored in the storage unit 35 in association with each other. It becomes possible.

  Further, as a modification of the above-described embodiment, for example, when the person to whom the temporary authentication information is assigned is specified by the identification information adding unit 36, the temporary information stored in the storage unit 35 is specified. The identification information may be updated to new identification information. Furthermore, when the control device 30 is connected to a server outside the store such as a headquarters server via a network, and new information related to the customer stored in the storage unit 35 is registered, the storage unit 35 is based on this registration information. The temporary identification information and related information of the person stored in the table may be updated.

  Furthermore, in the above-described embodiment, the case where the predetermined behavior related to work is mainly stored has been described. For example, a video camera 20 that captures the sofa in the store from the side is installed, and a person stands up. A simple movement of a person such as being seated may be detected as a predetermined action, and character data of this action may be associated with the identification information and used as the storage unit 35.

  In the above-described embodiment, the case where the identification information and temporary identification information stored in the storage unit 35 are searched based on the keyword has been described. However, related information may be searched based on the keyword. good. Moreover, as information memorize | stored in the memory | storage part 35, it is good also as a structure which links | relates and memorize | stores the other information regarding the person other than predetermined | prescribed action, a moving image, and a still image with the identification information and temporary identification information mentioned above.

It is a schematic structure figure of action management device 10 in an embodiment of the invention. It is a figure which shows an example of the layout of the store in embodiment of this invention.

Explanation of symbols

20 Video camera 31 Person specifying part (person specifying means)
32 Authentication section (authentication means)
33 Action detection unit (behavior detection means)
34 Character data converter (character data converter)
35 storage unit (storage means)
36 Identification information adding unit (identification information adding means)
37 Article detection unit (article detection means)
38 Still image extraction unit (extraction means)
39 Search part (search means)
41 Keyword input section (keyword input means)

Claims (8)

In an action management device that manages the actions of persons in a predetermined area,
Imaging means for imaging video in the predetermined area;
Person identifying means for extracting features of a person's face from the video imaged by the imaging means;
Authenticating means for authenticating the person based on the facial features of the person extracted by the person identifying means, and determining whether the authenticated person is a store clerk or a customer ;
Action detecting means for detecting a predetermined action by a person authenticated by the authenticating means;
Character data conversion means for converting the predetermined action specified by the action detection means into character data;
Storing the character data converted by the character data conversion means in association with the identification information of the person authenticated by the authentication means, and storing the layout information relating to the layout in the predetermined area ,
The behavior detection means includes a positional relationship between a person determined to be a store clerk by the authentication means and a person determined to be a customer by the authentication means, a business content handled by the store clerk, and the storage means. An action management apparatus that detects an action of a person who is determined to be a customer by at least the authentication unit based on stored layout information . When the person specified by the person specifying means cannot be authenticated by the authentication means, the information specifying means for adding identification information for newly specifying a person who cannot be authenticated is provided.
The behavior management apparatus according to claim 1, wherein the storage unit stores the identification information newly given by the identification information addition unit and character data in association with each other.   The behavior management apparatus according to claim 2, wherein the storage unit stores the identification information newly given by the identification information addition unit and the video of the person to whom the identification information is given in association with each other. .   The behavior management according to claim 2 or 3, wherein the authentication unit authenticates the person specified by the person specifying unit including the identification information newly added by the identification information adding unit. apparatus. Extraction means for extracting, as a still image, a video when the behavior detection means detects a predetermined behavior from the video imaged by the imaging means,
5. The behavior management apparatus according to claim 1, wherein the storage unit stores the still image extracted by the extraction unit in association with the character data. 6. Further comprising an article detection means for identifying a predetermined article from the video imaged by the imaging means,
The said action detection means detects the action of the person authenticated by the said authentication means based on the kind of article | item specified by the said article detection means, The Claim 1 thru | or 5 characterized by the above-mentioned. Behavior management device. Wherein the storage unit, the image of the authenticated person by the authentication unit, the behavior management device according to any one of claims 1 to 6, characterized in that in association with the identification information of the person. A keyword input means for receiving input of a predetermined keyword;
Identification information including identification information that matches the keyword input by the keyword input means or related information from the identification information stored in the storage means and the related information stored in association with the identification information. A search means for searching,
The behavior management apparatus according to any one of claims 1 to 7 , wherein necessary information is output from the identification information searched by the search means and the related information.

Images