JP7109520B2 - Purchased product estimation device - Google Patents

Description

The present disclosure relates to a people flow analysis method for analyzing the flow of people, a people flow analysis device, and a people flow analysis system including the people flow analysis device. The present invention relates to an analysis device and a people flow analysis system provided with the people flow analysis device.

In recent years, many commercial facilities such as shopping malls with many shops have been built, and users can purchase various products from many shops. In such commercial facilities or each store, in order to grasp the consumption trends of users and further increase sales, by investigating what stores the users visited and in what order they purchased products. People flow analysis is being conducted to analyze the flow of people.

Regarding the above-mentioned people flow analysis, for example, in Patent Document 1, a flow line management system includes a registrant database for registering face images, a face detection engine for detecting face images from images taken in a plurality of shooting areas, Equipped with a face recognition engine that matches face images detected by the face detection engine with face images registered in the registrant database, it calculates and records the flow lines of customers reflected in images taken in multiple shooting areas. is disclosed.

In addition, Patent Document 2 includes an information processing terminal and an information processing system connected to an access point for short-range wireless communication whose communication range is set according to the section of each store, and the information processing terminal is connected to the store. a store entry information transmission unit that transmits store entry information of a customer to an information processing system in response to acquisition of user identification information from a customer's mobile terminal that has entered the communication range of an installed access point; and a mobile terminal. and a store exit information transmission unit that transmits the customer's store exit information to the information processing system in response to disconnection of the short-range wireless communication established between the customer and the information processing system. An information collection system including a store information recording unit, a store exit information recording unit that records store exit information, and an action history information generation unit that generates action history information based on the store entry information and store exit information of each customer. is disclosed.

JP 2010-55594 A JP 2016-177539 A

However, in the above-described conventional technology, property owned by a person is not considered at all, and further improvement is needed for a method of analyzing the order of stores visited by a person.

The present disclosure has been made to solve the above problems, and provides a people flow analysis method, a people flow analysis device, and a people flow analysis system that can accurately determine the order of stores visited by people with a simple configuration. It is intended to provide

A people flow analysis method according to an aspect of the present disclosure is a people flow analysis method in a people flow analysis device, in which an appearance image of a person is acquired, the person is identified from the appearance image, and a person owned by the person is identified from the appearance image. Identifying a property, estimating a store from which the property was obtained from the identified property, and acquiring personal information indicating the identified person, store information indicating the estimated store, and the exterior image. store in a storage unit in association with time information indicating the time when the person visited the store, and determine the order of stores visited by the person based on changes in time series of stores indicated by the store information stored in the storage unit. do.

Advantageous Effects of Invention According to the present disclosure, it is possible to accurately determine the order of stores visited by a person with a simple configuration.

1 is a block diagram showing an example of a configuration of a people flow analysis system according to Embodiment 1 of the present disclosure; FIG. FIG. 10 is a schematic diagram showing an example of a time-series change of a person image; FIG. 2 is a diagram showing an arrangement example of the cameras shown in FIG. 1; FIG. 2 is a block diagram showing an example of the configuration of the people flow analysis device shown in FIG. 1; FIG. FIG. 10 is a schematic diagram for explaining an example of a change in properties owned by a person; 5 is a diagram showing an example of an image file table created by an image information acquisition unit shown in FIG. 4; FIG. 5 is a diagram showing an example of a person identification table used by the person identification unit shown in FIG. 4; FIG. It is a figure which shows an example of the table for store estimation which the store estimation part shown in FIG. 4 uses. It is a figure which shows an example of the store table which the visit order determination part shown in FIG. 4 uses. It is a figure which shows an example of a store-visit order estimation table stored in the database shown in FIG. FIG. 11 is a schematic diagram for explaining another example of changes in property owned by a person; 5 is a diagram showing another example of a store-visit order estimation table stored in the database shown in FIG. 4; FIG. It is a figure which shows an example of the integrated store table which the visit order determination part shown in FIG. 4 produces. 2 is a sequence diagram showing an example of people flow analysis processing by the people flow analysis system shown in FIG. 1; FIG. FIG. 7 is a block diagram showing an example of a configuration of a people flow analysis device of a people flow analysis system according to Embodiment 2 of the present disclosure; FIG. 16 is a diagram showing an example of a person group table created by the person group identifying unit shown in FIG. 15; 16 is a diagram showing an example of a visit order estimation table stored in the database shown in FIG. 15; FIG. FIG. 12 is a sequence diagram showing an example of people flow analysis processing by the people flow analysis system according to Embodiment 2 of the present disclosure; FIG. 10 is a schematic diagram showing a first arrangement example when using a camera for photographing a person and a camera for photographing a property; FIG. 10 is a side view of a person holding a luggage bag; FIG. 11 is a front view of a person holding a luggage bag; FIG. 12 is a schematic diagram showing a second example of arrangement when a camera for photographing a person and a camera for photographing a property are used; FIG. 2 is a schematic diagram showing a first example of a hand-held portion of a luggage bag and how to hold the luggage bag in this example; FIG. 5 is a schematic diagram showing a second example of a hand-held portion of a luggage bag and how to hold the luggage bag in this example; FIG. 10 is a schematic diagram showing a third example of a hand-held portion of the luggage bag and how to hold the luggage bag in this example; FIG. 3 is a diagram showing a first example of a sticking member to be stuck on a luggage bag; FIG. 10 is a diagram showing a second example of a sticking member to be stuck on a luggage bag; It is a figure which shows an example of the 1st color information table which shows the correspondence of the kind of goods, and the color of an attachment member. FIG. 10 is a diagram showing an example of a second color information table showing the correspondence relationship between the price of the product and the color density of the attachment member; FIG. 20 is a diagram showing an example of an exterior image captured by a camera that captures an exterior image of the luggage bag shown in FIG. 19; FIG. 4 is a diagram showing an example of a sticking member to which a barcode is attached;

(Findings on which this disclosure is based)
In Patent Document 1 above, surveillance cameras are arranged in a plurality of photographing areas (for example, a front entrance, a cosmetics department, a miscellaneous goods department, a women's clothing department, a side entrance, etc.), and images of customers in the photographing areas are captured. are getting Therefore, in a commercial facility with a large number of shops, it is necessary to arrange surveillance cameras for each shop or sales floor. Therefore, it is necessary to use a large number of monitoring cameras, which complicates the configuration of the flow line management system.

Moreover, in Patent Document 2, access points installed in stores are also used, so in a commercial facility with many stores, it is necessary to arrange an access point for each store or sales floor. Therefore, it is necessary to use a large number of access points, which complicates the configuration of the information gathering system.

Further, in Japanese Patent Laid-Open No. 2002-100003, facial images are detected from photographed images and compared with facial images registered in a registrant database, so that the flow lines of customers reflected in images photographed in a plurality of photographing areas are calculated. Calculated and recorded. However, when a person's face is not photographed so as to be sufficiently identifiable, the person cannot be accurately identified only by the face image, and the order of the stores visited by the person cannot be accurately determined.

On the other hand, the luggage bag (including bags, wrapping paper, etc.) handed to customers at the store when they purchase products has the store name and brand name printed on it, so after purchasing products, they can The customer has a luggage bag as property. Therefore, by recognizing the luggage bag possessed by a person and taking the difference from the luggage bag held by the same person immediately before, the order in which the person moved between stores can be known. In this way, by using a person's belongings, it is possible to accurately determine the store visited by the person.

Based on the above findings, the inventors of the present application have made extensive studies on how to determine the order of stores visited by a person with a simple configuration and with high accuracy. , completed the present disclosure.

A people flow analysis method according to an aspect of the present disclosure is a people flow analysis method in a people flow analysis device that analyzes the flow of people, in which an appearance image of a person is acquired, the person is identified from the appearance image, and the person is identified from the appearance image. identifying the property owned by the person from the above, estimating the store that acquired the property from the identified property, personal information indicating the identified person, and store information indicating the estimated store and time information indicating the time when the appearance image was acquired is stored in a storage unit, and the person visits the store based on a time-series change in the store indicated by the store information stored in the storage unit. determine the order of the stores that

With such a configuration, an appearance image of a person is acquired, the person is identified from the acquired appearance image, the property owned by the person is specified from the acquired appearance image, and the property is acquired from the specified property. The store is estimated, and the person information indicating the identified person, the store information indicating the estimated store, and the time information indicating the time when the exterior image was acquired are associated and stored in the storage unit, and stored in the storage unit. Based on the time-series changes in the stores indicated by the store information, the order of the stores visited by the person is determined.

Therefore, without arranging a camera for capturing an exterior image for each store, it is possible to identify the property from the exterior image of the person and accurately estimate the store that acquired the property from the identified property. It is possible to determine a time-series change in the store visited, for example, a newly added store as the next store visited. As a result, the order of stores visited by a person can be determined with high accuracy with a simple configuration.

The property may include a luggage bag obtained at the store.

With such a configuration, since the luggage bag acquired at the store has a mark, color, shape, etc. that can identify the store, it is possible to accurately estimate the store that acquired the luggage bag from the luggage bag. Therefore, the order of stores visited by a person can be determined with higher accuracy.

The luggage bag may have a mark for identifying the store, and the estimation may be performed by estimating the store that acquired the luggage bag based on the mark of the luggage bag.

With such a configuration, the store from which the luggage bag was acquired is estimated based on the mark for identifying the store that the luggage bag has, so the store visited by the person can be estimated more accurately. .

The determination includes first store list information indicating a list of stores estimated from the property owned by the person at a first time, and information owned by the person at a second time after the first time. A difference from the second store list information indicating a list of stores estimated from the owned property may be calculated, and the order of the stores visited by the person may be determined based on the difference.

With such a configuration, the first store list information indicating a list of stores estimated from the property owned by the person at the first time and the property owned by the person at the second time after the first time The difference from the second store list information indicating the list of stores estimated from the item is calculated, and the order of the stores visited by the person is determined based on the calculated difference, so the calculated difference is newly added. When indicating the store that has been added, it can be determined that the newly added store is the next store visited.

If there is a lost store that does not correspond to the store indicated by the second store list information among the stores indicated by the first store list information, the determination is made that the property acquired at the lost store is It may be determined that the property is integrated with any of the properties acquired at the store indicated by the second store list information.

With such a configuration, if there is a lost store that does not correspond to the store indicated by the second store list information among the stores indicated by the first store list information, the property acquired at the lost store is transferred to the second store list information. Since it is determined that it has been integrated into one of the possessions acquired at the store indicated by the store list information, the person puts the small possession into the large possession, and the store estimated from the small possession is the store list information. Even if it is not in the store in , the newly added store can be accurately determined as the next store visited.

The acquisition acquires a plurality of person appearance images including a plurality of persons, the identification determines whether or not the plurality of persons belong to the same group from the plurality of person appearance images, and determines whether or not the plurality of persons belong to the same group from the plurality of person appearance images. cutting out a group image including the plurality of persons determined as above, identifying the plurality of persons of the same group from the group image, Identifying an object, estimating a store that acquired the property from the identified property, and storing group information indicating the plurality of persons belonging to the same group and information owned by the plurality of persons belonging to the same group Store information indicating the store estimated from the owned property and time information indicating the time when the appearance images of the plurality of persons were obtained are associated with each other and stored in the storage unit, and the determination is stored in the storage unit. The order of the stores visited by the same group may be determined based on the time-series changes in the stores indicated by the store information.

With such a configuration, a plurality of person appearance images including a plurality of persons are acquired, whether or not the plurality of persons are in the same group is determined from the acquired plurality of person appearance images, and the plurality of persons determined to be in the same group is determined from the plurality of person appearance images. A group image containing a group of people is cut out, multiple people in the same group are identified from the cut out group image, properties owned by multiple people in the same group are identified from the cut out group image, and the identified property After estimating the store from which the property was acquired, group information indicating multiple people in the same group, store information indicating the store estimated from the properties owned by multiple people in the same group, and appearance images of multiple people are acquired. store in the storage unit in association with the time information indicating the time when the store information is stored in the storage unit, and determine the order of stores visited by the same group based on changes in the time series of stores indicated by the store information stored in the storage unit. there is

Therefore, even if multiple people from the same group visit multiple stores and the person holding the property acquired at each store changes within the group, the order of the stores visited by the same group can be determined with a simple configuration. can be judged with accuracy.

In addition, the present disclosure can be implemented not only as a people flow analysis method that executes the above-described characteristic processing, but also as a people flow analysis method having a characteristic configuration corresponding to the characteristic processing included in the people flow analysis method. It can also be implemented as a device, a people flow analysis system, and the like. Therefore, the following other aspects can also achieve the same effect as the above-described people flow analysis method.

A people flow analysis device according to another aspect of the present disclosure includes: an image information acquisition unit that acquires an appearance image of a person; a person identification unit that identifies the person from the appearance image; a store estimation unit that identifies an object and estimates a store that acquired the property from the identified property; person information that indicates the identified person; store information that indicates the estimated store; a storage unit that stores time information indicating the time at which the image was obtained in association with each other; and a store visit order determination unit that determines the order.

The property may include a luggage bag obtained at the store.

With such a configuration, since the luggage bag acquired at the store has a mark, color, shape, etc. that can identify the store, it is possible to accurately estimate the store that acquired the luggage bag from the luggage bag. Therefore, the order of stores visited by a person can be determined with higher accuracy.

A luggage bag image information acquisition unit that acquires an appearance image of the luggage bag may be further provided, and an imaging area of the appearance image of the luggage bag may be closer to the ground than an imaging area of the appearance image of the person.

With such a configuration, the image information acquisition unit acquires the appearance image of the person, the luggage bag image information acquisition unit acquires the appearance image of the luggage bag, and the imaging area of the appearance image of the luggage bag is the appearance image of the person. Since the area is closer to the ground than the imaging area, it is possible to identify the person and the luggage bag with high accuracy.

The apparatus further comprises a luggage bag image information acquisition unit that acquires an appearance image of the luggage bag, wherein the appearance image of the person is an image of the person photographed from above, and the appearance image of the luggage bag is a side view of the luggage bag. It may be an image taken from the other side.

With such a configuration, the image information acquisition unit acquires the appearance image of the person, the luggage bag image information acquisition unit acquires the appearance image of the luggage bag, and the appearance image of the person is an image of the person photographed from above, Since the exterior image of the luggage bag is an image of the luggage bag photographed from the side, it is possible to specify the person and the luggage bag with high accuracy.

The hand-held portion of the luggage bag may be smaller than the upper arm of a person.

With such a configuration, the hand-held portion of the luggage bag is smaller than the upper arm of a person, so that the luggage bag can be held by hand without hanging over the shoulder. Therefore, the distance of the mark of the luggage bag from the ground becomes constant, the imaging area of the exterior image of the luggage bag can be narrowed down, and the performance of recognition processing can be improved.

A sticking member to which information about the product purchased by the person is attached may be attached to the luggage bag.

With such a configuration, since the sticking member to which the information about the product purchased by the person is attached is attached to the luggage bag, the information about the purchased product can be given to the luggage bag.

The pasting member may have color information indicating the type and price of the product purchased by the person.

With such a configuration, since the attached member has color information indicating the type and price of the product purchased by the person, the type and price of the product purchased by the person can be specified based on the color information of the attached member. can be done.

A people flow analysis system according to another aspect of the present disclosure includes a photographing device for photographing an appearance image of a person, and a people flow analysis device, and the people flow analysis device captures the appearance image photographed by the photographing device. an image information acquisition unit for acquiring; a person identification unit for identifying the person from the appearance image; and a store that identifies property owned by the person from the appearance image and obtains the property from the identified property. a store estimation unit for estimating the person, person information indicating the identified person, store information indicating the estimated store, and time information indicating the time when the appearance image was acquired are stored in association with each other. and a store visit order determination unit that determines the order of stores visited by the person based on changes in the time series of stores indicated by the store information stored in the storage unit.

Moreover, it can also be realized as a computer program that causes a computer to execute the characteristic processing included in the people flow analysis method of the present disclosure. It goes without saying that such a computer program can be distributed via a computer-readable non-temporary recording medium such as a CD-ROM or a communication network such as the Internet.

It should be noted that each of the embodiments described below is a specific example of the present disclosure. Numerical values, shapes, components, steps, order of steps, and the like shown in the following embodiments are examples and are not intended to limit the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in independent claims representing the highest concept will be described as arbitrary constituent elements. Moreover, each content can also be combined in all the embodiments. In addition, a system may be configured in which some of the components of the people flow analysis system or the people flow analysis device according to the embodiment of the present disclosure and other components are distributed among a plurality of computers.

Embodiments of the present disclosure will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing an example of the configuration of a people flow analysis system according to Embodiment 1 of the present disclosure. The people flow analysis system shown in FIG. 1 includes a people flow analysis device 1, a plurality of cameras 21 to 2n, and a display 3.

The people flow analysis device 1 is composed of, for example, a cloud server or the like including a processor, memory, communication device, external storage device, etc., and communicates with the plurality of cameras 21 to 2n and the display 3 via a wired or wireless network 4. connected as possible.

The plurality of cameras 21 to 2n are composed of, for example, an omnidirectional camera capable of photographing a 360-degree range, a communication device, etc., and are installed at predetermined positions such as commercial facilities with a plurality of shops. The plurality of cameras 21 to 2n is an example of a photographing device for photographing an exterior image of a person visiting the commercial facility, and transmits the photographed exterior image to the people flow analysis device 1 via the network 4 . Note that the configuration of the cameras 21 to 2n is not particularly limited to the above example, and various image capturing devices can be used as long as they can capture an external image of a person.

The people flow analysis device 1 uses appearance information received from a plurality of cameras 21 to 2n to determine the order in which people move between stores, and transmits, via the network 4, store visit order information indicating the store visit order of people. etc. to the display 3 .

The display 3 is composed of, for example, a tablet or the like, and displays the store visit order information and the like received from the people flow analysis device 1 to the manager or manager of the commercial facility or each store. Note that the configuration of the display 3 is not particularly limited to the above example, and one or a plurality of communication terminals such as smartphones and personal computers may be used.

FIG. 2 is a schematic diagram showing an example of time-series changes in a person image. As shown in FIG. 2, for example, a person P1 is captured as a person image at time 13:10, a person P1 carrying luggage bag B1 is captured as a person image at time 14:15, and a person P1 is captured at time 14:38. A person P1 carrying luggage bags B1 and B2 is captured as a person image.

In this way, when shopping at a commercial facility such as a shopping mall, the person P1 is handed a luggage bag with the name of the store, the brand, etc. drawn on it. At time 14:38, the number of luggage bags B2 is increasing. Therefore, it can be seen that the person P1 visited the store where the luggage bag B1 was purchased between 13:10 and 14:15, and visited the store where the luggage bag B2 was purchased between 14:15 and 14:38. .

In the present embodiment, as will be described later, by using the luggage bags B1 and B2, stores visited by people are determined. The cameras 21 to 2n should be installed only at representative locations.

FIG. 3 is a diagram showing an arrangement example of the cameras 21 to 2n shown in FIG. 1, and shows an example using two cameras 21 and 22 out of the plurality of cameras 21 to 2n. In the example shown in FIG. 3, as a store layout on a floor with a commercial facility, for example, a plaza is set up in the center, passages are set up so as to surround the plaza, and six stores A to F are arranged in a substantially U-shape. are placed in

In this case, since it is necessary to install surveillance cameras such as the conventional flow line management system for each store, one camera, that is, seven surveillance cameras, must be installed for each store A to F and the plaza. did not become.

On the other hand, in the present embodiment, as shown in FIG. 3, the camera 21 is arranged at a position where it is possible to photograph people visiting or opening stores in the shops A to C and the plaza, and the cameras 22 are arranged in the shops D to F and the plaza. It is arranged at a position where a person who visits or opens a store can be photographed. The camera 21 captures images of people visiting or opening stores in stores A to C and the plaza, and transmits the captured exterior images to the people flow analysis device 1, and the camera 22 captures people visiting or opening stores in stores D to F and the plaza. A photograph is taken, and the photographed appearance image is transmitted to the people flow analysis device 1. - 特許庁Therefore, in the present embodiment, for example, the two cameras 21 and 22 photograph the exterior images of the persons, and the exterior images are used to accurately determine the order in which the persons visiting the stores A to F and the plaza are visited. can do.

As described above, the conventional flow line management system or the like requires seven surveillance cameras. By using the cameras 21 and 22 on the stand, the order of stores visited by a person can be determined with high accuracy with a simple configuration. Although the two cameras 21 and 22 are used in the above example, the present invention is not particularly limited to this example, and one or three or more cameras may be used.

FIG. 4 is a block diagram showing an example of the configuration of the people flow analysis device 1 shown in FIG. 1. As shown in FIG. As shown in FIG. 4, the people flow analysis device 1 includes an image information acquisition unit 11, a person identification unit 12, a store estimation unit 13, a time information acquisition unit 14, a database 15, a store visit order determination unit 16, and a communication unit 17. .

The communication unit 17 receives appearance information transmitted from the cameras 21 to 2n via the network 4 and outputs the information to the image information acquisition unit 11 . The image information acquisition unit 11 acquires the appearance image of the person from the communication unit 17, outputs the appearance image to the person identification unit 12 and the store estimation unit 13, and transmits time information indicating the time when the appearance image was acquired to the time information acquisition unit. 14.

The person identification unit 12 identifies a person from the appearance image acquired by the image information acquisition unit 11 and outputs person information indicating the identified person to the database 15 .

The store estimation unit 13 identifies a property owned by the person from the appearance image acquired by the image information acquisition unit 11, and estimates the store from which the person acquired the property from the specified property. In addition, when the property is a luggage bag acquired at a store, the store estimation unit 13 estimates the store that acquired the luggage bag based on the mark, color, shape, etc. that can identify the store possessed by the luggage bag. do. Further, if the luggage bag has a mark for identifying the store, the store estimation unit 13 estimates the store that acquired the luggage bag based on the mark of the luggage bag. The store estimation unit 13 outputs store information indicating the estimated store to the database 15 .

The time information acquisition section 14 outputs the time information acquired from the image information acquisition section 11 to the database 15 .

The database 15 is composed of a memory and/or an external storage device, etc., and stores the person information acquired from the person identification unit 12, the store information acquired from the store estimation unit 13, and the time information acquired from the time information acquisition unit 14. Store in association.

The store visit order determination unit 16 determines the order of stores visited by the person based on the time-series changes in the stores indicated by the store information stored in the database 15, and determines the determined store visit order for each person. The shown store order information is output to the communication unit 17 together with the person information.

Specifically, the store visit order determination unit 16 generates first store list information indicating a list of stores estimated from the property owned by the person at a first time and a second time after the first time. Then, based on the calculated difference, the order of the stores visited by the person is determined for each person.

The communication unit 17 transmits the visit order information and the person information to the display 3 via the network 4 . The display 3 displays the visit order for each person based on the received visit order information and person information, and notifies the manager or manager of the commercial facility or each store of the visit order for each person.

Next, the operation of the people flow analysis device 1 configured as described above will be described using a specific example. FIG. 5 is a schematic diagram for explaining an example of changes in property owned by a person. Note that one of the cameras 21 to 2n is referred to as camera 2 in the following description.

As shown in FIG. 5, for example, at time 13:10, a person P1 owns a luggage bag B1 as a property, and the camera 2 photographs the person P1 holding the luggage bag B1. At this time, the camera 2 selects the image file name FA246. png image data is sent to the people flow analysis device 1 .

Next, at time 14:15, the person P1 further owns the luggage bag B2 as a property, and the camera 2 photographs the person P1 holding the luggage bags B1 and B2. At this time, the camera 2 obtains an image file name FA247. png image data is sent to the people flow analysis device 1 .

Next, at time 14:38, the person P1 has a luggage bag B3 as a property, and the camera 2 photographs the person P1 who has the luggage bags B1, B2, and B3. At this time, the camera 2 captures an image file name FA248. png image data is sent to the people flow analysis device 1 .

FIG. 6 is a diagram showing an example of an image file table created by the image information acquiring section 11 shown in FIG. For example, when an exterior image as shown in FIG. For example, an image file table shown in FIG. 6 is created as a table.

In the image file table shown in FIG. 6, times and image file names are stored in a table format. For example, the image file name of the appearance image at time 13:10 is FA246. png. The image information acquisition unit 11 outputs the created image file table and appearance image to the person identification unit 12 and the shop estimation unit 13, and also outputs the image file table to the time information acquisition unit .

FIG. 7 is a diagram showing an example of a person identification table used by the person identification unit 12 shown in FIG. The person identification unit 12 pre-stores, for example, a person identification table shown in FIG. 7 as a static table that associates person IDs (identification information) with labeled person image file lists.

In the person identification table shown in FIG. 7, person IDs and labeled person image file lists are stored in advance in a table format. FA111. of the image file list. png, FA112. png, . . . are stored in advance.

The person identification unit 12 constructs a learning model based on the data in the person identification table, and uses the constructed learning model to estimate the person in the appearance image. In this case, the person identification unit 12 can comprehensively learn not only the face image of the person but also the person's appearance, clothes, body shape, etc., and can identify the person with high accuracy even if the number of cameras 21 to 2n is small. can be identified.

FIG. 8 is a diagram showing an example of a store estimation table used by the store estimation unit 13 shown in FIG. The store estimation unit 13 stores in advance, for example, a store estimation table shown in FIG. 8 as a static table in which store IDs (identification information) and labeled property image file lists are associated with each other.

In the store estimation table shown in FIG. 8, store IDs and labeled property image file lists are stored in advance in a table format. As an image, labeled property image file list B777. png, B778. png, . . . are stored in advance. The store estimating unit 13 builds a learning model based on the data of the store estimation table, identifies the property possessed by the person in the exterior image using the built learning model, and estimates the shop from the identified property. do.

Here, for example, when a luggage bag is used as the property, a mark for identifying the store such as a brand name, store name, logo mark, trademark, or characteristic design is formed on the surface of the luggage bag. there is Therefore, the store estimating unit 13 can learn the entire luggage bag including not only the form of the luggage bag but also the mark for identifying the store. The luggage bag can be specified with high accuracy, and the store can be estimated with high accuracy from the specified luggage bag. Note that the property is not particularly limited to the above example, and various property can be used. For example, other property such as a pamphlet distributed by a store may be used as long as the store can be identified.

FIG. 9 is a diagram showing an example of a shop table used by the visit order determination unit 16 shown in FIG. The store visit order determination unit 16 stores in advance, for example, a store table shown in FIG. 9 as a static table in which store IDs and store names are associated with each other. In the store table shown in FIG. 9, store IDs and store names are stored in advance in a table format. For example, store A is stored in advance as a store name whose store ID is SH234.

FIG. 10 is a diagram showing an example of a visit order estimation table stored in the database 15 shown in FIG. The time information acquisition unit 14 stores the time in the image file table shown in FIG. 6 in the time (an example of time information) in the visit order estimation table shown in FIG. The person identification unit 12 stores the person ID estimated using the appearance image of the image file table shown in FIG. 6 in the estimated person ID (an example of person information) of the visit order estimation table shown in FIG. The store estimating unit 13 stores the store ID estimated using the appearance image of the image file table shown in FIG. 6 in the estimated store ID list (an example of store information or store list information) of the visit order estimation table shown in FIG. .

The store visit order determination unit 16 obtains the difference between the current estimated store ID list and the previous estimated store ID list, and determines the stores that are present in the current estimated store ID list and not present in the previous estimated store ID list. The ID is determined to be the store ID of the store that the person has newly visited. The store visit order determination unit 16 refers to the store table shown in FIG. 9 and stores the store name corresponding to the store ID determined to be new as the new store name in the store visit order estimation table shown in FIG.

By the above operation, a store visit order estimation table is created in the database 15, in which the time, the estimated person ID, the estimated store ID list, and the new store name are associated. By reading the stored store names in order of time, the order of the stores visited by the person is determined for each person, and the store visit order information indicating the store visit order for each person is output to the communication unit 17 . For example, in the example shown in FIG. 10, it can be seen that the person whose person ID is P123 visited stores A, B, and C in this order. In the above example, png-format image data is used as the appearance image, but the present invention is not limited to this example, and image data in other formats such as gif format and jpg format may be used.

Next, the operation of the visit order determination unit 16 when there is a missing store that does not exist in the current estimated store ID list among the stores that exist in the estimated store ID list at the immediately preceding time will be described in detail.

FIG. 11 is a schematic diagram for explaining another example of changes in property owned by a person. As shown in FIG. 11, after an external image of a person P1 holding a small luggage bag B1 is captured, the person P1 does not carry a small luggage bag B1 and only carries a large luggage bag B2. Exterior images may be taken. This is because the person P1 may carry a small luggage bag B1 in a large luggage bag B2.

In this case, in this embodiment, when the luggage bag B1 that was being held until just before is not recognized, it is interpreted that the luggage bag B1 is combined into the luggage bag B2. , and the order of the stores visited by the person is determined based on the time-series changes in the stores indicated by the store information.

Specifically, the visit order determination unit 16 determines the current estimated store ID list (an example of the second store list information) among the stores indicated by the estimated store ID list (an example of the first store list information) at the immediately preceding time. ) indicates that there is no corresponding store, it is determined that the property acquired at the lost store has been merged with one of the properties acquired at the store indicated by the estimated store ID list at the previous time. do.

That is, the store visit order determination unit 16 puts the property of a small person into the property of a large person, and if the store estimated from the property of a small person is not in the store of the store list information, the store visit order determination unit 16 puts the property of the small person in any of the store list information. By merging into the large possession acquired at that store and excluding the store acquiring the small possession from the subsequent difference, the newly added store is determined to be the store visited next, and the person judges the order of the stores visited by each person.

FIG. 12 is a diagram showing another example of the store-visit order estimation table stored in the database 15 shown in FIG. The time information acquisition unit 14 stores the time in the image file table shown in FIG. 6 into the time in the visit order estimation table shown in FIG. The person identification unit 12 stores the person ID estimated using the appearance image of the image file table shown in FIG. 6 in the estimated person ID of the visit order estimation table shown in FIG. The store estimation unit 13 stores the store ID estimated using the appearance image of the image file table shown in FIG. 6 in the estimated store ID list of the visit order estimation table shown in FIG.

Here, the store visit order determination unit 16 obtains the difference between the current estimated store ID list and the estimated store ID list at the immediately preceding time, and determines whether there is a store ID (missing store) that is not included in the current estimated store ID list. For example, it is determined that the property corresponding to the shop ID has been merged with the property corresponding to the first shop ID in the presumed shop ID list at the immediately preceding time, and the destination shop ID and the source shop ID are associated with each other. create an integrated store table.

FIG. 13 is a diagram showing an example of an integrated shop table created by the visit order determination unit 16 shown in FIG. For example, as shown in FIG. 12, SH234 exists as a store ID in the estimated store ID list at the previous time 14:15, and SH234 (disappearing store) exists as the store ID in the estimated store ID list at the current time 14:38. If SH567 does not exist and SH567 exists as the head store ID, as shown in FIG. 13, the store-visit-order determining unit 16 stores SH567 in the integrated store ID in order to integrate SH234 of the lost store into the head SH567. Then, an integrated store table is created in which SH234 is stored in the integration source store ID.

Next, the store visit order determination unit 16 refers to the integrated store table, and the integration source store ID that does not exist in the current estimated store ID list exists in the current estimated store ID list and the estimated store ID list at the immediately preceding time. Judging that it has been integrated with the integration target store ID, it is excluded from the difference, and the store ID that exists in the current estimated store ID list and does not exist in the estimated store ID list at the previous time is the store of the store that the person has newly visited. ID, and referring to the store table shown in FIG. 9, the store name corresponding to the store ID determined to be new is stored in the newly visited store name shown in FIG.

In the example shown in FIG. 12, when the current time is 14:38, SH234 exists in the list of presumed shop IDs at 14:15 immediately before, but SH234 does not exist in the list of presumed shop IDs at present. SH567 is present in the estimated store ID list and the current estimated store ID list. In this case, the visit order determination unit 16 determines that the property acquired at the store with the store ID SH234 that existed at time 14:15 is owned by the store with the store ID SH567 that is the top of the current estimated store ID list. After judging that it has been integrated into the product, SH234 is excluded, the difference between the previous estimated store ID list and the current estimated store ID list is obtained, and it is determined that the difference SH789 is the store ID of the newly visited store. 12, and store C as the name of the newly visited store shown in FIG.

Therefore, even if a person puts a small property into a large property, and the store estimated from the small property is not in the stores in the store list information (estimated store ID list), the small property is included in the store list information (estimated store ID list). By excluding the store that acquired the small property from the subsequent difference, the newly added store will be visited next It is possible to accurately determine that the store is a store that has

Next, the people flow analysis processing by the people flow analysis system configured as described above will be described. FIG. 14 is a sequence diagram showing an example of people flow analysis processing by the people flow analysis system shown in FIG.

First, the plurality of cameras 21 to 2n capture an appearance image of a person visiting the commercial facility, and execute an image transmission process of transmitting the captured appearance image to the people flow analysis device 1 (step S11). Here, the plurality of cameras 21 to 2n may transmit a camera ID (identification information) for identifying themselves together with the appearance image. In this case, in person identification processing, store estimation processing, visit order determination processing, etc., which will be described later, the position of the camera that captured the exterior image is specified using the camera ID, and the accuracy of each process is determined using the specified camera position. can be enhanced.

Next, the communication unit 17 of the people flow analysis device 1 receives the appearance information transmitted from the plurality of cameras 21 to 2n and outputs it to the image information acquisition unit 11, and the image information acquisition unit 11 obtains the appearance image of the person. Image information acquisition processing is executed to acquire and output the appearance image to the person identification unit 12 and the store estimation unit 13, and to output time information indicating the time when the appearance image was acquired to the time information acquisition unit 14 (step S12).

Next, the time information acquisition unit 14 executes time information acquisition processing for storing the time information acquired from the image information acquisition unit 11 in the visit order estimation table of the database 15 (step S13).

Next, the person identification unit 12 identifies a person from the appearance image acquired by the image information acquisition unit 11, and stores person information (estimated person ID) indicating the identified person in the visit order estimation table of the database 15. Processing is executed (step S14).

Next, the store estimation unit 13 identifies property owned by the person from the appearance image acquired by the image information acquisition unit 11, estimates the store from which the person acquired the property from the specified property, and indicates the estimated store. Store estimation processing for storing the store information (estimated store ID list) in the store visit order estimation table of the database 15 is executed (step S15).

Next, the store visit order determination unit 16 determines the order of stores visited by each person from the store visit order estimation table of the database 15 based on changes in the time series of stores, and determines the order in which the person newly visits the store. The store name corresponding to the store ID is stored in the store visit order estimation table of the database 15, and the store visit order information indicating the store visit order for each person is output to the communication unit 17 together with the person information. A visit order determination process for transmitting information to the display 3 is executed (step S16).

Next, the display 3 executes a visit order reception process for receiving the visit order information and the person information (step S17).

Finally, the display 3 executes display processing for displaying the order of coming to the store for each person based on the received order information of coming to the store (step S18).

By the above-described processing, in the present embodiment, an appearance image of a person is acquired, the person is identified from the acquired appearance image, the property owned by the person is specified from the acquired appearance image, and the specified property is identified. Person information (estimated person ID) indicating the identified person, store information (estimated store ID list) indicating the estimated store, and time indicating the time when the appearance image was acquired The order of the stores visited by the person is determined based on the time-series changes in the stores indicated by the store information stored in the database 15 .

Therefore, in the present embodiment, without arranging the cameras 21 to 2n for capturing exterior images for each store, the property is specified from the exterior image of the person, and the store that acquired the property from the specified property is accurately identified. , the newly added store can be determined as the store visited next, and the order of the stores visited by the person can be determined with high precision with a simple configuration.

Note that, in the present embodiment, properties owned by a person are specified from the acquired appearance image, and all the shops that acquired the properties are estimated from all the specified properties. Here, a person may visit a commercial facility carrying a property with a store name drawn thereon, such as a personal bag. In this case, even though the person has not visited the store, it looks as if the person has visited the store on the exterior image. Therefore, a camera is also installed near the entrance of a commercial facility, and from the exterior image acquired by the camera installed near the entrance, belongings that are identified as belonging to a person (for example, my bag) are excluded from the estimation of the order of coming to the store. You may make it exclude.

(Embodiment 2)
In the first embodiment described above, the order of coming to the store is determined for each person, but the customers may not only shop alone, but may also come in groups of parents, children, friends, or the like. In this case, the person holding the property, such as the luggage bag, may change among the members of the group. For this reason, in the present embodiment, groups of visitors are identified, properties owned by each group are specified, stores visited are estimated for each group based on the specified properties, and the order of visits is determined for each group. do.

FIG. 15 is a block diagram showing an example of a configuration of a people flow analysis device of the people flow analysis system according to Embodiment 2 of the present disclosure. The configuration of the people flow analysis system of this embodiment is the same as the people flow analysis system shown in FIG. 1 except that the people flow analysis device 1 shown in FIG. 1 is changed to the people flow analysis device 1a shown in FIG. Therefore, the illustration of the people flow analysis system is omitted, and the points different from the people flow analysis system shown in FIG. 1 will be described in detail below using the reference numerals in FIG. 1 as appropriate.

The people flow analysis device 1a shown in FIG. It is communicably connected to the display 3 . The people flow analysis device 1a includes an image information acquisition unit 11, a store estimation unit 13, a time information acquisition unit 14, a database 15, a visit order determination unit 16, a communication unit 17, a group determination unit 18, a group image cutout unit 19, and a person group. An identification unit 20 is provided.

A plurality of cameras 21 to 2n capture multiple person appearance images including multiple persons visiting the commercial facility, and transmit the captured multiple person appearance images to the people flow analysis device 1a via the network 4. FIG.

The communication unit 17 receives multiple person appearance information transmitted from the cameras 21 to 2n via the network 4 and outputs the information to the image information acquisition unit 11 . The image information acquiring unit 11 acquires the appearance images of the plurality of persons from the communication unit 17, outputs the appearance images of the plurality of persons to the group determination unit 18, and transmits the time information indicating the time when the appearance images of the plurality of persons were obtained to the time information obtaining unit 14. output to

The group judgment unit 18 judges whether or not a plurality of persons belong to the same group from the appearance images of the plurality of persons acquired by the image information acquisition unit 11, and extracts the appearance images of the persons judged to belong to the same group. and output to the group image clipping unit 19 . Note that when a plurality of persons are not in the same group, each person may be identified as in the first embodiment.

The group image cutout unit 19 cuts out a group image including a plurality of persons judged to belong to the same group from the appearance image of the plurality of persons judged to belong to the same group, and outputs the group image to the person group identification unit 20 and the shop estimation unit 13. Output.

The person group identification unit 20 identifies a plurality of persons in the same group from the group image cut out by the group image cutout unit 19 and outputs group information indicating the identified plurality of persons in the same group to the database 15 .

The store estimation unit 13 identifies properties owned by a plurality of persons belonging to the same group from the group image cut out by the group image cutout unit 19, estimates the store from which the property was acquired from the identified properties, and determines the estimated store. to the database 15.

The time information acquisition section 14 outputs the time information acquired from the image information acquisition section 11 to the database 15 .

The database 15 stores the group information acquired from the person group identification unit 20, the store information acquired from the store estimation unit 13, and the time information acquired from the time information acquisition unit 14 in association with each other.

A store visit order determination unit 16 determines the order of stores visited by a plurality of persons in the same group for each group based on changes in the time series of stores indicated by the store information stored in the database 15, and determines the determined group. It outputs to the communication unit 17 together with the group information information indicating the order of coming to the store.

The communication unit 17 transmits the visit order information and the group information to the display 3 via the network 4 . The display 3 displays the order of visits for each group from the received order information of visits, and notifies the order of visits for each group to the manager or manager of the commercial facility or each store.

Next, the operation of the people flow analysis device 1a configured as described above will be described using a specific example. The group determination unit 18 uses a known group determination method to determine an image file ( It determines whether or not a plurality of persons belong to the same group from the appearance images of the plurality of persons), extracts appearance images of the plurality of persons determined to belong to the same group, and outputs the images to the group image clipping section 19 . A group image clipping unit 19 clips a group image including a plurality of persons determined to belong to the same group from a plurality of persons appearance images determined to belong to the same group, and outputs the group image to a person group identification unit 20 . The person group identification unit 20 uses a known group determination method to estimate a person ID from a group image, and creates a person group table in which a plurality of persons belonging to the same group are summarized.

FIG. 16 is a diagram showing an example of a person group table created by the person group identification unit 20 shown in FIG. For example, as shown in FIG. 16, a person group table is created in which a person group ID (identification information) for identifying a group is associated with a person ID list indicating person IDs of a plurality of persons included in the group. In the example shown in FIG. 16, it can be seen that the group with the person group ID of G111 includes three persons with person IDs of P123, P124, and P125.

Here, as the above group discrimination method, for example, the method described in Japanese Patent Application Laid-Open No. 2014-229068 (obtaining the flow line of the person by estimating the coordinates of the person from the image, and performing the action of drawing the same flow line) A method of estimating that a plurality of people are acting as a group) or a method described in Japanese Patent Application Laid-Open No. 2013-50945 (obtaining the body posture direction and line-of-sight direction of a person from a camera image, and facing each other) or a method of regarding a plurality of persons facing the same direction as a group).

The above-described group determination method is an example, and is not particularly limited to this example. is detected, use a method to determine that it is a group, use a method that uses location information of mobile terminals and other wireless terminals in addition to images, acquire voices in addition to images, and detect conversations A method of judging a group of persons who have been identified as a group may be used.

FIG. 17 is a diagram showing an example of a store visit order estimation table stored in the database 15 shown in FIG. The time information acquisition unit 14 stores the time in the image file table that associates the time when the appearance images of multiple persons are acquired with the image file name in the time in the visit order estimation table shown in FIG. 17 (an example of time information). The person group identification unit 20 identifies the person group ID in the person group table shown in FIG. Stored in a group ID (an example of group information). The store estimation unit 13 stores the store IDs estimated using the multiple person appearance images of the image file table in the estimated store ID list (an example of store information or store list information) of the visit order estimation table shown in FIG. .

The store visit order determination unit 16 obtains the difference between the current estimated store ID list and the previous estimated store ID list, and determines the stores that are present in the current estimated store ID list and not present in the previous estimated store ID list. The ID is determined to be the store ID of the store that the person has newly visited. The store visit order determination unit 16 refers to the store table shown in FIG. 9 and stores the store name corresponding to the store ID determined to be new as the new store name in the store visit order estimation table shown in FIG.

By the above operation, a store visit order estimation table is created in the database 15 in which the time, the estimated person group ID, the estimated store ID list, and the new store name are associated. By reading out the store names stored in the order of time, the order of stores visited by a plurality of persons in the same group is determined for each group, and visiting order information indicating the visiting order for each group is output to a communication section 17. - 特許庁For example, in the example shown in FIG. 17, it can be seen that the group whose person group ID is G111 visited stores A, B, and C in this order.

Next, the people flow analysis processing by the people flow analysis system configured as described above will be described. FIG. 18 is a sequence diagram showing an example of people flow analysis processing by the people flow analysis system according to Embodiment 2 of the present disclosure.

First, the plurality of cameras 21 to 2n capture multiple-person appearance images including multiple people who have visited the commercial facility, and execute image transmission processing for transmitting the captured multiple-person appearance images to the people flow analysis device 1a ( step S11).

Next, the communication unit 17 of the people flow analysis device 1a receives the multiple person appearance information transmitted from the multiple cameras 21 to 2n and outputs it to the image information acquisition unit 11, and the image information acquisition unit 11 receives the multiple person appearance information. , outputting the multiple-person appearance image to the group determination unit 18, and outputting time information indicating the time when the multiple-person appearance image was acquired to the time information acquisition unit 14. (step S12).

Next, the time information acquisition unit 14 executes time information acquisition processing for storing the time information acquired from the image information acquisition unit 11 in the visit order estimation table of the database 15 (step S13).

Next, the group determination unit 18 determines whether or not the plurality of persons belong to the same group from the plurality of persons appearance images acquired by the image information acquisition unit 11, and determines whether or not the plurality of persons belong to the same group. are extracted and output to the group image clipping unit 19 (step S21).

Next, the group image cut-out unit 19 cuts out a group image including a plurality of persons judged to belong to the same group from the appearance image of the plurality of persons judged to belong to the same group. A group image clipping process to be output to the unit 13 is executed (step S22).

Next, the person group identification unit 20 identifies a plurality of persons belonging to the same group from the group image extracted by the group image extraction unit 19, and generates group information (estimated person group ID) indicating the identified plurality of persons belonging to the same group. A person group identification process stored in the store visit order estimation table of the database 15 is executed (step S23).

Next, the store estimation unit 13 identifies properties owned by a plurality of persons belonging to the same group from the group image cut out by the group image cutout unit 19, estimates the store from which the properties were acquired from the identified properties, A store estimation process is executed to store the store information (estimated store ID list) indicating the estimated stores in the visit order estimation table of the database 15 (step S15).

Next, the store visit order determination unit 16 determines, for each group, the order of stores visited by a plurality of people in the same group based on changes in store time series from the store visit order estimation table in the database 15 . stores store names corresponding to store IDs where a plurality of people in the group have newly visited the store in a store visit order estimation table of the database 15, and outputs store visit order information indicating the store visit order for each group to the communication unit 17 together with the group information; The communication unit 17 executes a visit order determination process for transmitting the visit order information and the group information to the display 3 (step S16).

Next, the display 3 executes a visit order reception process for receiving the visit order information and the group information (step S17).

Finally, the display 3 executes display processing for displaying the order of coming to the store for each group based on the received order information of coming to the store (step S18).

By the above-described processing, in the present embodiment, a multi-person appearance image including a plurality of persons is acquired, it is determined from the acquired multi-person appearance image whether or not the plurality of persons belong to the same group, and from the multi-person appearance image, it is determined whether or not the plurality of persons belong to the same group. A group image containing a plurality of persons determined to be a group is cut out, a plurality of persons belonging to the same group are identified from the cut out group image, and properties owned by a plurality of persons belonging to the same group are identified from the cut out group image. , the store that acquired the property from the specified property is estimated, and group information (estimated person group ID) indicating multiple people in the same group and the store estimated from the property owned by multiple people in the same group are obtained. The store information (estimated store ID list) shown and the time information indicating the time when the appearance images of the plurality of persons were obtained are associated with each other and stored in the database 15, and the stores indicated by the store information stored in the database 15 are displayed in chronological order. Based on the changes, the order of stores visited by the same group is determined.

Therefore, in the present embodiment, even if a plurality of persons belonging to the same group visits a plurality of shops and the person holding the property acquired at each shop changes within the group, the order of the shops visited by the same group is determined. can be determined with high precision with a simple configuration.

(Modification)
In each of the above-described embodiments, an example in which the same cameras 21 to 2n are used as a camera for photographing a person and a camera for photographing a property has been described. It is not particularly limited to the example, and various modifications are possible as described below.

FIG. 19 is a schematic diagram showing a first arrangement example when using a camera for photographing a person and a camera for photographing a property. In order to recognize a person, it is necessary to obtain a high-resolution image near the face, but in order to recognize an object such as a luggage bag, an image distant from the face is required. Therefore, in the first arrangement example shown in FIG. 19, instead of camera 21, cameras 21a and 21b are used (cameras 22 to 2n are the same), and camera 21a captures an external image of person P1, The camera 21b captures an external image of the property, ie, the baggage bag B1. In this case, the image information acquisition section 11 shown in FIG. 4 is composed of an image information acquisition section that acquires an appearance image of a person from the camera 21a, and a baggage bag image information acquisition section that acquires an appearance image of the luggage bag from the camera 21b. be.

When the person P1 is moving in the direction of the flow of people, that is, in the traveling direction D1, the camera 21a is mounted on the post SP so that the optical axis O1 of the camera 21a, that is, the photographing direction intersects the traveling direction D1 of the person P1 from the front. Installed, the camera 21a photographs the face and the like of the person P1. The camera 22a is installed on the post SP so that the optical axis O2 of the camera 22a, that is, the photographing direction, is perpendicular to the traveling direction D1 of the person P1 from the side, and the camera 22a photographs the baggage bag B1. Therefore, the image capturing area of the appearance image of the luggage bag (the broken line area in the drawing) is closer to the ground than the image capturing area of the person's appearance image (the broken line area in the drawing).

Thus, by installing the camera 21a for photographing the whole body of the person P1 and the camera 21b for photographing the image of the luggage bag B1, the person P1 and the luggage bag B1 can be specified with high accuracy. be able to. In addition, by using the part of the clothing imaged in common by the imaging area of the camera 21a and the imaging area of the camera 21b, the person P1 and the luggage bag B1 can be collated, and the baggage bag B1 is carried by the person P1. It is possible to specify the luggage bag B1 with high accuracy.

FIG. 20 is a side view of a person carrying a luggage bag, and FIG. 21 is a front view of a person carrying a luggage bag. As shown in FIGS. 20 and 21, when the person P1 moves with the luggage bag B1 in hand, the large surface of the luggage bag B1 is often perpendicular to the traveling direction of the person P1. Also, the large mark T1 (for example, "ABC") formed on the large surface of the luggage bag B1 shown in FIG. 20 is often perpendicular to the traveling direction of the person P1. On the other hand, the small mark T2 (for example, "ABC") formed on the small side surface of the luggage bag B1 shown in FIG. Therefore, in order to set the camera for photographing the facial image of the person at a position where the facial image can be easily photographed, and to photograph the large surface of the luggage bag B1 and the large mark T1, the traveling direction of the person P1 and substantially Image information of luggage bags can be obtained more easily by using cameras whose optical axes are set in orthogonal directions.

As described above, the camera for photographing the facial image of the person is installed at a position where the facial image can be easily photographed, and the camera for photographing the luggage bag is arranged such that the optical axis is oriented in a direction substantially perpendicular to the traveling direction of the person. An example using a camera in which is set will be described. FIG. 22 is a schematic diagram showing a second arrangement example when using a camera for photographing a person and a camera for photographing a property.

In the second arrangement example shown in FIG. 22, a camera 21a and a camera 21b are used instead of the camera 21. The camera 21a is attached to the ceiling CE, for example, and photographs the person P1 from above, and the camera 21b Attached to the post SP, the luggage bag B1 is photographed from the side.

That is, when the person P1 is moving in the direction of the flow of people, that is, in the traveling direction D1, the camera 21a is moved so that the optical axis O1 of the camera 21a, that is, the photographing direction, is perpendicular to the traveling direction D1 of the person P1 from above. Installed on the ceiling CE. Further, the camera 22a is installed on the post SP so that the optical axis O2 of the camera 22a, that is, the photographing direction, is perpendicular to the traveling direction D1 of the person P1 from the side.

By installing the camera 21a so that the angle formed by the optical axis O1 of the camera 21a and the optical axis O2 of the camera 21b is nearly orthogonal, the appearance image of the person P1 can be changed from the image of the person P1 photographed from above. Thus, the exterior image of the luggage bag B1 is an image of the luggage bag B1 photographed from the side, and the mark on the luggage bag B1 (for example, the large mark T1 in FIG. 20) can be recognized with higher accuracy. When the camera 21a for photographing the face image simultaneously photographs the image of the mark on the luggage bag B1, it is preferable to provide the mark T2 on the side surface of the luggage bag B1 as shown in FIG. In this case, it becomes easy to recognize the mark from the appearance image taken by the camera 21a for recognizing the face image.

In general, luggage bags may have a long string-like portion to be held, that is, a hand-held portion. 23A and 23B are schematic diagrams showing a first example of the handheld part of the luggage bag and how to hold the luggage bag in this example, and FIG. FIG. 25 is a schematic diagram showing a third example of the handheld part of the luggage bag and a schematic diagram showing how to hold the luggage bag in this example.

As shown in FIG. 23, when the hand-held portion F1 of the luggage bag B1 is larger than the upper arm of a person, the luggage bag B1 can be held by hand as in the case of the person P1a or by the person P1b. , may be lowered from the shoulder through the arm. In this case, the distance from the ground to the position of the mark T1 on the luggage bag B1 is greatly different, and it is necessary to widen the imaging area of the camera that acquires the image of the luggage bag B1. In addition, when the bag is hung from the shoulder, the mark T1 on the luggage bag B1 may be hidden by the hand, and the recognition performance may be degraded.

On the other hand, as shown in FIG. 24, if the hand-held portion F2 of the luggage bag B2 is smaller than the upper arm of a person, or as shown in FIG. If the cut-out portion F3 functioning as a hand-held portion is smaller than the upper arm of a person, if the luggage bags B2 and B3 are of the same size, the marks T2 and T3 of the luggage bags B2 and B3 are positioned from the ground. The distance is within a certain range.

Therefore, the handle portion of the luggage bag is preferably smaller than the upper arm of a person. In this case, the hand-holding portion of the luggage bag becomes smaller than the upper arm of a person, and the luggage bag can be held by the hand without being put on the shoulder, so the distance of the logo, etc. of the luggage bag from the ground is constant. . As a result, it is possible to narrow down the imaging area of the camera that acquires the image of the baggage bag, and to improve the performance of recognition processing.

In addition, when purchasing a product, an attachment member such as a seal may be attached to the luggage bag so that the opening of the luggage bag is not left open. FIG. 26 is a diagram showing a first example of the sticking member to be stuck on the luggage bag, and FIG. 27 is a diagram showing a second example of the sticking member to be stuck on the luggage bag.

As shown in FIG. 26, the sticking member S1 is stuck to the side upper part of the luggage bag B1 having the mark T1, and as shown in FIG. It is pasted on the upper center of B1. In this way, the sticking members S1 and S2 are often attached to the opening of the luggage bag B1 so that the opening of the luggage bag B1 does not open. In this case, the sticking members S1 and S2 such as stickers can have information about the purchased product.

Therefore, it is preferable that the attachment member attached with information about the product purchased by the person is attached to the luggage bag. For example, a store clerk attaches a sticking member provided with information about the purchased product to a luggage bag, and attaches a different sticking member for each purchased product. As a result, in a large shopping center or the like, information on the type and price of a product purchased at a certain store can be estimated from the attached member, and when entering another store, the product recommended by the store clerk can be found based on the information on the attached member. , the customer will be able to select according to the products they have already purchased.

For example, if the purchased item is an expensive item, there is a high possibility that the customer will purchase an expensive item even at the new store, and the store clerk can recommend such an item. In addition, in a large shopping mall, a plurality of stores cooperate to make it possible for customers to understand the information of the purchased product from the appearance, thereby promoting the purchase of the product.

Also, the pasting member preferably has color information indicating the type and price of the product purchased by the person. FIG. 28 is a diagram showing an example of a first color information table showing the correspondence relationship between the product type and the color of the attachment member, and FIG. 29 is a diagram showing the correspondence relationship between the price of the product and the color density of the attachment member. It is a figure which shows an example of the 2nd color information table which shows.

As shown in FIG. 28, the database 15 stores in advance a first color information table in which the types of products and the colors of the attachment members are associated with each other. For example, the product type "outerwear" and the color of the attachment member "red" are stored in association with each other. Further, as shown in FIG. 29, the database 15 stores in advance a second color information table in which the price of the product and the density of the color of the sticking member are associated with each other. For example, the product price "1 yen to 999 yen" and the color density "10%" of the adhesive member are stored in association with each other. For example, if the type of product purchased is "outerwear" and the price is 800 yen, a red sticking member with a concentration of 10% will be stuck to the luggage bag.

As described above, by associating the type and price information of the product with the color and density of the pasting member, it becomes possible to know the information of the purchased product from the appearance. On the other hand, since there is also a problem of privacy of the purchaser, the correspondence between the first and second color information tables in FIGS. 28 and 29 may be changed depending on the day of the week, for example. In this case, it is possible to share information about the product purchased only by the store clerk.

FIG. 30 is a diagram showing an example of an exterior image captured by the camera 21b that captures an exterior image of the baggage bag shown in FIG. For example, a first person P1 carries a luggage bag B1 having a mark T1 and an attachment member S1, a second person P2 carries a luggage bag B2 having a mark T2 and an attachment member S2, and a third person P1 carries a luggage bag B2 having a mark T2 and an attachment member S2. When P3 carries luggage bag B3 having mark T3 and sticking member S3, camera 21b captures an exterior image shown in FIG. Here, the colors and densities of the adhesive members S1 to S3 differ depending on the types and prices of the purchased products, and the types and prices of the commodities can be specified by the colors and densities of the adhesive members S1 to S3. Therefore, the people flow analyzer 1 acquires the appearance image of the luggage bag photographed by the camera 21b, obtains color information of the colors and densities of the attachment members S1 to S3, specifies the type and price of the product, and determines the purchase price. It is possible to associate the person with the information of the destination.

Information to be added to the pasting member is not particularly limited to the above example, and other information may be added. FIG. 31 is a diagram showing an example of a sticking member provided with a barcode. For example, as shown in FIG. 31, the bar code BC may be attached to the attaching member S1 of the baggage bag B1. In this case, it becomes possible to give more detailed information as the information of the barcode BC.

The people flow analysis method, people flow analysis device, and people flow analysis system according to the present disclosure can accurately determine the order of stores visited by people with a simple configuration. It is useful for a people flow analysis device and a people flow analysis system equipped with the people flow analysis device.

Reference Signs List 1, 1a people flow analyzer 2, 21 to 2n multiple cameras 3 display 11 image information acquisition unit 12 person identification unit 13 store estimation unit 14 time information acquisition unit 15 database 16 visit order determination unit 17 communication unit 18 group determination unit 19 group image clipping unit 20 person group identification unit

Claims (8)

an image information acquisition unit that acquires an appearance image of a person;
a person identification unit that identifies the person from the appearance image of the person;
(1) Identify , from the appearance image of the person, a luggage bag attached with a sticking member to which product information indicating the type or price of the product purchased by the person at a store is attached, among the belongings owned by the person. (2) extracting feature information indicating the features of the attached member of the identified luggage bag; a product information identifying unit that identifies the product information for a product;
a storage unit that stores personal information indicating the identified person and the product information in association with each other;
Purchasing product estimation device. further comprising a luggage bag image information acquisition unit that acquires an exterior image of the luggage bag,
The imaging area of the appearance image of the luggage bag is a portion closer to the ground than the imaging area of the appearance image of the person,
The purchased product estimation device according to claim 1. The luggage bag possessed by the person in the appearance image is identified by using a learning model constructed based on the data of the store estimation table in which the store ID and the property image are stored in association with each other. further comprising a store estimating unit for estimating a store that acquired the luggage bag from the luggage bag obtained from the
The product information includes store information indicating the estimated store,
The purchased product estimation device according to claim 1. further comprising a luggage bag image information acquisition unit that acquires an exterior image of the luggage bag,
further comprising a property identification unit that compares the person and the luggage bag by using an appearance image of the person's clothing captured in common with the appearance image of the person and the appearance image of the luggage bag,
The purchased product estimation device according to claim 1. further comprising a luggage bag image information acquisition unit that acquires an exterior image of the luggage bag,
The appearance image of the person is an image of the person photographed from above,
The exterior image of the luggage bag is an image of the luggage bag taken from the side,
The purchased product estimation device according to claim 1. the hand-held portion of the luggage bag is smaller than a human upper arm;
The purchased product estimation device according to claim 1. The pasting member has color information indicating the type and price of the product purchased by the person,
The purchased product estimation device according to claim 1. The pasting member has, as the product information, bar code information indicating the type and price of the product purchased by the person.
The purchased product estimation device according to claim 1.

Images