JP2019145022A - Store information providing system, server, store information providing method, and program - Google Patents

Abstract

To provide a store information providing system capable of providing information on whether a situation in a store suits a customer's request and preference.SOLUTION: A store information providing system includes a plurality of sensors provided in a store, a server, and a display device. The plurality of sensors acquire detection data by sensing, and transmit the detection data. The server receives the detection data, recognizes the positions of at least ones between persons and objects located in the store on the basis of the detection data, calculates the degree of congestion of the store on the basis of the positions of at least ones between the persons and the objects, and determines a user use for using a user store on the basis of the degree of congestion and the detection data.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to a store information providing system, a server, a store information providing method, and a program for displaying or outputting store information.

  Conventionally, as a system for notifying vacant seat information in stores such as restaurants, information on vacant seats is displayed when the number of vacant seats in store facilities is large, and advertisement content information on stores is distributed when the number of vacant seats in store facilities is small A digital signage system is known (see Patent Document 1).

Japanese Patent No. 6153177

  In the system of Patent Document 1, vacant seat information can be obtained, but the situation and atmosphere in the store are not taken into consideration. Therefore, there is a possibility that the in-store situation of this store does not meet the demands and preferences of the confirmer who has confirmed the vacant seat situation. That is, it is difficult to provide information as to whether the in-store situation matches the customer's request or preference.

  The present disclosure has been made in view of the above circumstances, and provides a store information providing system, a server, a store information providing method, and a program that can provide information as to whether the in-store situation matches a customer's request or preference.

  One aspect of the present disclosure is a store information providing system including a plurality of sensors provided in a store and a server, wherein the plurality of sensors acquire detection data by sensing and transmit the detection data. The server receives the detection data, recognizes the position of at least one of a person and an object located in the store based on the detection data, and based on the position of at least one of the person and the object, It is a store information providing system that calculates a congestion degree of a store and determines a user use for the user to use the store based on the congestion degree and the detection data.

  One aspect of the present disclosure is a server that provides information about a store, and includes a reception unit, a processing unit, and an output unit, and the reception unit is detected by a plurality of sensors provided in the store. And the processing unit recognizes the position of at least one of a person and an object located in the store based on the detection data, and based on the position of at least one of the person and the object, It is a server which calculates the congestion degree of a store and determines a user application for a user to use the store based on the congestion level and the detection data.

  One aspect of the present disclosure is a store information providing method in a server that provides information about a store, the detection data detected by a plurality of sensors provided in the store is acquired, and based on the detection data, Recognizing the position of at least one of a person and an object located in the store, calculating the congestion level of the store based on the position of at least one of the person and the object, and based on the congestion level and the detection data, It is a store information provision method in which a user determines a user application for using the store.

  One aspect of the present disclosure is a program for causing a computer to execute the store information providing method.

  According to the present disclosure, it is possible to provide information as to whether the in-store situation matches a customer's request or preference.

The figure which shows an example of a structure of the shop information provision system in 1st Embodiment. Table showing judgment sheets registered in DB server Flow chart showing vacant seat status display processing procedure by analysis server The flowchart which shows the acquisition procedure of the store layout in S3 Diagram showing an example of store layout The figure which shows the registration contents of the table information table The figure which shows the registration contents of the chair information table The flowchart which shows the seating condition processing procedure of S4 Figure showing an example of the seating situation in the store Table showing registration contents of person information table Table showing the registered contents of the object information table A table showing a record of the seating status of chairs in chronological order A table showing a record of the seating status of a table in chronological order The flowchart which shows the recognition processing procedure of the person or thing in S42 It is a table which shows attribute information of a person. Table showing category judgment by appearance Table showing object estimation Table showing sound attributes The flowchart which shows the update processing procedure of the display content in S5 A table showing a record of the seating status of a table in chronological order A table showing time records of people's stay Table showing registration contents of store status table The figure which shows the display screen of the seat availability which is displayed with digital signage The sequence diagram which shows the procedure which distributes the present congestion information of each store in 2nd Embodiment to a smart phone. Table showing availability at each store The figure which shows the screen which displays the present congestion degree of each store displayed on the smart phone The sequence diagram which shows the procedure which delivers the future congestion information of each store in 3rd Embodiment to a smart phone. The flowchart which shows the congestion degree prediction procedure of the analysis server in procedure T18 The figure which shows the screen showing the present congestion degree of each store in the vicinity of I point which is a departure point displayed on a smart phone The figure which shows the screen which shows the congestion degree of each future store in the vicinity of J point which is a destination displayed on a smart phone

  Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the claimed subject matter.

(First embodiment)
FIG. 1 is a diagram illustrating an example of a configuration of a store information providing system 5 according to the first embodiment. The store information providing system 5 has a configuration including a server 15 installed in the center 9, a plurality of electronic devices 30 installed in the store 35, and an information device 40 possessed by the user. The store information providing system 5 may be introduced into stores such as restaurants and fast food stores, for example.

  The server 15 includes a Web server 11, a database (DB) server 12, and an analysis server 13. The Web server 11, the database (DB) server 12, and the analysis server 13 may exist as a plurality of functions in one server 15, or may exist as independent physical servers 15. FIG. 1 mainly illustrates that the server 15 has the functions of a Web server 11, a database (DB) server 12, and an analysis server 13.

  The server 15 (for example, the Web server 11, the DB server 12, and the analysis server 13) is disposed in the center 9, is connected to the center network, and can perform data communication with each other. Communication in the center 9 is managed by the center network device 10. In addition, the server 15 may be arrange | positioned in the center 9, may be arrange | positioned on a cloud, and may be arrange | positioned in one of the one or more stores 35.

  The plurality of electronic devices 30 are arranged in each store and connected to each store network. Communication in the store is managed by the store network device 20. The plurality of information devices 40 are connected to the Internet 8.

  The center network device 10 includes, for example, a gateway device and a router device, and is connected to one or more store networks and connected to the Internet 8. The store network device 20 includes a gateway device and a router, for example, and connects a plurality of electronic devices 30 in the store 35 to the center network device 10. For example, in addition to the center network device 10 and the plurality of information devices 40, the geographic information service server 7 may be connected to the Internet 8.

  The Web server 11 includes a communication unit 111, and the Web server 11 transmits information stored in the DB server 12 and information analyzed by the analysis server 13 to other devices. The Web server 11 may include a processing unit that performs various processes and a memory that stores various data. The communication unit 111 is an example of a receiving unit and an output unit.

  The DB server 12 includes a memory 121 and a storage 122, and accumulates various data, information on various tables, and the like. The DB server 12 may include a processing unit that performs various processes and a communication unit that communicates various data.

  The analysis server 13 includes a processing unit 131 and performs various processes (for example, acquisition, analysis, calculation, determination, and determination of various data). The analysis server 13 may include a memory that holds various data and a communication unit that communicates the various data.

  The plurality of electronic devices 30 includes a camera 31, a microphone 32, a digital signage 33, and a camera 34. The camera 31 and the microphone 32 are examples of sensors that detect various data. The sensor may include a sensor other than the camera 31 and the microphone 32 (for example, an infrared sensor or a human sensor).

  The camera 31 is a camera that is installed on a ceiling, a wall, a table, or the like in the store 35, for example, and images (takes) a food and drink space, for example. The camera 31 captures the eating and drinking space and stores the captured image data. The camera 31 may hold additional information (metadata such as shooting time and shooting location) regarding the image data together with the image data. The number of cameras 31 is one or more. The captured image data and its additional information may be stored in a memory inside the camera 31, or may be sent to the DB server 12 and stored in the storage 122. Based on the image data, the person (customer) and the object reflected in the image and the state in the store may be analyzed. This analysis may be performed by the processing unit in the camera 31, or may be sent to the analysis server 13 and performed by the processing unit 131. The camera 31 includes an imaging unit, a processing unit, a memory, a communication unit, and the like.

  The microphone 32 is a microphone that is installed on, for example, a ceiling, a wall, or a table in the store 35 and can collect sound generated in, for example, a dining space. The microphone 32 acquires sound that is pronounced in the eating and drinking space as audio data. The microphone 32 may hold additional information (metadata such as a sound collection time and a sound collection place) related to the sound data together with the sound data. The number of microphones 32 is one or more. The collected voice data and its additional information may be stored in a memory inside the microphone 32, or may be sent to the DB server 12 and stored in the storage 122. That is, the audio data may be recorded. Based on the audio data, the person (customer) and the object reflected in the image and the state in the store may be analyzed. This analysis may be performed by the processing unit in the microphone 32 or may be sent to the analysis server 13 and performed by the processing unit 131. Also, information such as sound pitch (for example, the pitch of a human voice), sound pressure level, and the like may be analyzed based on the sound data. In addition, the voice situation for each area may be recognized and held by a directional microphone as the microphone 32. The microphone 32 includes a sound collection unit, a processing unit, a memory, a communication unit, and the like.

  The digital signage 33 is installed, for example, near a cash register or the entrance of the store 35 and displays information such as characters and images. The device for displaying information is not limited to the digital signage 33 but may be a tablet terminal or the like. The tablet terminal may be disposed, for example, near a cash register, the entrance of the store 35, or on a table in a dining space. The digital signage 33 is an example of a display device. The digital signage 33 includes a processing unit, a memory, a communication unit, and the like.

  The camera 34 is installed, for example, in the vicinity of an entrance of a cash register or a store 35, and images, for example, a row of customers who are waiting for a cash register. The camera 34 may image, for example, the faces and whole bodies of customers who are waiting for a cash register. The camera 34 may hold additional information (metadata such as shooting time and shooting location) related to the image data together with the image data. The number of cameras 34 is one or more. The captured image data and its additional information may be stored in the memory inside the camera 34, or may be sent to the DB server 12 and stored in the storage 122. Based on the image data, the person (customer) and the object reflected in the image and the state in the store may be analyzed. This analysis may be performed by the processing unit in the camera 31, or may be sent to the analysis server 13 and performed by the processing unit 131. Further, the image captured by the camera 34 may be identifiable by the customer by face recognition, for example. The camera 34 includes an imaging unit, a processing unit, a memory, a communication unit, and the like. The image captured by the camera 34 may be used to obtain attribute information (gender, age, customer group, etc.) of the customer who has visited the store.

  The cameras 31 and 34 may be RGB cameras, infrared cameras, or other cameras.

  The information device 40 includes a smartphone 41 and a viewer 42. The smartphone 41 is a mobile terminal that a customer possesses. The mobile terminal is not limited to a smartphone, and may be a tablet terminal. The viewer 42 may be a display device such as a PC (Personal Computer). The smartphone 41 and the viewer 42 display customer-oriented information according to a browser or a dedicated application. The smartphone 41 has a configuration including a processing unit 411, a communication unit 412, a touch panel 413, and a memory 414. The touch panel 413 includes an operation unit and a display unit, but the operation unit and the display unit may be configured separately. Similarly, the viewer 42 may have a configuration including a processing unit, a communication unit, a touch panel, and a memory. The information device 40 is an example of a display device.

  The geographic information service server 7 includes a processing unit, a memory, a communication unit, and the like. The geographic information service server 7 provides position information (for example, GPS coordinates) of each point and information on the travel time to each point.

  The processing unit of each device according to the present embodiment realizes various functions by, for example, a processor executing a program held in a memory. The processor may include an MPU (Micro Processing Unit), a CPU (Central Processing Unit), a DSP (Digital Signal Processor), a GPU (Graphical Processing Unit), and the like.

  The memory of each device of the present embodiment includes a primary storage device (for example, a RAM (Random Access Memory) or a ROM (Read Only Memory)). The memory may include a secondary storage device (for example, HDD (Hard Disk Drive) or SSD (Solid State Drive)) or a tertiary storage device (for example, an optical disk or an SD card). The memory may include other storage devices. In the DB server 12 of the server 15, the memory 121 and the storage 122 are described separately. For example, the memory 121 may include a primary storage device, and the storage 122 may include a secondary storage device or a tertiary storage device.

  The communication unit of each device of the present embodiment communicates via wireless or wired communication. The communication method by the communication unit is, for example, a communication method such as WAN (Wide Area Network), LAN (Local Area Network), power line communication, short-range wireless communication (for example, Bluetooth (registered trademark) communication), communication for mobile phones, and the like. May include.

  Next, an overview of the store information providing system 5 will be described.

  In the store information providing system 5, a plurality of sensors (for example, the cameras 31 and 34, the microphone 32, and other sensors) detect various data and transmit various data (detection data) to the server 15. The Web server 11 receives detection data from the sensor. The analysis server 13 acquires the detection data, and the processing unit 131 analyzes the detection data, performs various types of analysis, and obtains analysis results. Various types of analysis may include, for example, customer classification, store atmosphere classification, customer behavior analysis, and customer trend analysis. The Web server 11 transmits the analysis result from the analysis server 13 to the digital signage 33. The digital signage 33 receives and displays the analysis result.

  For example, the processing unit 131 may analyze image data captured by the camera 31 and acquire information on the appearance and number of people included in the image. The processing unit 131 may analyze the image data captured by the camera 31 and acquire information on the space in the store and the sense of distance of people. The processing unit 131 may analyze the voice data collected by the microphone 32 and obtain information on the pitch and loudness of a person in the store. The processing unit 131 may detect humans by analyzing detection data from various sensors.

  The processing unit 131 may analyze various types of detection data and classify customers by segment (customer segment). For example, the processing unit 131 may detect a suit, a tie, or the number of persons included in sensor detection data (for example, image data) and classify the person or group located in the store as a business person. The processing unit 131 may detect a voice of a child included in the audio data and classify the person or group located in the store as a family. The processing unit 131 may detect a female voice included in the audio data and classify the person or group located in the store as a housewife. The processing unit 131 may detect a female voice included in the audio data and classify the person or group located in the store as OL (Office Lady). A child's voice and a female (housewife, OL) voice may be detected based on the frequency of the human voice in the voice data.

  The processing unit 131 may analyze various detection data and classify the atmosphere in the store. For example, the processing unit 131 may classify the store as quiet, busy, or the like based on voice information (for example, volume of audio data). Based on the image data, the processing unit 131 may classify the seats (tables and chairs) reflected in the image as being wide or narrow. The processing unit 131 may classify the degree of table filling (table utilization rate) based on data from the camera 31 or the infrared sensor.

  The processing unit 131 may classify the sense of distance between people based on data from the camera 31 or the infrared sensor, that is, classify the personal space situation and classify the atmosphere in the store. In this case, the processing unit 131 may classify the store as a family-friendly atmosphere when the distance between people is relatively close. Moreover, the process part 131 may classify | categorize a shop as the atmosphere for girls-only gatherings, when the distance between persons is a medium distance. Moreover, the process part 131 may classify | categorize that a shop is the atmosphere for business, when the distance between persons is a comparatively long distance.

  The processing unit 131 may analyze various types of detection data and perform behavior analysis of a person (customer). For example, the processing unit 131 may count the queue before the cash register, that is, the number of cashiers waiting for the cash register, based on the image captured by the camera 34. The processing unit 131 may track a person's movement based on detection data from the sensor, and may estimate the remaining staying time of the person when the person moves to a predetermined area in the store. For example, when the person moves to the toilet, the processing unit 131 may estimate that the remaining stay time is short, assuming that the preparation is for leaving the store. In addition, when the person moves to the free drink corner, the processing unit 131 may estimate that the remaining staying time is long because the drink is changed.

  The processing unit 131 may analyze various detection data and perform a tendency analysis of a person (customer). For example, the processing unit 131 may estimate that the staying time is short when it is detected that one young person is seated on one table based on detection data from the sensor. In this case, the processing unit 131 may estimate that the staying time is short when detecting that the user is seated for a meal without spreading a notebook or the like on the table. The fact that the notebook is not spread and that it is for a meal may be detected by object recognition of the notebook or tableware. When the processing unit 131 detects that a plurality of business persons are seated in one table based on the detection data from the sensor, the processing unit 131 may estimate that the stay time is about 30 minutes to 1 hour. When the processing unit 131 detects that a plurality of persons including infants are seated in a family at one table, the processing unit 131 may estimate that the person will leave immediately after eating, and estimate that the staying time is about one hour. It's okay. In this case, for example, the processing unit 131 may recognize an image captured by the camera 34 and determine that it is after a meal when no food is present on the tableware. When the processing unit 131 detects that a plurality of women, college students, and high school students are seated in one table, the processing unit 131 may estimate that the stay time is long (for example, one hour or more).

  The digital signage 33 receives and displays the analysis result from the analysis server 13. For example, the digital signage 33 may display information on the classified customer segment. The digital signage 33 may display the classified atmosphere in the store as an image such as a photo content. The digital signage 33 may display an image indicating that it is a business-oriented atmosphere when the atmosphere in the store is quiet and the distance between people is relatively long. The digital signage 33 displays information on the prediction of the stay time of the person (customer) in the store, or displays the person located in the store in a different display mode (for example, display color, display pattern) for each stay time. You can do it. For example, the digital signage 33 displays the position of a person with a relatively short stay time (for example, less than 30 minutes) in a light color, and displays the position of a person with a medium stay time (for example, 30 minutes to 1 hour) in an intermediate color. The position of a person having a relatively long stay time (for example, 1 hour or more) may be displayed in a dark color.

  The analysis process performed by the analysis server 13 may be performed by a processing unit on the sensor (for example, camera 31, microphone 32) side. For example, the camera 31 may recognize a person or an object from the captured image, transmit the image recognition result to the server 15, and register it in the DB server 12. In this case, the camera 31 may or may not transmit the captured image to the server 15. Similarly, the microphone 32 may perform voice recognition on the collected voice, transmit the voice recognition result to the server 15, and register it in the DB server 12. In this case, the microphone 32 may or may not transmit the collected sound to the server 15. Therefore, the analysis server 13 may use an image recognition result or a voice recognition result as detection data from the sensor.

  FIG. 2 is a table showing an example of the determination sheet 100 registered in the DB server 12.

  The DB server 12 stores the determination sheet 100 in the storage 122. The determination sheet 100 is a store suitable for the atmosphere of the store 35 based on the analysis result of the image data (image information) captured by the camera 31 and the analysis result of the sound data (voice information) collected by the microphone 32. 35 user uses are defined. The user application is a user application for the user to use the store 35. Note that detection data detected by other sensors may be considered in the determination sheet 100.

  The determination sheet 100 may be updated at a predetermined timing (for example, periodically) or may not be updated. The determination sheet 100 can be managed on the management side of the store 35. Therefore, the determination sheet 100 can be appropriately changed according to the policy of the store 35.

  The determination sheet 100 describes a user application determination material for determining a user application obtained as a result of analysis of at least one of image data and audio data. The user usage determination material includes, for example, the congestion degree of the store 35, the seat spacing of seats (for example, chairs) in the store 35, and the acoustic determination material of the store 35 (for example, the volume of the sound generated in the store 35 (signal level) ), Height (frequency), type (for example, sound of hitting a keyboard)). The user usage of the store 35 is determined (determined) by a combination of a plurality of user usage determination materials in the determination sheet 100. Details of the acquisition method of each user application determining material will be described later.

  Examples of user uses of the store 35 include, for example, for studying, for business, for dating, for students, for women's associations, and for families. For example, when the user usage of the store 35 is for studying, it indicates that the atmosphere of the store 35 matches the customer who wants to study at the store 35. Note that other user applications may be assumed as the user applications.

  Examples of the user usage determination material include “congestion degree”, “seat spacing”, “quiet”, “noisy”, “lively”, “high voice”, “low voice”, and the like. Note that other user application determination materials may be assumed as the user application determination materials.

  “Congestion degree” indicates the degree of congestion in the store 35. For example, the higher the usage rate of a seat (chair) or table, the higher the congestion level, and the lower the usage rate of the seat or table, the lower the congestion level. The degree of congestion is indicated by 0 to 100%, for example. For example, the notation of the degree of congestion “0% −” represents a range of 0% to 30%, and the notation of “30% −” represents a range of 30% to 50%. In FIG. 2, the degree of congestion is shown in a range of 5 levels, but may be shown in a range of 4 levels or less or 6 levels or more.

  The “seat interval” indicates an interval between a plurality of seats where a plurality of customers are seated. For example, the notation of the seat interval “−45 cm” represents a range of 0 cm to 45 cm, and the notation of “−120 cm” represents a range of 45 cm to 120 cm. In FIG. 2, the seat interval is shown in a range of two stages, but may be shown in a range of three or more stages.

  “Quiet” indicates that the volume of the audio data is less than a threshold th1 (eg, 45 db). “Zawazawa” indicates that the volume of the audio data is greater than or equal to a threshold th1 and less than a threshold th2 (for example, 55 db). “Lively” indicates that the volume of the audio data is equal to or greater than the threshold th2. The threshold values th1 and th2 are examples, and other values may be used. The thresholds th1 and th2 may be fixed values or variable values. Note that a voice of a person is extracted from the voice data, and the volume of the voice of the person is compared with threshold values th1 and th2, and it may be determined whether the voice is quiet, noisy, or busy.

  “Low voice” indicates that the frequency of the audio data is less than the threshold th3. “High voice” indicates that the frequency of the voice data is equal to or higher than the threshold th3. Note that a human voice may be extracted from the voice data, and the frequency of the human voice may be compared with the threshold th3 to determine whether the voice is low or high.

  Moreover, information other than the above may be included as the user usage determination material. For example, the user application determination material may include a sound type (audio data type). The type of sound is, for example, the type specified in the voice recognition process, and examples include a sound of hitting a keyboard, a laughing voice, a child's crying, and BGM (background music).

  In addition, the user usage of the store 35 may include “DC (Do n’t Care)” and “music” for which the user usage is not limited. For example, when the degree of congestion is low (for example, 0 to 30%), since there are not many people in the store, the atmosphere of the store is not influenced by the people in the store. Moreover, since there are not many people, the seat interval may be arbitrary. Therefore, it is considered that any user application is allowed. In addition, “music” indicates that when the degree of congestion is low and busy, there are not many people, so it is estimated that music is flowing in the store. In this case, the processing unit 131 may perform processing for determining the user usage of the store, or may be omitted.

  Further, when the degree of congestion is not low (for example, 30% or more), the store atmosphere is influenced by the presence of people in the store. In this case, the processing unit 131 performs store user usage determination processing. The processing unit 131 compares the detection data acquired from each sensor and the values of various recognition results for the detection data with the values in the user usage determination material of the determination sheet 100 to determine the user usage of the store 35 ( You may decide). As an example, when the degree of congestion is a value between 50% and 70%, the seat interval is a value between 45 cm and 120 cm, and the store 35 is quiet, the processing unit 131 is used for business users. You may decide.

  Note that the user application determination material and the user application in the determination sheet 100 illustrated in FIG. 2 are examples, and the user application may be determined by a combination of various other user application determination materials.

  For example, the determination sheet 100 defines that a user application is suitable for “business” when a person and a PC are detected by image recognition on image data and the number of detected PCs is 50% or more of the number of persons. You can do it. Further, the determination sheet 100 may be defined to be suitable for “students” when the person and PC are detected by image recognition on the image data and the number of notes is 50% or more of the number of persons. Good. Further, the determination sheet 100 may be defined as “children” when the number of persons estimated to be 20 years old or younger among the number of persons is 50% or more. Note that these 50% values are examples, and other values may be used as threshold values. Further, a plurality of user usages corresponding to the detected in-store situation may be assigned. For example, the situation in the store may be suitable for both “business” and “study” as a user application.

Next, an operation example of the store information providing system 5 is shown.
FIG. 3 is a flowchart showing an example of the entire processing procedure by the analysis server 13.

  The processing unit 131 of the analysis server 13 creates the determination sheet 100 in advance and stores it in the storage 122 of the DB server 12 (S1).

  The processing unit 131 determines whether or not a certain time (for example, 1 minute) has elapsed since the previous vacant seat status display process (S2). If the certain time has not elapsed, the processing unit 131 repeats the process of S2.

  When the predetermined time has elapsed, the processing unit 131 performs store layout acquisition processing (S3). Details of the store layout acquisition process will be described later. The processing unit 131 performs a seating status process (S4). Details of the seating status process will be described later. Based on the determination sheet 100, the processing unit 131 updates the display contents of the vacant seat status (S5). Details of the update processing of the display contents of the vacant seat status will be described later. Thereafter, the processing unit 131 returns to the process of S2.

  In the case of the store 35 whose store layout is not frequently changed, for example, a process for determining whether or not the store layout has been changed may be added before S3. For example, the processing unit 131 may determine whether or not the store layout has been changed by image analysis or the like based on the image of the store 35 captured by the cameras 31 and 34. The processing unit 131 may skip the process of S3 when the store layout is not changed, and may execute the process of S3 when the store layout is changed.

  FIG. 4 is a flowchart showing an example of a store layout acquisition processing procedure in S3. FIG. 5 is a diagram illustrating an example of a store layout.

  In the store layout of FIG. 5, at least two tables T are arranged in the store 35. The table T is identified by Ti (i = 1, 2,...) That is a table ID. The two tables T are identified by table IDs T1 and T2. Hereinafter, the table T whose table ID is T1 is referred to as a table T1. The same applies to other tables.

  In the store layout of FIG. 5, four chairs C (C1, C2, C3, C4) are arranged on opposite sides of the table T1. Similarly, four chairs C (C5, C6, C7, C8) are arranged on opposite sides of the table T2. The chair C is identified by a table ID Cm (m = 1, 2,...). Eight chairs C are identified by chair IDs C1, C2,..., C8. Hereinafter, a chair C whose chair ID is C1 is referred to as a chair C1. The same applies to other chairs.

  In the store layout of FIG. 5, an XY coordinate system is set.

  In the store layout acquisition process, first, the processing unit 131 performs image recognition on the image of the store 35 captured by the camera 31 (S31). As a result of the image recognition, the processing unit 131 recognizes the two tables T included in the image, and registers the coordinate information of the two tables T in the table information table 151 (S32). The processing unit 131 acquires the shortest distance Min (Ti, Ti + 1) between adjacent tables (between tables) and registers it in the table information table 151 (S33).

  FIG. 6A is a diagram showing the registered contents of the table information table 151. The table information table 151 is registered (held) in the storage 122 of the DB server 12. In the table information table 151, for example, table ID, X coordinate, Y coordinate, seating rate, shortest distance Min, number of seats, area, and time stamp are registered. The shortest distance Min may be an example of a seat interval of the determination seat 100.

  Here, the X coordinate indicates the X direction component of the arrangement position of the table T. The Y coordinate indicates the Y direction component of the arrangement position of the table T. The seating rate is one index indicating the usage status of the table T, and indicates the usage rate of the chair C belonging to the same table T. The shortest distance Min indicates the shortest distance among any one or more adjacent tables. The unit of the shortest distance may be cm. The area indicates an area where the table T in the store 35 is arranged. The number of seats indicates the number of chairs C belonging to the table T1 (forming a group with the table T1). The time stamp represents the time when information related to the table T is recorded in the table information table 151.

  In the case of the table T1, the table information table 151 includes a table ID: T1, X coordinate: 30, Y coordinate: 30, shortest distance Min: 100, number of seats: 4, area: 1st floor, time stamp: 00:00: 00 is included and registered. In the case of the table T2, the table ID: T2, the X coordinate: 130, the Y coordinate: 30, the shortest distance Min: 100 cm, the number of seats: 4, the area: the first floor, and the time stamp: 00:00:00 are included. Are registered in the table information table 151. In the store layout acquisition process, it is not determined whether or not the table T1 is used, so the presence status information is blank.

  As a result of the image recognition, the processing unit 131 recognizes the eight chairs C included in the image of the store 35 and registers the coordinate information of the eight chairs C in the chair information table 153 (S34).

  In addition, information on distances other than the shortest distance Min may be used, and for example, an average distance (an average of distances with one or more adjacent tables) may be used.

  FIG. 6B is a diagram showing the registered contents of the chair information table 153. The chair information table 153 is registered (held) in the storage 122 of the DB server 12. In the chair information table 153, a chair ID, an X coordinate, a Y coordinate, a user, a table ID, and a time stamp are registered.

  Here, the X coordinate indicates the X direction component of the arrangement position of the chair C. The Y coordinate indicates the Y direction component of the arrangement position of the chair C. The user is a customer who uses the chair C, and is identified by, for example, a customer ID (human ID). The table ID is the table ID of the table T to which the chair C belongs. The table T to which the chair C belongs may be, for example, a table located in the vicinity of the chair C when compared with other tables T, or a table having a design that is the same as or corresponding to the design of the chair C. The time stamp represents the time when information related to the chair C is recorded in the chair information table 153.

  In the case of the chair C1, the chair ID: C1, the X coordinate: 20, the Y coordinate: 20, the table ID = T1, and the time stamp: 00:00:00 are included and registered in the chair information table 153. In the case of the chair C5, the chair ID: C5, the X coordinate: 120, the Y coordinate: 20, the table ID = T2, and the time stamp: 00:00:00 are included and registered in the chair information table 153. In the store layout acquisition process, it is not determined whether or not the chair C is used, so the information on the user who uses the chair C is blank.

  The processing unit 131 groups the table T and the chair C for each table T, creates a group TG (Ti, Cm) of the table T and the chair C, and registers them in the DB server 12 (S35). In this case, the processing unit 131 may register the table ID of the table T forming the group with the chair C in the record of the chair C in the chair information table 153. In FIG. 6B, the processing unit 131 may register the table ID of the table T1 in the record of the chair C3, for example. Thereafter, the processing unit 131 ends the store layout acquisition process.

  FIG. 7 is a flowchart showing an example of the processing procedure of the seating situation in S4. FIG. 8 is a diagram illustrating an example of a seating situation in the store 35. FIG. 8 shows a situation where a person H (H1) holding a bag (object O (O1)) is seated on the table T1 out of the two tables T1 and T2 shown in FIG.

  FIG. 9A is a table showing an example of registered contents of the person information table 154. The person information table 154 is registered (held) in the storage 122 of the DB server 12. The person H is identified by a human ID Hn (n = 1, 2,...). Hereinafter, the person H whose human ID is H1 is referred to as a person H1. In the human information table 154, a human ID, an attribute, an X coordinate, a Y coordinate, an area, a human distance, and a time stamp are registered.

  Here, the attribute is an example of feature information indicating the feature of the person H. The X coordinate indicates the X direction component of the position where the person H is located (for example, the position of the head). The Y coordinate indicates the Y direction component of the position where the person H is located. The area indicates an area where the person H in the store 35 is located. The human distance indicates a distance to another person, for example, may indicate a distance to the other person who is closest. The time stamp represents the time when the information about the person H is recorded in the person information table 154.

  In FIG. 9A, information on the position of the person H (X coordinate, Y coordinate) is accumulated in the person information table 154 at each timing, so that the seating status record of the person H may be shown in time series. Information on the position (X coordinate, Y coordinate) of the person H accumulated at each timing may be used for flow line management for tracking and managing the person H.

  FIG. 9B is a table showing an example of registered contents of the object information table 155. The object information table 155 is registered (held) in the storage 122 of the DB server 12. The object O is identified by the object ID Oj (j = 1, 2,...). An object O whose product ID is O1 is referred to as an object O1. In the object information table 155, a product ID, an estimation, an X coordinate, a Y coordinate, an area, and a time stamp are registered.

  Here, the estimation indicates an estimation result as to what kind of object the object O is. The estimation result may be obtained by, for example, image recognition or object recognition on the image of the store 35 by the processing unit 131. The X coordinate indicates the X direction component of the position where the object O is located. The Y coordinate indicates the Y direction component of the position where the object O is located. The area indicates an area where the object O in the store 35 is located. The time stamp represents the time when information related to the object O is recorded in the object information table 155.

  In FIG. 9B, the information on the position of the object O (X coordinate, Y coordinate) is accumulated at each timing in the object information table 155, so that the seating status record of the object O may be shown in time series.

  In the seating situation shown in FIG. 8, a person H1 with an object O1 (鞄) sits on a chair C3 provided on the table T1, and places the object O1 on a chair C4 adjacent to the chair C3.

  In FIG. 9A, in the record of the first seating situation, the human information table 154 includes human ID: H1, X coordinate: 27, Y coordinate: 41, area: 1F, time stamp: 00:00:00. be registered. In the subsequent recording of the seating status, the human information table 154 includes a human ID: H1, an X coordinate: 27, a Y coordinate: 41, an area: 1F, a time stamp: 00:01:00, and is additionally registered. .

  Also, by placing the object O1 on the adjacent chair C4, the object O1 is recognized, and in the object information table 155, the object ID: O1, the estimation: 鞄, the X coordinate: 40, the Y coordinate: 40, the area: 1F Time stamp: 00:01:00 is registered.

  In the processing of the seating situation shown in FIG. 7, first, the processing unit 131 captures images in the vicinity of each of the eight chairs C <b> 1 to C <b> 8 out of images captured by the camera 31 (e.g., area 1F). Image) is extracted (S41). The extracted image may be an image held in the camera 31, or an image stored in the DB server 12 may be acquired by communication.

  The processing unit 131 performs recognition processing for recognizing the person H or the object O from the extracted image, for example, by image recognition (S42). When a person is recognized in this recognition processing, the processing unit 131 further recognizes the object O from the extracted image. Here, the processing unit 131 performs processing for recognizing an object after a person is recognized. However, the processing unit 131 may perform processing for recognizing an object even if the person is not recognized.

  As a result of the recognition of the person or the object, the processing unit 131 determines whether or not the person or the object is recognized (S43). If no person or object is recognized, the processing unit 131 returns to the process of S41. On the other hand, when a person or an object is recognized, the processing unit 131 acquires the coordinates of the person or the object (S44). In this case, the processing unit 131 may acquire the coordinates of the person or the object (an example of the location) based on the position of the person or the object in the image in which the person or the object is reflected. The processing unit 131 can acquire information on the shooting location of an image in which a person or an object is captured, for example, from metadata about the image.

  The processing unit 131 determines whether or not the coordinates of the person or thing acquired this time have changed significantly compared to the coordinates of the person or thing acquired last time (S45). In S45, for example, the processing unit 131 may determine whether the amount of change in the coordinates of the person or the object is less than the threshold th4, thereby determining whether or not the change is large. That is, in S45, the processing unit 131 may determine whether a person or an object is moving.

  When the coordinates of the recognized person or object are not greatly changed from the previously acquired coordinates, the processing unit 131 indicates that the recognized person or object is the nearest among all the chairs Cm (C1 to C8). Assuming that the user is seated in the chair Cm to be used, the use (sitting) status is added (S46).

  FIG. 9C is a table showing an example of a record of the seating situation of chair C in time series. The chair seating situation table 156 is registered in the storage 122 (holding) of the DB server 12. In the chair seating situation table 156, items of chair ID, X coordinate, Y coordinate, user, table ID, and time stamp are registered. The time stamp represents the time when the seating situation was recorded.

  The items held in the chair seating situation table 156 may be the same as the items held in the chair information table 153. The processing unit 131 may write at least part of the information held in the chair information table 153 in the chair seating situation table 156. Further, the seating status of the chair C may be recorded by adding a record to the chair information table 153 without preparing the chair seating status table 156.

  Assuming the seating situation shown in FIG. 8, the chair seating situation table 156 includes a chair ID: C3, an X coordinate: 20, a Y coordinate: 40, a user: H1, a table ID: T1, and a time stamp: 00:01. : 00 is registered. Furthermore, chair ID: C4, X coordinate: 40, Y coordinate: 40, user: O1, table ID: T1, and time stamp: 00:01:00 are registered in the chair seating status table 156. Thereby, the situation where the person H1 puts the object O1 (１) on the adjacent chair C4 and is seated on the chair C3 is registered in the DB server 12 in the table T1.

  On the other hand, in S45, when the coordinates of the recognized person or object have changed greatly from the previously acquired coordinates, the processing unit 131 determines that the person or object is moving, and the chair Cm (here, C3 , C4) is deleted from the chair seating situation table 156 (S47). In this case, the record information related to the chairs C3 and C4 in FIG. 9C may be deleted from the storage 122 of the DB server 12.

  The processing unit 131 calculates and updates the seating rates of the tables T1 having the chairs C1 to C4 and the tables T2 having the chairs C5 to C8 for all the tables Ti (i = 1, 2). (S48).

  FIG. 9D is a table showing an example of the recording of the seating status of the table T in time series. The table seating situation table 157 is registered (held) in the storage 122 of the DB server 12. In the table seating situation table 157, table ID, X coordinate, Y coordinate, seating rate, minimum distance Min, number of seats, and time stamp are registered. The time stamp represents the time when the seating situation processing is executed.

  The items held in the table seating situation table 157 may be the same as the items held in the table information table 151. The processing unit 131 may write at least part of the information held in the table information table 151 in the table seating situation table 157. Alternatively, the seating status of the table T1 may be recorded by sequentially adding records to the table information table 151 without preparing the table seating status table 157.

  The seating rate is a value obtained by dividing the number of chairs C that are not blank (in use) among the chairs C included in the table T by the number of seats. When the seating rate is not 0, the processing unit 131 can consider that the table is being used. Assuming the seating situation shown in FIG. 8, the table seating situation table 157 includes a table ID: T1, an X coordinate: 30, a Y coordinate: 30, a seating situation: 0.50, a shortest distance Min: 100, and the number of seats: 4 and the time stamp: 00:01:00 are registered.

  In S47, it is shown that the record related to the chair C used by the moving customer is deleted, but it is also conceivable that the moving customer leaves the seat C and then sits down again. In this case, the processing unit 131 may associate whether or not the customers before, during, and after the movement are the same person by a flow line analysis process that tracks the movement of the customer. When linked with the same person, even if the position coordinate of the customer who has left the seat changes greatly, the processing unit 131 does not have to delete the record related to the chair C used by the customer. Accordingly, the store information providing system 5 can recognize that the customer who has left the seat is seated again and continues to use the chair C.

  FIG. 10 is a flowchart showing an example of the recognition processing procedure of the person H or the object O in S42.

  The processing unit 131 acquires image data of an image captured by the camera 31 (S61). The processing unit 131 estimates the attribute of the person shown in the image from the appearance (S62). The estimation from the appearance refers to the estimation by the image recognition process by the processing unit. The attribute of the person H may include the age, sex, and category of the person H. The category may indicate what kind of customer the person H is, and may correspond to a user application. The processing unit 131 may use learning data registered in the DB server 12 when estimating human attributes. The learning data is accumulated in the storage 122 by machine learning such as deep learning. For example, the processing unit 131 generates a learned model based on accumulated learning data using artificial intelligence, inputs an image representing a person's appearance into the learned model, and outputs human attribute data as an output thereof. get.

  FIG. 11A is a table showing an example of human attribute information.

  The human attribute information table 161 includes, for example, age, sex, and category attributes. FIG. 11B is a table showing category determination based on appearance. The category type table 164 is used for the category determination based on appearance. The category type table 164 is registered (held) in the storage 122 of the DB server 12. The category type table 164 registers the category type and the result of recognition (image recognition) by appearance.

  For example, in recognition by appearance, when a tie, suit, and work clothes are recognized, it is determined that the category is business in the category type. That is, it is estimated that the person H is a businessman. Further, in the recognition by appearance, when the uniform is recognized, it is determined as a student in the category type. That is, it is estimated that the person H is a student. Further, in the recognition by appearance, when the school bag is recognized, the category type is determined as an elementary school student. That is, it is estimated that the person H is an elementary school student. As a result of the estimation, age: 30, sex: male, category: business are registered in the human attribute information table 161 as human attributes.

  The processing unit 131 recognizes an object from the image data (S63). Object recognition may be performed by image recognition. The recognition of the object may be performed using the learning data, similarly to the estimation of the human attribute.

  FIG. 11C is a table showing an example of object estimation. In the object estimation table 162, product ID: O1, estimation: dark blue, color: brown, size: medium are registered as estimation results. The object estimation table 162 is registered (held) in the storage 122 of the DB server 12. Note that the object estimation table 162 may be provided separately from the object information table 155 illustrated in FIG. 9B, and is the same as the object information table 155, and an estimation result is stored in the item “estimation” of the object information table 155. May be written.

  Subsequently, the processing unit 131 acquires the sound (audio data) collected by the microphone 32 (S64). The processing unit 131 estimates the degree of success of the store 35 from the volume (sound pressure level) of the sound collected by the microphone 32 (S65). The processing unit 131 estimates the age and gender of the person H based on the characteristics of the voice (for example, pitch and frequency) (S66). The processing unit 131 performs speech recognition and estimates an object (S67). As an estimation of an object by speech recognition, for example, it is estimated that the sound is a keyboard hitting sound.

  FIG. 11D is a table showing an example of sound attributes. In the sound attribute table 163, information values related to sound attributes are registered. The sound attributes may include, for example, the degree of success, age, sex, and recognition result (estimation result) of the object O. The prosperity indicates the liveliness of the store 35, and may include, for example, information on quietness, troublesomeness, and liveliness. In FIG. 11D, as a sound attribute and its value, the degree of success: quiet, age / sex: male 20-60s, voice recognition: keyboard is registered. Note that the audio attribute may be estimated, for example, at least one of volume and pitch.

  The person H or the object O is recognized based on detection data (for example, image data captured by the camera 31) by the main sensor, and detection data (for example, voice data collected by the microphone 32 or infrared rays) by the sub sensor. Sensor detection data) may be used supplementarily. Which sensor is used as a main sensor or a sub sensor is arbitrary. The store information providing system 5 can recognize the person H and the object O in consideration of the main sensor and the sub sensor, thereby reducing the recognition processing while reducing the processing load of the recognition process as much as possible.

  FIG. 12 is a flowchart illustrating an example of a display content update processing procedure in S5.

  The processing unit 131 calculates the congestion level of the store 35 (S71). The degree of congestion is, for example, a value obtained by dividing the number of tables T in use by the total number of tables T arranged in the store 35. Therefore, the degree of congestion corresponds to the usage rate of the table T. As described above, the table T is treated as being used when the seating rate is not 0. For example, the seating rate is calculated based on the number of chairs C that are not blank (that is, in use) among the chairs C provided in the table T based on the chair seating situation table 156, and the number of chairs C provided in the table T (seats). The value divided by the number.

  FIG. 13A is a table showing an example of the recording of the seating status of the table T in time series. The registration contents of the table seating situation table 157A are substantially the same as the table seating situation table 157 shown in FIG. 9D except for the area item.

  In FIG. 13A, the table seating status table 157A includes table ID: T1, X coordinate: 30, Y coordinate: 30, seating rate: 0.50, shortest distance Min: 100, number of seats: 4, and time stamp: 00. : 00: 00 is registered. The table seating status table 157A includes table ID: T2, X coordinate: 130, Y coordinate: 30, seating rate: 0.50, shortest distance Min: 100, number of seats: 4, and time stamp: 0:00. : 00 is registered.

  Note that an area item may be added to the table seating status table 157A or an area item may be deleted from the table seating status table 157, and a common table may be used as the table seating status tables 157 and 157A and updated as appropriate. Instead of using the store layout information, the degree of congestion may be obtained by comparing the past history and the reference image with the position information and the number of people detected. The past history and the reference image are held in the storage 122, for example, and may be referred to as appropriate.

  The processing unit 131 calculates the stay time of the person H (S72). In this case, the processing unit 131 extracts images in the vicinity of each of the chairs Cm for the eight chairs Cm (m = 1, 2,...) From the image captured by the camera 31, and the person H is extracted from the extracted images. The process of recognizing may be performed.

  FIG. 13B is a table showing an example of a record of a person's stay time. In the person staying time table 158, human ID: H1, attribute: male, 30s, business, X coordinate: 27, Y coordinate: 41, average of person distance: 41, staying time: 1:00, and time stamp : 0: 01: 00 are registered. The person staying time table 158 is registered (held) in the storage 122 of the DB server 12. Note that the average of the distances of the person H may be an average value of the distances between the plurality of persons H recognized from the image. The average distance to the adjacent seat may be an average value of distances between adjacent chairs C among the plurality of chairs C recognized from the image. The average of the distance of the person H or the average of the distance to the adjacent seat may be an example of the seat interval of the determination sheet 100.

  The processing unit 131 determines whether or not the degree of congestion is greater than or equal to a threshold th5 (for example, 30%) (S73). When the congestion level is less than the threshold th5 (for example, 30%), the processing unit 131 proceeds to the process of S76. On the other hand, when the degree of congestion is 30% or more, the processing unit 131 calculates the average value of the distance to the adjacent seat based on the seat interval (S74). The seat interval may be calculated based on the arrangement position (X coordinate, Y coordinate) of each chair recorded in the chair information table 153, for example, and the interval of each chair C reflected in the image may be image recognition. What was acquired by etc. may be sufficient. The seat interval may be an average value of a plurality of seat intervals. The processing unit 131 calculates the average value of the distance of the person H based on the seat interval (S75). For example, when the person H is seated, the distance between the chairs C being used may be the distance of the person H. The processing unit 131 reflects the average value of the distance to the adjacent seat obtained in S74 and the average value of the distance of the person obtained in S75 in the person staying time table 158.

  The processing unit 131 determines the store situation using the determination sheet 100 (S76). The store situation may be expressed as a user application for the customer to use the store 35. For example, the processing unit 131 may determine the user usage of the store 35 based on the congestion degree and the seat interval (however, when the congestion degree is less than 30%, the seat interval is not used as a condition). The processing unit 131 may determine the user application of the store 35 based on at least one of the degree of congestion, the seat interval, the volume in the store 35, the pitch of the sound, and the like. The processing unit 131 reflects the result of the determination in the store situation table 159.

  FIG. 13C is a table showing an example of registered contents of the store status table 159. In the store status table 159, for example, store ID: A store, area: 1F, category: business, congestion: 50%, distribution of goods O: {鞄: 1}, distribution of staying time: {30 minutes or less: 1, 30 minutes or more: 0, 60 minutes or more: 0}, sound volume: 30 dB, sound pitch distribution: blank, and time stamp: 00:01:00 are registered. When the object O is placed on the chair C, the object O occupies the chair C. Therefore, it can be considered that the object O is seated on the chair C. In this case, the object O affects the seating rate and affects the determination of the user application. Therefore, information on the object O is held and managed in a table. Further, the processing unit 131 detects the type of the object O, for example, when detecting that the number of notebook PCs is equal to or greater than a certain number compared to the number of tables, the user application is for nomad workers, etc. A determination may be made. In order to make this determination, an item relating to the object O may be added to the determination table.

  The store ID is an example of identification information for identifying the store 35. The area is a partial area in the store 35. Moreover, the category here shows the category of the store 35, and shows the user use (for example, for business) of the store 35. The value in the stay time distribution indicates how many people H stay for each stay time (for example, 30 minutes or less, 60 minutes or more). The stay time distribution is an index that indicates how long the store 35 is to be withdrawn from, and indicates a turnover rate that indicates the frequency of customer entry and exit (rotation) of the store 35.

  The processing unit 131 changes the display contents of the store status displayed on the digital signage 33 or the like based on the store status table 159 (S77). In this case, the communication unit 111 of the Web server 11 transmits information (display information) necessary for displaying the display contents of the store status to the digital signage 33 (an example of a display device). The display information includes at least a part of the information held in the store status table 159. The display information includes, for example, information on a time stamp (time for changing display contents), an area (for example, 1F), a user application (for example, for business), a degree of congestion (for example, 10%), and a stay time distribution. It's okay. The digital signage 33 receives the display information and displays it according to the display information. In this case, the digital signage 33 updates the display contents when there is display information at the time of previous display and changed information. After S77, the processing unit 131 ends the display content update process.

  As described above, according to the operation of the store information providing system 5, it is possible to determine the user usage of the store based on the seating situation and update the display content at regular intervals. Therefore, for example, unlike the fixed user usage that the store 35 transmits by itself, the user usage of the store 35 that can change with time can be appropriately presented to the customer. Moreover, the information for the user use which the store 35 transmits by itself becomes subjective information. On the other hand, the store information providing system 5 can transmit store information such as a user's use using objective data based on detection data from a plurality of sensors.

  Further, the operation of the store information providing system 5 may be performed for each area (for example, 1F, 2F, 3F,...) In the store 35. That is, the store information providing system 5 may perform store layout acquisition processing, seating status processing, and display content update processing for each area. Thereby, also in the same store 35, it can be determined for a different user use for every area.

  Note that the process of S73 may be omitted. In this case, the store information providing system 5 can omit the user application determination process that is likely to be “DC” in the determination sheet 100, and can reduce the processing load.

  FIG. 14 is a diagram illustrating an example of the display screen GM1 displayed on the digital signage 33.

  On the display screen GM1, the vacant seat status is shown, and the user application, the degree of congestion, and the rotation rate are displayed for each area (3F, 2F, 1F, B1F) of the store 35. The user application may be expressed by an image (user application image) (character, photograph, video, illustration, etc.) indicating the user application. In 3F, a business user use image GZ1 is displayed. In 2F, a user application image GZ2 of the women's association is displayed. In 1F, a family user-use image GZ3 is displayed. In B1F, a business / study user-use image GZ4 is displayed. In B1F, user application images that commonly indicate two user applications may be displayed, or two user application images that individually indicate the two user applications may be displayed. When the user-use image is a photograph or a moving image, a real-time image may be an image captured in the past and accumulated in the storage 122. In addition, when the information for identifying the person H such as the face is included in the user-use image or the information to be kept secret is included, the information for specifying the person H or the information to be kept secret is masked. May be.

  The rotation rate indicates the ratio of the time during which the person H is staying with a bar display. For example, out of the bars ba representing the overall ratio of staying time, the bar ba1 represents the ratio of the person H staying in less than 30 minutes. Bar ba2 represents the percentage of people H staying for 30 minutes or more and less than 60 minutes. Bar ba3 represents the percentage of people H staying in 60 minutes or more. Information about the turnover rate and stay time may be sent from the server to the digital signage.

  For example, in 3F, the user application is business. Since the degree of congestion is 10%, a clear mark is displayed. As for the turnover rate, as indicated by the length of the bar ba1, the percentage of people staying in less than 30 minutes is 100%. On the 2nd floor, the user application is the girls' association. Since the degree of congestion is 50%, a cloudy mark is displayed. As for the rotation rate, as indicated by the length of the bar ba1, the proportion of people staying in less than 30 minutes is approximately 33%. As indicated by the length of the bar ba2, the percentage of people staying in the range of 30 minutes to less than 60 minutes is approximately 33%. As indicated by the length of the bar ba3, the proportion of people staying in 60 minutes or more is approximately 33%.

  In 1F, a user use is a family. Since the degree of congestion is 100%, a rain mark is displayed. As for the rotation rate, as indicated by the length of the bar ba1, the percentage of people staying in less than 30 minutes is approximately 10%. As indicated by the length of the bar ba2, the proportion of people staying in the range of 30 minutes to less than 60 minutes is approximately 30%. As indicated by the length of the bar ba3, the percentage of people staying in 60 minutes or more is approximately 60%.

  In B1F, the user uses business and study. Since the degree of congestion is 10%, a clear mark is displayed. As for the rotation rate, as indicated by the length of the bar ba1, the rate of people staying in less than 30 minutes is about 33%. As indicated by the length of the bar ba2, the percentage of people staying in the range of 30 minutes to less than 60 minutes is approximately 33%. As indicated by the length of the bar ba3, the proportion of people staying in 60 minutes or more is approximately 33%.

  As described above, the digital signage 33 displays the display screen GM1, so that a customer who intends to use the store 35 from now on can determine which area is empty. Therefore, the store information providing system 5 has a store 35 having a plurality of floors (for example, 1F, 2F, 3F), and even when the seats near the entrance of the 1F are crowded, for example, when there is a relatively vacant space on the 3F. The fact that there is a vacancy on the 3rd floor can be presented together with the usage of the store 35 user. Therefore, the customer can check the usage of the user and the congestion level for each area, so that even if the vicinity of the entrance is crowded, if the store is suitable for the customer's taste, the area can be smoothed out. Can visit.

  The various tables shown in the present embodiment may be registered (held) in the DB server 12 for each store 35. The DB server 12 may hold various tables for each of the plurality of stores 35.

  As described above, the store information providing system 5 analyzes the usage rate, the seating rate, the seat interval, the bustleness, etc. of the table T by the recognition process for the detection data obtained by the sensor, and stores the store information suitable for each customer segment. Information can be displayed on the digital signage 33 or the information device 40 (for example, the smartphone 41) in the store. Thereby, the store information providing system 5 can realize store guidance that matches the taste of the customer.

  As described above, the store information providing system 5 of the first embodiment includes the camera 31 and the microphone 32 provided in the store 35, the Web server 11, the DB server 12 and the analysis server 13, and the digital signage 33. Prepare. The camera 31 images the store 35 to acquire image data, and transmits the image data to the Web server 11. The microphone 32 collects sound in the store 35 to acquire voice data and transmits it to the Web server 11. The Web server 11 receives image data and audio data. The analysis server 13 recognizes the positions of the person H and the object O located in the store 35 based on the image data. The analysis server 13 calculates the congestion degree of the store 35 based on the positions of the person H and the object O. The analysis server 13 determines a user application for the user to use the store 35 based on the congestion level and at least one of the image data and the audio data.

  The microphone 32 is an example of a microphone. The camera 31 and the microphone 32 are examples of a plurality of sensors. The Web server 11, the DB server 12, and the analysis server 13 are examples of servers. The digital signage 33 is an example of a display device. Acquiring and acquiring image data, and acquiring sound data by collecting sound in the store 35 are examples of acquiring a plurality of detection data by sensing. The object O is an example of an object.

  As a result, the store information providing system 5 can provide user-use information for the customer to use the store 35. Since this user usage information is estimated based on detection data from the camera 31 and the microphone 32 arranged in the store 35, the atmosphere of the store 35 is taken into account. Therefore, the customer who has received the information on the usage of the user of the store 35 can select the user-used store 35 that meets the customer's own needs and preferences and visit the store 35. In this case, it is less likely that the store 35 visited by the customer is a store 35 that does not meet the user's desires and preferences. That is, the store information providing system 5 can perform store guidance that meets customer demands and preferences depending on the user application. In this way, the store information providing system 5 can provide information as to whether the in-store situation matches the customer's request or preference.

  Further, the Web server 11 may acquire information on the store layout of the table T and the chair C in the store 35. The analysis server 13 may calculate the degree of congestion of the store 35 based on the information on the store layout of the table T and the chair C and the position of at least one of the person H and the object O. The table T and the chair C are examples of seats. The table layout information of the table T and the chair C is an example of seat arrangement information.

  Thereby, since the store information providing system 5 takes into account the store layout, for example, a person or an object O located near the chair C or the table T in the store 35 can be associated with the chair C or the table T and managed. Then, the store information providing system 5 can determine that the person H or the object O associated with the store 5 is seated and estimate the degree of congestion in the store 35.

  The store information providing system 5 may include a digital signage 33, and the Web server 11 may transmit user usage information of the store 35. The digital signage 33 may receive and display information about the user usage of the store 35.

  Thereby, the store information provision system 5 can visually provide information on whether the in-store situation matches the customer's request or preference.

  The analysis server 13 determines the user usage of the store 35 when the congestion level is equal to or higher than a threshold th5 (for example, 30%), and determines the user usage of the store 35 when the congestion level is less than the threshold th5. You don't have to.

  Thereby, when the degree of congestion is low, it is considered that there are relatively many vacant seats and there are few people H and objects O. Therefore, it is considered that the store atmosphere based on the attribute of the person H, the attribute of the object O, and the seating situation is not extremely different, and it is considered that the provision of information of the store 35 is not so necessary. Therefore, the store information providing system 5 can provide the user usage information efficiently by providing the user usage information only when it is congested to some extent. Further, even if the customer is not provided with information for user use when the degree of congestion is low, the atmosphere of each store 35 is similar, and it is considered that there is not much difference from the customer's request and preference.

  The analysis server 13 may receive image data and audio data at a plurality of points in time. The analysis server 13 may recognize the positions of the person H and the object O located in the store 35 based on the image data. The analysis server 13 may calculate the congestion degree of the store 35 based on the arrangement information of the table T and the chair C and the positions of the person H and the object O. The analysis server 13 may determine the user usage of the store 35 based on the degree of congestion and at least one of image data and audio data. The analysis server 13 may transmit the determined user usage information of the store 35 to the digital signage 33. The digital signage 33 may receive the user usage information of the store 35 and update the display of the user usage information of the store 35 at a later point in time.

  Thereby, the customer can determine the store 35 to be used by looking at the user usage of the store 35 displayed on the digital signage 33, and can satisfy the user's request and preference. In addition, the store information providing system 5 performs user usage determination at a plurality of timings and updates the user usage information, so that the user usage that can be changed in consideration of the customer's entry and exit from the store 35. Can be provided accurately.

  Further, the analysis server 13 recognizes the seating positions of a plurality of people at the store 35 based on the arrangement information of the table T and the chair C and at least one position of the person H and the object O. The analysis server 13 calculates the interval between the seating positions of a plurality of people (for example, the seat interval in FIG. 2), and determines the usage of the store 35 based on the interval between the seating positions of the plurality of people.

  Thereby, the interval between the seating positions of the customers in the store 35 can be an index indicating the relationship between the customers. Therefore, by determining the user usage of the store 35 based on the interval between the seating positions, the user usage considering the relationship between customers is determined, and store information such as the degree of congestion corresponding to the user usage is provided. it can.

  For example, when the interval between the seating positions is short, it is assumed that the parent and child are customers, and the store information providing system 5 can provide information that the store is a family oriented store 35 as a user application. For example, when the interval between the seating positions is long, it is assumed that the business person who is engaged in a business negotiation is a customer, and the store information providing system can provide information indicating that the store 35 is for business.

  Further, the analysis server 13 may determine the user application of the store 35 based on the signal level of the audio signal (audio data) collected by the microphone 32.

  The signal level of the audio signal in the store 35 indicates, for example, the degree of success of the store 35 and can serve as an index indicating the characteristics of the sound space of the entire store 35. Therefore, the store information providing system 5 determines the user usage of the store 35 based on the sound level of the store 35 by determining the user usage of the store 35 based on the signal level of the audio signal in the store 35, Can provide store information.

  Further, the analysis server 13 may determine the pitch of a human voice included in the audio signal collected by the microphone 32. The analysis server 13 may determine the user usage of the store 35 based on the voice level of the person.

  The loudness of the voice of the person in the store 35 can be an index indicating the customer group (customer group) of the store 35 (for example, male, female, adult, child). Therefore, the store information providing system 5 can provide information on the user usage of the store 35 according to the customer segment of the customer. Note that since the audio signal includes noise, the height (frequency) of the collected audio signal is not simply determined, but after the customer's voice is extracted, the pitch (frequency) of the customer's voice is extracted. ) May be determined.

  Further, the analysis server 13 may recognize the object O located in the store 35. The analysis server 13 may determine the user usage of the store 35 based on the recognized object O.

  Thereby, the store information provision system 5 can estimate the customer segment of the customer corresponding to the possession by recognizing the customer's possession, and can provide the user usage information of the store 35 corresponding to the customer segment.

  Further, the analysis server 13 may transmit information on the congestion degree of the store 35 to the digital signage 33. The digital signage 33 may receive and display information on the degree of congestion of the store 35.

  As a result, the customer can check the congestion degree information of the store 35 along with the user usage information, and can check the congestion degree of the store 35 that meets the customer's request and preference. Therefore, the customer who has received information on the degree of congestion of the store 35 and the user's use is the user's use store 35 that matches the customer's own desires and preferences, and selects the store 35 having a low degree of congestion. Can visit.

  The analysis server 13 is located in the store 35 based on at least one of the image data of the image captured by the camera 31 and the audio signal collected by the microphone 32 obtained at a plurality of timings. The stay time of the person H and the object O may be calculated. The analysis server 13 transmits the calculated stay time information to the digital signage 33. The digital signage 33 may receive and display the stay time information.

  Thereby, the confirmer of the display of a display apparatus (for example, digital signage 33, smart phone 41) can confirm how long the customer located in the store 35 is a customer staying. That is, the display checker can check the turnover rate of the customer at the store 35. Therefore, for example, when the turnover rate of the store 35 is high, the display confirmer can estimate that the possibility of use is high even if the degree of congestion is somewhat high. Further, for example, when the turnover rate of the store 35 is low, the person who confirms the display can estimate that if the degree of congestion is high, the customer is less likely to leave the store and is less likely to be used.

(Second Embodiment)
In 1st Embodiment, the case where the vacant seat condition of the single store 35 was displayed on the digital signage 33 etc. was shown. In 2nd Embodiment, the case where the present congestion information in a some store is delivered to the smart phone etc. which a customer possesses according to the request | requirement from a customer is shown.

  The store information providing system of the second embodiment has almost the same configuration as that of the first embodiment. About the same component as 1st Embodiment, the description is abbreviate | omitted or simplified by using the same code | symbol.

  FIG. 15 is a sequence diagram illustrating a procedure for distributing current congestion information of each store 35 to the smartphone 41 in the second embodiment. The smartphone 41 has already installed an application (also referred to as a store congestion information distribution application) for distributing information on the congestion status of each store 35. The store mixed information distribution application can access the server 15 via an API (Application Programming Interface) provided by the server 15, and can send and receive data to and from the server 15.

  The processing unit 411 of the smartphone 41 activates a store congestion information distribution application. The user of the smartphone 41 performs an operation of inputting user usage instruction information on the touch panel 413 of the smartphone 41. The processing unit 411 receives input of user usage instruction information from the user via the touch panel 413 (T1).

  The user usage instruction information is information on the user usage of the store 35 desired by the user. For example, when the user wants to perform work at the store 35, the user usage instruction information includes information that instructs the business-oriented store 35. The user usage instruction information may include information on the order of presentation of the stores 35 (sort information) when there are a plurality of stores 35 that match the user usage. For example, the user usage instruction information includes, for example, information in the order of category (user usage): business, sort: low congestion.

  The processing unit 411 transmits user usage instruction information to the Web server 11 via the communication unit 412 (T2). When receiving the user usage instruction information from the smartphone 41, the Web server 11 requests the DB server 12 to search for data (T3). The DB server 12 narrows down the information (store information) of the stores 35 registered in the vacant seat status table 171 (to be described later) using the business for the user as a search key, and one or more stores for which the user is used for business. Data on the degree of congestion in the 35 store information is extracted (T4). The store information is information related to the store 35 and includes, for example, at least one item of information included in the vacant seat status table 171. Then, the DB server 12 rearranges the data on the degree of congestion of one or more stores 35 whose business is a user application in ascending order of the degree of congestion.

  The DB server 12 responds to the Web server 11 with the data of the congestion degree of each store 35, which is narrowed down in the order of the congestion degree as a search key: business (T5). This data may include the store ID of the store 35.

  When the Web server 11 receives data on the degree of congestion of each store 35 from the DB server 12, the Web server 11 converts the data into, for example, JSON (JavaScript (registered trademark) Object Notation) format data and sends it back to the smartphone 41 (T6).

  When the processing unit 411 of the smartphone 41 receives, for example, JSON format data via the communication unit 412, information on each store 35 (for example, a user application image corresponding to a user application, a store name corresponding to a store ID). , The degree of congestion) is displayed on the touch panel 413 so as to be viewable (T7). In addition, the data format of the congestion degree of each store 35 is not limited to the JSON format, but may be another format (for example, an XML format or a CSV format).

  In T4 to T7, it has been described that the congestion degree data is extracted and communicated and displayed. However, at least one store information other than the congestion degree may be extracted and communicated and displayed.

  FIG. 16 is a table showing an example of the vacant seat status of each store 35. The degree of congestion can be expressed as one index indicating the availability of seats. The vacant seat status table 171 includes store ID, GPS coordinates, category (user use), congestion, distribution of objects O, average distance of people, average distance to adjacent seats, distribution of staying time, sound The volume, pitch distribution, and time stamp are registered. For example, if store 35 is store B, store ID: store B, category: business, congestion level: {average: 50%, 1F: 50%, 2F: 50%}, average distance of people: 50 cm, next door The information of the average distance to the seat: 50 cm, the sound volume: 30 dB, and the time stamp: 00:01:00 is registered. The GPS coordinates in FIG. 16 are an example of position information where the store 35 is located.

  The items held in the vacant seat status table 171 may be the same as the items held in the store status table 159. The processing unit 131 may write at least part of the information held in the store status table 159 in the vacant seat status table 171. Further, without preparing the vacant seat status table 171, the store status information of each store 35 is written in the store status table 159 prepared for each store 35, so that the store status of each store (for example, user usage and congestion level). May be recorded.

  FIG. 17 is a diagram illustrating an example of a screen that displays the current congestion level of each store 35 displayed on the smartphone 41. This display is an example of the display of T7 in FIG.

  On the screen GM2 displayed on the touch panel 413, which displays the current congestion level of each store 35, the usage of the user is business, and the status of each store 35 searched in descending order of the congestion level is displayed. Here, it is displayed that the store A is for business use and the degree of congestion is 10%. It is displayed that store B is for business use and the degree of congestion is 50%. It is displayed that the store C is for business use and the degree of congestion is 100%. On this screen, the user of the smartphone 41 can select the user usage by checking a selection item in the category column via the touch panel 413, and the display order of the search results can be selected in the sort column.

  As described above, the store information providing system 5 of the second embodiment further includes the smartphone 41. The DB server 12 includes a storage 122 that stores, in association with a plurality of stores 35, store IDs, congestion degree information, and store user information for each store 35 in association with each other. The Web server 11 receives user usage instruction information for instructing a user usage to be searched. The DB server 12 extracts, from the storage 122, the store ID and the congestion level information associated with the user usage information of the store 35 that matches the user usage specified by the user usage instruction information. The Web server 11 transmits the extracted store ID and store congestion information to the smartphone 41. The smartphone 41 receives the store ID and the store congestion information. The smartphone 41 displays the store ID and the degree of congestion of the store.

  The smartphone 41 is an example of a mobile terminal. The store ID is an example of store identification information. The storage 122 is an example of a storage unit.

  Thereby, the user of the smartphone 41 can check the store 35 corresponding to the user application desired by the user and the congestion degree of the store 35. The smartphone 41 displays, for example, the store 35 corresponding to the user application and the congestion degree of the store 35 in a list, so that the user can easily grasp the desired store 35 and the congestion degree.

(Third embodiment)
In 2nd Embodiment, the case where the present congestion degree of each store 35 was displayed was shown. In 3rd Embodiment, the case where the future congestion information in the some store 35 is delivered to the smart phone 41 according to the request | requirement from a customer is shown.

  Moreover, the store information provision system of 3rd Embodiment has a structure substantially the same as 1st Embodiment. About the same component as 1st Embodiment, the description is abbreviate | omitted or simplified by using the same code | symbol.

  FIG. 18 is a sequence diagram illustrating a procedure for distributing future congestion information of each store 35 to the smartphone 41 in the third embodiment. Assume that the user of the smartphone 41 moves from the starting point I point to the destination J point and wants to know the vacant store at the I point or the J point. The departure location may or may not match the current location of the user of the smartphone 41.

  In the present embodiment, as in the second embodiment, the store congestion information distribution application is already installed in the smartphone 41. The store mixed information distribution application can access the server 15 via the API provided by the server 15 and can send and receive data to and from the server 15.

  The processing unit 411 of the smartphone 41 activates a store congestion information distribution application. The user of the smartphone 41 performs an operation of inputting user usage instruction information on the touch panel 413 of the smartphone 41. The processing unit 411 of the smartphone 41 receives input of user usage instruction information from the user via the touch panel 413 (T11). The user usage instruction information may include category (user usage) information, sorting information, current location and destination information. The user usage instruction information includes, for example, category (user usage): business, sort: in the order of low congestion, location information: information on I point as a departure point and J point as a destination.

  The processing unit 411 transmits user usage instruction information to the Web server 11 via the communication unit 412 (T12). Upon receiving the user usage instruction information from the smartphone 41, the Web server 11 requests the external geographic information service server 7 via the Internet 8 for the GPS coordinates and the travel time from the I point to the J point (T13). . In response to this request, the geographic information service server 7 returns the GPS coordinates of the I point, the GPS coordinates from the J point to the J point, and the movement time T seconds to the Web server 11 (T14).

  The Web server 11 uses the category (user application): business, sort: ascending order of the degree of congestion, location information: the vicinity of the I point as a search key, and the store 35 near the current I point (after 0 second travel time). Then, the DB server 12 is requested to search for data (T15). Further, the Web server 11 uses the category: business, sort: ascending order of the degree of congestion, position information: the vicinity of the J point as a search key, and the DB server 12 predicts the store 35 near the J point after the moving time T seconds. A data search is requested (T15). In addition, the store 35 in the vicinity of the I point or the J point indicates that the store 35 is located within the threshold th6 from the I or J point.

  The DB server 12 narrows down the store information registered in the vacant seat status table 171 using the category (user usage): business, vicinity of the I point as a search key, and the usage of each store 35 near the I point is business. Data on the degree of congestion in the store information is extracted (T16). Then, the DB server 12 rearranges the data on the degree of congestion of each store 35 in the vicinity of the point I for business, in ascending order of the degree of congestion. Further, the DB server 12 requests the analysis server 13 to predict the congestion degree of each business use store 35 in the vicinity of the point J after the movement time T seconds (T17).

  In response to the prediction request from the DB server 12, the analysis server 13 predicts the congestion degree of each business use store 35 in the vicinity of the point J after the movement time T seconds (T18). Details of the prediction of the congestion level by the analysis server 13 will be described later.

  The analysis server 13 returns the prediction result of the degree of congestion of each business use store 35 in the vicinity of the point J to the DB server 12 (T19). The DB server 12 sorts the congestion degree data of each store 35 in the vicinity of the point J in ascending order of congestion based on the prediction result of the congestion degree of each store 35 for business use received from the analysis server 13 (T20). ).

  The DB server 12 narrows down the congestion degree data of each store 35 for business use narrowed down in the order of the low degree of congestion in the vicinity of the current point I, and the order of low congestion in the vicinity of the point J after the movement time T seconds. The data of the congestion degree (future congestion degree) of each store 35 for business use narrowed down in response to the Web server 11 (T21).

  When the Web server 11 receives from the DB server 12 data on the degree of congestion of each store 35 for business use at the present time and after the movement time T seconds (in the future), the Web server 11 converts the data into, for example, JSON format data and returns it to the smartphone 41 (T22).

  When the processing unit 411 of the smartphone 41 receives, for example, JSON format data via the communication unit 412, information on each store 35 (for example, a user application image corresponding to a user application, a store name corresponding to a store ID). , The degree of congestion) is displayed on the touch panel 413 so as to be viewable (T23). The data format of the congestion degree data of each store 35 is not limited to the JSON format, but may be an XML format or a CSV format.

  In T16 to T23, it has been described that the congestion degree data is extracted and communicated and displayed. However, at least one of the store information other than the congestion degree may be extracted and communicated and displayed. The store information includes a queue at the current store 35, a turnover rate of the store 35, and other information, and waits for a cash register until seating at the store 35 at the present or future becomes possible (described later). Information may be included.

  FIG. 19 is a flowchart showing an example of the congestion degree prediction procedure of the analysis server 13 in step T18.

  The processing unit 131 of the analysis server 13 acquires the travel time T seconds obtained by the geographic information service server 7 (S91). The information of the movement time T seconds may be information acquired from the geographic information service server 7 at T14.

  The processing unit 131 acquires the current number of seats based on the chair seating situation table 156 (S92). The number of seats may be the number of chairs C that are in use (the user is present).

  The processing unit 131 acquires the cashier waiting number and the cashier waiting time (S93). The cashier waiting number and the cashier waiting time can be used to reflect the waiting number of cashiers in the degree of congestion. The cashier waiting time here is the current cashier waiting time, and is obtained by calculating the current cashier waiting number × average cashier processing time per person. For example, the processing unit 131 may acquire the number of people waiting in line at the cash register by analyzing an image captured by the camera 34 installed in the vicinity of the cash register. The processing unit 131 may acquire the information on the average cashier processing time per person, for example, by calculating based on the past average cashier processing time value. The actual value of the past average cash register processing time may be held in the DB server 12, for example.

  The processing unit 131 refers to the table seating situation table 157 registered in the DB server 12 and estimates the number of seats left after T seconds based on the past performance value (S94). For example, the processing unit 131 estimates the turnover rate after T seconds in the same day of the week and similar weather, and estimates the number of absences after T seconds based on the estimated turnover rate (S94). In this case, for example, the processing unit 131 may calculate the turnover rate of the store 35 on the basis of the stay time distribution information held in the store status table 159 or the vacant seat status table 171, for example. Moreover, based on the type stamp etc. in which the store information including the stay time distribution is recorded, the day of the week and weather information may be acquired.

The processing unit 131 calculates the congestion degree (future congestion degree) after T seconds according to the equation (1) (S95). That is, the processing unit 131 predicts the degree of congestion after T seconds.
Future congestion level = (Number of seats + Number of people waiting for cash register-Number of seats left) / Number of chairs ...... (1)
This number of seats may be the number of seats obtained in S92. This waiting number may be the number of waiting for the cash register obtained in S93. The number of leaving seats may be the number of leaving seats obtained in S94. In (Equation 1), since the future congestion degree is calculated based on the number of chairs, it is assumed that seating is allowed without considering the table T. In the case where no seating is permitted, the future congestion degree may be calculated based on the table usage rate and the number of tables.

  The processing unit 131 estimates the number of waiting cashiers after T seconds based on the change in the waiting number of cashiers (S95). The number of cashier waiting after T seconds may be, for example, a value obtained by subtracting the number of leaving seats after T seconds obtained in S94 from the current cashier waiting number. In addition, the processing unit 131 adds the predetermined number of people waiting for the cash register every predetermined time or adds the actual value of increase / decrease in the number of cashiers waiting for the same time period in the past to determine the number of cashiers waiting for the cash register after T seconds. It may be estimated.

  Note that the store information of each store 35 transmitted in step T21 of FIG. 18 may include information on the degree of congestion obtained by Expression (1) and the cashier waiting time. Thereafter, the analysis server 13 ends the congestion degree prediction process. Therefore, information on the future congestion degree and cashier waiting time can be transmitted to the smartphone 41 and displayed.

  FIG. 20A is a diagram illustrating an example of a screen GM3 displayed on the smartphone 41 and representing the current congestion level of each store 35 in the vicinity of the point I that is the departure point. In the screen GM3, for example, the current point of time: 0:00: The point of time, the information about the stores A, B, C, which are stores near the point I, and the current cashier waiting time are in ascending order of congestion. , Displayed along with the value of congestion. In FIG. 20A, guidance information such as “immediate guidance” and “waiting time of about 3 minutes” corresponding to the waiting time being 0 minutes, for example, is displayed as an example of information regarding the cashier waiting time.

  FIG. 20B is a diagram showing an example of a screen GM4 displayed on the smartphone 41 and showing the degree of congestion of each future store 35 in the vicinity of the destination J which is the destination. On the screen GM4, for example, information about the future 0:00:00 (that is, prediction), stores D, E, F, which are stores near the point J, and future cashier waiting time are low in congestion. It is displayed with the numerical value of the degree of congestion in order. In FIG. 20B, guidance information such as “immediate guidance” and “waiting time of about 10 minutes” corresponding to the waiting time being 0 minutes is displayed as an example of information regarding the cashier waiting time.

  As described above, the processing unit 411 of the smartphone 41 displays the information regarding the store information retrieval time such as the degree of congestion, the user application, the degree of congestion, and the cashier waiting time on the screens GM3 and GM4 via the touch panel 413. Thereby, the store information provision system 5 can confirm at a glance the present congestion degree and the future congestion degree of the store according to the user use which a customer desires. In addition, the customer can select the store 35 according to the customer's convenience while confirming the information regarding the cashier waiting time, along with the user usage desired by the customer.

  As described above, in the store information providing system 5 of the third embodiment, the Web server 11 may acquire information on the travel time to the store 35. The DB server 12 extracts, from the storage 122, the store ID associated with the user usage information of the store 35 that matches the user usage specified by the user usage instruction information. The analysis server 13 calculates the degree of congestion (for example, the degree of future congestion) after the travel time T seconds of the store 35 for user usage that matches the user usage specified by the user usage instruction information. The Web server 11 transmits information on the store ID and the future congestion level. The smartphone 41 receives and displays the store ID and information on the future congestion degree of the store 35.

  Further, even if the store information providing system 5 is applied to a store 35 that does not have a reservation system for reserving seats in a store, customers who want to use the store 35 are not limited to the degree of congestion, user usage, turnover, etc. Store information.

  Thereby, the store information providing system 5 estimates the degree of congestion of the store 35 not only at the present time but also in the future (future congestion level), and obtains information on the future congestion degree in the store 35 that can be used for the user desired user. Can be provided to the user. Therefore, for example, the user of the smartphone 41 can be used to determine whether to enter the store at the point I as the departure point or to enter the store 35 at the point J as the destination. Further, the store information providing system 5 may present information on the rotation rate of the store 35 after the movement time T seconds together with the congestion rate.

  While various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present disclosure. Understood.

  In the said embodiment, although the bag was mainly illustrated and demonstrated about the thing O, as a recognition object with the cameras 31 and 34, a shop may provide the food and drink which exists on a table. Information on whether the order and offer is only a dish, a drink such as coffee, or both may be used to estimate the in-store situation and the stay time. Furthermore, it may be used to estimate the remaining amount of the provided food and drink and to estimate the situation in the store and the staying time.

  In the above embodiment, the store information providing system 5 allows the analysis server 13 to store the information of various tables registered in the past in the DB server 12 and the information derived from the information of various tables when providing the store information. May be acquired as actual information. The analysis server 13 may provide store information based on past performance information.

  In the above embodiment, the store information providing system 5 may provide store information to the customer in conjunction with the reservation system. For example, in the store information providing system 5, the processing unit 411 of the smartphone 41 acquires reservation information (for example, store ID, store visit time, number of people) registered in the reservation system, and the number of people related to the reservation information is registered in the store visit time. You may add to the number of people waiting. Thereby, the smart phone 41 can estimate the congestion degree and the staying time in consideration of the reservation data included in the reservation system together with the detection data by the plurality of sensors. In addition, reservation data may be used for determination of a user use.

  In the above embodiment, the vacant seat status and the store information may be presented not only on a monitor including the digital signage 33 or a mobile terminal such as the smartphone 41 but also by voice or other presentation methods.

  In the above embodiment, the store information providing system 5 is introduced into the store 35 (for example, a restaurant). However, the store information providing system 5 may be introduced into an entertainment facility such as a game center or a movie theater. Further, the store information providing system 5 may be introduced at one store instead of the plurality of stores 35.

  In the above embodiment, the processor may be physically configured in any manner. Further, if a programmable processor is used, the processing contents can be changed by changing the program, so that the degree of freedom in designing the processor can be increased. The processor may be composed of one semiconductor chip or physically composed of a plurality of semiconductor chips. When configured by a plurality of semiconductor chips, each control of the above embodiment may be realized by a separate semiconductor chip. In this case, it can be considered that a plurality of semiconductor chips constitute one processor. Further, the processor may be configured by a member (capacitor or the like) having a function different from that of the semiconductor chip. Further, one semiconductor chip may be configured so as to realize the functions of the processor and other functions. In addition, a plurality of processors may be configured by one processor.

  Further, according to the present disclosure, a program that realizes at least a part of the functions of the store information providing system 5 of the above embodiment is supplied to a device via a network or various storage media, and a computer in the device reads and executes the program. The program is also applicable.

  The present disclosure is useful for a store information providing system, a store information providing method, a program, and the like that can provide information as to whether the in-store situation matches a customer's request or preference.

DESCRIPTION OF SYMBOLS 5 Store information provision system 7 Geographic information service server 8 Internet 9 Center 10 Center network apparatus 11 Web server 12 Database (DB) server 13 Analysis server 20 Store network apparatus 31 Camera 32 Microphone 33 Digital signage 34 Camera 35 Store 41 Smartphone 42 Viewer 100 Determination sheet 154 Person information table 155 Object information table 157,157A Table seating situation table 158 Person staying time table 159 Store situation table 161 Person attribute information table 162 Object estimation table 163 Sound attribute table 164 Category type table 171 Vacancy situation table

Claims (16)

A store information providing system comprising a plurality of sensors provided in a store and a server,
The plurality of sensors are:
The detection data is acquired by sensing,
Sending the detection data,
The server
Receiving the detection data;
Recognizing the position of at least one of a person and an object located in the store based on the detection data,
Based on the position of at least one of the person and the object, the degree of congestion of the store is calculated,
Based on the degree of congestion and the detection data, the user determines a user application for using the store.
Store information provision system. The server
Obtain seat placement information in the store,
Based on the arrangement information of the seat and the position of at least one of the person and the object, the degree of congestion of the store is calculated.
The store information providing system according to claim 1. A display device,
The server transmits information on the user use of the store,
The display device
Receiving information on the user usage of the store,
Displaying information about the user usage of the store;
The store information providing system according to claim 2. The server
If the degree of congestion is greater than or equal to a threshold, determine the user usage of the store,
If the degree of congestion is less than the threshold, do not determine the user application of the store,
The store information providing system according to claim 3. The server
Receiving the detection data at a plurality of time points;
Recognizing the position of at least one of a person and an object located in the store based on the detection data at the plurality of points in time;
At the plurality of times, based on the arrangement information of the seat and the position of at least one of the person and the object, the congestion degree of the store is calculated,
At the plurality of points in time, based on the degree of congestion and the detection data, determine a user application of the store,
At the plurality of points in time, information on the user usage of the determined store is transmitted,
The display device
At the plurality of points in time, information on the user usage of the store is received,
Updating the display of the user usage information of the store at a later time in the plurality of times,
The store information provision system according to claim 3 or 4. The server
Recognizing the seating positions of a plurality of people in the store based on the arrangement information of the seats and the position of at least one of the person and the object,
Calculating the interval between the seating positions of the plurality of people,
Based on the interval between the seating positions of the plurality of people, determine the user application of the store,
The store information provision system of any one of Claims 3-5. The sensor is a microphone;
The server determines a user application of the store based on a signal level of an audio signal collected by the microphone.
The store information provision system of any one of Claims 3-6. The sensor is a microphone;
The server
Determine the pitch of a person's voice included in the audio signal collected by the microphone;
Based on the voice level of the person, determine the user application of the store,
The store information provision system of any one of Claims 3-7. The server
Recognizing an object located in the store,
Based on the object, the user usage of the store is determined.
The store information provision system of any one of Claims 3-8. The server transmits information on the calculated congestion degree of the store,
The display device
Receiving information on the congestion of the store,
Displaying information on the congestion of the store;
The store information provision system of any one of Claims 3-9. The server
Based on detection data obtained at a plurality of timings by the plurality of sensors, calculating a stay time in the store of at least one of a person and an object located in the store,
Transmitting the calculated stay time information;
The display device
Receiving the information of the stay time,
Displaying information on the stay time;
The store information provision system of any one of Claims 3-10. A portable terminal,
There are a plurality of the stores,
The server
For each store, the store includes a storage unit that holds the identification information of the store, information on the degree of congestion of the store, and information on the user usage of the store,
Receive usage instruction information for instructing user usage to be searched,
The store identification information and the congestion level information of the store associated with the user usage information of the store that matches the user usage specified in the usage instruction information is extracted from the storage unit,
Send the extracted identification information of the store and the information of the congestion degree of the store,
The portable terminal is
Receiving the store identification information and the store congestion information,
Displaying the store identification information and the congestion information of the store;
The store information provision system of any one of Claims 3-11. The server
Get information on travel time to the store,
Extracting the store identification information associated with the user usage information of the store that matches the user usage indicated by the usage instruction information from the storage unit;
Calculating the degree of congestion after elapse of the travel time of the store for the user usage that matches the user usage specified in the usage instruction information;
Send information of the store identification information and the degree of congestion after the travel time of the store has passed,
The portable terminal is
Receiving congestion information after the store's identification information and the travel time of the store has passed,
Displaying the identification information of the store and the information of the degree of congestion after the travel time of the store has passed,
The store information providing system according to claim 12. A server that provides information about a store,
A receiving unit, a processing unit,
The receiving unit acquires detection data detected by a plurality of sensors provided in the store,
The processor is
Recognizing the position of at least one of a person and an object located in the store based on the detection data,
Based on the position of at least one of the person and the object, the degree of congestion of the store is calculated,
Based on the degree of congestion and the detection data, determine the user application of the store,
server. A store information providing method in a server that provides information about stores,
Obtaining detection data detected by a plurality of sensors provided in the store;
Recognizing the position of at least one of a person and an object located in the store based on the detection data,
Based on the position of at least one of the person and the object, the degree of congestion of the store is calculated,
Based on the degree of congestion and the detection data, determine the user application of the store,
How to provide store information.   A program for causing a computer to execute the store information providing method according to claim 15.

Images