WO2018146999A1

WO2018146999A1 - Seating display device, seating display program, and seating display system

Info

Publication number: WO2018146999A1
Application number: PCT/JP2018/000768
Authority: WO
Inventors: 剛進河野
Original assignee: 株式会社バカン
Priority date: 2017-02-08
Filing date: 2018-01-15
Publication date: 2018-08-16
Also published as: JP6157044B1; JP2018128859A

Abstract

The purpose of the present invention is to reduce a load involved in store seating map creation and to simplify operation management of a system relating to a seating display. A seating display device according to the present invention, which displays a seating map denoting a layout of chairs within a store, comprises: a receiving means which receives images from an image capture device which continuously captures images; an image processing means which detects placement positions of tables and chairs in a store on the basis of the images; and a generating means which generates a seating map on the basis of the placement positions of the tables and chairs. During a given business period, the image capture device continuously captures images including persons. On the basis of the images that are captured during the given business period, the image processing means estimates a site for which a repeated presence or absence of a plate-shaped object is recognized at a prescribed frequency for a given time or longer to be a table placement position.

Description

Seat display device, seat display program, and seat display system

The present invention relates to a seat display device, a seat display program, and a seat display system.

Conventionally, a seat display system is known. As a familiar example, in the case of a seat display system in a commercial facility such as a movie theater, a theater, or a concert hall, a user accesses a seat map (seat chart) from a website in advance and visually displays it on the seat map. A seat reservation can be made by designating a seat to be used.

In addition, employee seat maps created based on the office layout are often used in corporate offices, etc., and on the intranet, the employee's seat position, extension number, attendance status, etc. are displayed visually. Some will do.

In such a case, the administrator side needs to create a seat map according to the actual seat arrangement in advance, but the seat map for movie theaters etc. should be created only once because the seat is fixed, When the seat layout is frequently changed as in an office or the like, there is a trouble that the seat map must be recreated each time.

As a technology related to this, for example, Patent Document 1 describes an invention that facilitates the creation of a floor map for displaying the presence status to a user in a free address style office and the like, and facilitates the operation of the seat display system. Has been.

JP 2011-170725 A

However, since the invention described in Patent Document 1 is an invention that assumes an office to the last, it is necessary to use a network relay device, and it is difficult to apply depending on the environment.

In other words, in the case of restaurants among commercial facilities, owners create seat maps according to actual seating arrangements using authoring tools, etc., but seat layouts frequently change even during business hours at restaurants. As a result, busy restaurants are not able to keep up with the time and effort to create a seat map. In the case of restaurants, not only seat layout changes due to events, decoration changes, etc., but also small seat layout changes such as changing two two-person tables to one four-person table are often performed. It is where

The present invention has been proposed in view of the above points, and an object of one aspect of the present invention is to reduce a burden required for creating a seat map of a store and to facilitate operation management of a system related to seat display.

In order to solve the above-described problems, a seat display device according to the present invention is a seat display device that displays a seat map indicating an in-store layout of a table and a chair arranged in a store, and the imaging range of the table and the chair is displayed. An image that is installed as a receiver and that receives the image from an imaging device that continuously captures an image including the table and chair, and an image that detects an arrangement position of the table and chair in the store based on the image Processing means and generation means for generating the seat map based on the arrangement position of the table and chair, and the imaging device continuously captures the image including a person during a certain business period. The image processing means is configured to detect the presence or absence of dish-shaped objects at a predetermined frequency for a certain time or more based on the images captured during the certain business period. When detecting the location of repeated standing is recognized, estimates the 該場 plant and the arrangement position of the table.

In order to solve the above problems, a seat display program according to the present invention provides an imaging range of the table and the chair on a seat display device that displays a seat map indicating the in-store layout of the table and the chair arranged in the store. An image that is installed as a receiver and that receives the image from an imaging device that continuously captures an image including the table and chair, and an image that detects an arrangement position of the table and chair in the store based on the image Based on an arrangement position of the processing means and the table and the chair, the functioning as a generating means for generating the seat map, the imaging device continuously captures the image including a person during a certain business period, The image processing means, based on the image captured during the certain business period, of the dish-shaped object at a predetermined frequency for a certain time or more. If repetition of standing and absence detects the location to be recognized, to estimate the 該場 plant and the arrangement position of the table.

In order to solve the above problems, a seat display system according to the present invention includes an imaging device and a seat display device, and displays a seat map indicating a store layout of tables and chairs arranged in the store. The seat display device includes the table and the chair as an imaging range, and receives the image from the imaging device that continuously captures the image including the table and the chair; and the image The image processing means for detecting the arrangement position of the table and chair in the store, and the generation means for generating the seat map based on the arrangement position of the table and chair, and the imaging device Continuously captures the image including a person during a certain business period, and the image processing means captures the image captured during the certain business period. Based on a certain time or more in a predetermined frequency when it detects the location of repeated presence and absence of an object dish shape is recognized, estimates the 該場 plant and the arrangement position of the table.

According to the embodiment of the present invention, it is possible to reduce a burden required for creating a seat map of a store, and to facilitate operation management of a system related to seat display.

It is a figure which shows the structural example of the seat display system which concerns on this embodiment. It is a figure which shows the hardware structural example of the seat display server which concerns on this embodiment. It is a figure which shows the software structural example of the seat display system which concerns on this embodiment. It is a figure which shows the example of data of store management DB which concerns on this embodiment. It is a figure which shows the example of data of image management DB which concerns on this embodiment. It is a figure which shows the example of data of the seat map management DB which concerns on this embodiment. It is a figure which shows the shop installation structural example of the seat display system which concerns on this embodiment. It is a figure which shows an example of the new registration screen of the shop information which concerns on this embodiment. It is a figure which shows an example of the seat map display screen 1 which concerns on this embodiment. It is a figure which shows an example of the seat map display screen 2 which concerns on this embodiment. It is a flowchart figure which shows the seat map production | generation process of the seat display server which concerns on this embodiment.

Embodiments of the present invention will be described in detail with reference to the drawings. First, the “seat map” in the present embodiment is a seat layout diagram and a seat arrangement diagram in a store. Also called seating chart. The “seat map” includes not only chairs but also tables and chairs.

<System configuration>
(Network configuration)
FIG. 1 is a diagram illustrating a configuration example of a seat display system according to the present embodiment. A seat display system 100 in FIG. 1 includes a seat display server 10, a relay device 20, a camera 21, and a reference terminal 30, which are connected via a network 50.

The seat display server 10 is a device that generates a seat map (seat layout map) based on images (including still images and moving images) captured by the camera 21. The generated seat map is displayed on the display of the reference terminal 30. Note that one seat display server 10 is not necessarily installed for each store, and at least one seat display server 10 is arranged on the network 50.

The relay device 20 is a device that relays between the camera 21 and the seat display server 10. Images installed by the stores and captured by the camera 21 are sequentially transmitted (transferred) to the seat display server 10.

The camera 21 is an imaging device for capturing an image including a table and a chair arranged in a store. In this embodiment, the camera 21 is installed so that all the tables and chairs constituting the seat layout of the store are within the imaging range.

Therefore, in order to fit all the tables and chairs within the imaging range, a plurality of cameras 21 are installed depending on the store area. However, even if a plurality of cameras are not necessarily installed, for example, when the camera 21 is an omnidirectional drive type, one camera 21 may cover all tables and chairs in the imaging range.

When the camera 21 itself has the function of the relay device 20, the camera 21 can directly transmit an image to the seat display server 10 without going through the relay device 20.

The reference terminal 30 is a computer terminal used when a person in charge of the store accesses the seat display server 10 and refers to the seat map.

The network 50 is a communication network including wired and wireless. The network 50 includes, for example, the Internet, a public line network, WiFi (registered trademark), and the like.

(Hardware configuration)
FIG. 2 is a diagram illustrating a hardware configuration example of the seat display server according to the present embodiment. As shown in FIG. 2, the seat display server 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an HDD (Hard Disk Drive) 14, and a communication device 15. Have

The CPU 11 executes various programs and performs arithmetic processing. The ROM 12 stores a program necessary for startup. The RAM 13 is a work area for temporarily storing processing performed by the CPU 11 and storing data. The HDD 14 stores various data and programs. The communication device 15 communicates with other devices via the network 50.

(Software configuration)
FIG. 3 is a diagram illustrating a software configuration example of the seat display system according to the present embodiment. The seat display system 100 illustrated in FIG. 3 includes an imaging unit 101, an image processing unit 102, a generation unit 103, and a storage unit 104 as main functional units.

The imaging unit 101 continuously captures an image including a table and a chair by a camera 21 (imaging device) installed with a table and a chair arranged in a restaurant as an imaging range.

The image processing unit 102 detects the arrangement position of the table and the chair in the restaurant based on the image captured by the imaging unit 101.

The generation unit 103 generates a seat map based on the table and chair arrangement positions detected by the image processing unit 102.

The storage unit 104 has a function of storing various DBs (Data Base).

The image processing unit 102, the generation unit 103, and the storage unit 104 are realized by a computer program that is executed on hardware resources such as a CPU, a ROM, and a RAM of a computer that configures the seat display server 10. . These functional units may be read as “means”, “module”, “unit”, or “circuit”. Further, the imaging unit 101 may be realized by the camera 21 and provided on the seat display system 100.

(DB)
FIG. 4 is a diagram illustrating a data example of the store management DB according to the present embodiment.

The store management DB has data items such as “store ID”, “store name”, “number of seats”, “number of tables”, “ON / OFF”, “generation interval”, for example.

“Store ID” indicates a unique identifier assigned to each store.

“Store name” indicates the name of the store.

“Number of seats” indicates the total number of seats in the store.

“Number of tables” indicates the total number of tables in the store.

“ON / OFF” indicates a setting value for setting whether to enable (ON) or disable (OFF) seat map generation of the seat display system 100.

“Generation interval” indicates a setting value for setting a time interval for generating a seat map when seat map generation is enabled (ON). For example, when it is 15, the seat display server 10 generates (updates) a seat map at a timing of once every 15 minutes. Of course, if there is a generation command at any time by the manager, the seat map is generated (updated) regardless of the timing of the “generation interval”.

FIG. 5 is a diagram showing an example of data in the image management DB according to the present embodiment.

The image management DB includes data items such as “store ID”, “camera identifier”, “shooting date”, and “shooting time” in order to identify and manage each image received from each store.

“Store ID” is used to identify an image received from each store.

“Camera identifier” indicates the identifier of the camera 21 that captured the image. Since a plurality of cameras 21 are installed in some stores, it is identified which camera 21 captured the image.

“Shooting date” and “Shooting time” indicate the shooting date and shooting time of the image. Images (still images) are continuously captured, but the frequency of capturing can be set as appropriate on the seat display server 10 side or the camera 21 side. However, since the seat map is generated at every “generation interval”, the image capturing frequency is set to be at least shorter than the “generation interval”. When the camera 21 captures a moving image, it is possible to continue capturing images (moving images) while seat map generation is enabled (ON).

FIG. 6 is a diagram showing a data example of the seat map management DB according to the present embodiment.

The seat map management DB has data items such as “store ID”, “seat map”, “generation date”, and “generation time” in order to manage the latest seat map generated for each store.

“Store ID” is used to identify the seat map of each store.

“Seat map” indicates newly generated seat map data (for example, “xxx.jpg”).

“Generation date” and “generation time” indicate the generation date and generation time of the seat map. Since new seat map data (for example, “xxx.jpg”) is generated for each “generation interval”, the “seat map” with the latest “generation date” and “generation time” is referred to. The latest seat map that should be.

<Example of installation>
FIG. 7 is a diagram illustrating a store installation configuration example of the seat display system according to the present embodiment. As shown in FIG. 7, in the store A, the relay device 20, the camera 21, and the reference terminal 30 (not shown) are installed.

First, on the ceiling or wall of the store, the camera 21a and the camera 21b are installed so that all the seats (table t and chair c) of the store A are within the imaging range. In the figure, the imaging range a indicates the imaging range of the camera 21a, the imaging range b indicates the imaging range of the camera 21b, and all the seats arranged in the store A are covered by the imaging range a and the imaging range b. ing.

Note that when the camera 21 is installed, the table t and the chair c are not limited to movable ones, and a fixed table or chair such as a counter table is also included in the imaging range.

The relay device 20 is connected to the camera 21a and the camera 21b via a network including a USB cable, wired or wireless. The relay device 20 may be installed at an arbitrary location in the store A.

<Screen>
FIG. 8 is a diagram showing an example of a store information new registration screen according to the present embodiment. First, the person in charge of the store accesses the new registration screen 800 of the seat display server 10 using the reference terminal 30 or the like.

URL 801 is a column for inputting location information (for example, a domain, an IP address, etc.) of the seat display server 10.

The store ID 802 indicates a unique identifier assigned to each store. The store ID given in advance from the seat display server 10 is input or is automatically issued to the store at the time of registration.

Store name 803 is a column for inputting the name of the store.

The number of seats 804 is a column for inputting the total number of seats in the store. The input of the number of seats is arbitrary, but if there is an input, it is used to improve the accuracy of creating a seat map. However, it is not necessary to input in the case of a store where the number of seats is changed frequently.

The table number 805 is a column for inputting the total number of tables in the store. The input of the number of tables is arbitrary, but if there is an input, it is used to improve the accuracy of seat map creation. However, in the case of a store where the table is frequently changed, it is not necessary to input.

Seat map generation 806 indicates a setting field for setting whether the seat map generation is valid (ON) or invalid (OFF). When the person in charge at the store wants to manually generate a seat map, it is set to invalid.

The generation interval 807 indicates a setting field for setting a time interval for generating a seat map when the seat map generation 606 is valid (ON). Note that as the generation interval is shorter, the seat map is generated at a pace near real time, while the stored image from the camera 21 used for generating the seat map is reduced, which may lead to a decrease in the accuracy of the seat map (trade-off). connection of). Therefore, it is desirable that the seat display system 100 has a default recommended value that ensures a certain level of accuracy in advance.

Registration 810 is a registration button for registering input information.

As described above, the input information input on the new registration screen 800 is stored in the store management DB (see FIG. 4).

FIG. 9 is a diagram showing an example of the seat map display screen 1 according to the present embodiment. The person in charge of the store accesses the seat map display screen of the seat display server 10 using the reference terminal 30 or the like.

As described above, the seat display server 10 generates a seat map based on an actual seat layout for each set generation interval. That is, when the seat display server 10 receives a reference request from the reference terminal 30 or the like, the seat display server 10 acquires the latest “seat map” 902 from the seat map management DB (FIG. 6) and displays it on the seat map display screen 900. .

FIG. 10 is a diagram showing an example of the seat map display screen 2 according to the present embodiment.

10 that the seat layout shown in FIG. 10 is partially changed as compared with the seat layout shown in FIG. In this modified example, since eight group customers come to the store, the seat layout is performed so that the tables for four people are joined together to form a table for eight people.

The seat display server 10 generates a seat map based on the actual changed seat layout at each set generation interval. When receiving a reference request from the reference terminal 30 or the like, the seat display server 10 acquires the “seat map” 1002 after the latest change from the seat map management DB (FIG. 6), and displays this on the seat map display screen 1000. To do.

<Seat map generation process>
As described above, the camera 21 sequentially transmits an image including a table and a chair arranged in the store to the seat display server 10, and the seat display server 10 is based on the image (accumulated image) received from the camera 21. To generate a seat map.

Here, when generating the seat map, it is possible to recognize the positions of the table and the chair from the image received from the camera 21 by using a general object detection algorithm (also referred to as a general object recognition algorithm) by DeepDLearning. .

Conventionally, various methods have been proposed for such an object detection algorithm, but taking into account that a seat map is generated at a pace close to real time, in this embodiment, the processing speed is particularly excellent. For example, an algorithm such as YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector) is applied.
-Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." ArXiv preprint arXiv: 1506.02640 (2015)
-Liu, Wei, et al. "SSD: Single Shot MultiBox Detector." ArXiv preprint arXiv: 1512.02325 (2015)
In addition, since the seat map of “restaurant” is generated in the present embodiment, the feature amount of the general-purpose type table and chair often used in the restaurant is held in advance, and the feature amount, It is also possible to compare the feature values of the table and chair of the store and identify the table and chair of the store from their appearance (shape, size, color, etc.) based on the degree of matching of the feature values. .

In addition, in the object detection process, for example, by taking into account the circumstances specific to store seats as described below, it is possible to further improve seat map generation accuracy.
・ Assuming that the table and chair in the store are always arranged in pairs (sets), if the table position is specified, a chair will be located in the vicinity, and the chair position will be specified in the same way. If so, it is estimated that a table exists and is arranged in the vicinity thereof.
・ Reliability is increased by comparing whether the number of seats and the number of tables analyzed based on the images match the “number of seats” and the “number of tables” (FIG. 6) input by the person in charge of the store. .

In addition, taking a certain business period (for example, about half a day to about one day), a considerable amount of images can be accumulated by operating imaging of the camera 21 during that period, and a seat map can be generated based on the accumulated images. it can. When a seat map is generated based on images that are actually open, images that are actually seated by people (customers) and images taken on the table are also used as analysis materials for image processing analysis. Therefore, it is possible to improve the generation accuracy of the seat map.
-Recognize a place where the presence of a person is recognized at a fixed frequency for a certain period of time (ie, a place where a person is seated) using a heat map, etc., and a chair is placed in such a place. It is estimated that it is a place. This is particularly effective when the chair is not completely reflected in the image because the chair is hidden behind a shadow or is a blind spot of the camera 21.
・ Conversely, although the presence of a person is recognized at a predetermined frequency, a place where a person does not stay fixed (ie, a passage) that is fluid and fixed for a certain period of time is identified and a table or chair is placed in such a place. Exclude from candidate locations.
-It is estimated that a table is placed where an object (for example, a dish shape) corresponding to actual cooking, serving, etc. is repeatedly placed or lowered at a predetermined frequency for a certain period of time.

In addition, the seat display server 10 accumulates the feature quantity of the table and chair of the store once specified as learning data. As a result, when the seat layout changes afterwards, the difference between the past image and the feature amount is compared to quickly and accurately detect the presence or absence of the occurrence of the movement arrangement and the table and chair moved to the new location. This is useful for generating (updating) the seat map.

<Information processing>
FIG. 11 is a flowchart showing seat map generation processing of the seat display server according to the present embodiment.

S1: The seat display server 10 determines whether an image captured by the camera 21 from the relay device 20 has been received. The image includes “store ID”, “camera identifier”, “shooting date”, “shooting time”, and the like as attribute information (FIG. 5).

S2: When the seat display server 10 receives the image, the seat display server 10 stores the received image together with the attribute information in the image management DB.

S3: The seat display server 10 determines whether or not the timing for generating the seat map has arrived. The seat display server 10 determines based on the timing according to the “generation interval” of the store management DB (FIG. 4). If there is a generation command at any time by the administrator, it is determined that the timing for generating the seat map has arrived regardless of the timing of the “generation interval”.

S4: The seat display server 10 generates a seat map based on the images (accumulated images) stored in the image management DB. As described above, the seat display server 10 receives an image captured and stored by the imaging unit 101, detects the arrangement position of the table and the chair in the restaurant by using the general object detection algorithm, and the restaurant An image of the table and chair is placed on the floor map where the table and chair are detected, and a seat map is generated.

S5: The seat display server 10 stores the generated seat map together with the attribute information in the seat map management DB. The seat map includes “store ID”, “seat map” (seat map data), “generation date”, “generation time” and the like as attribute information (FIG. 6).

<Summary>
The seat display system 100 according to the present embodiment detects the arrangement positions of the table and the chair based on images continuously captured by the camera 21 installed in the store, and automatically generates a seat map.

Thus, for example, in a system such as a seat display system, a seat reservation system, or a vacant seat management system, a seat map can be automatically generated in real time. Even in the case of a change, it is possible to update the seat map without trouble.

Also, in the case of restaurants, as seat layout changes are repeated while accumulating images that have been open for a certain period of time, it becomes possible to compare the seat arrangement status of the seat map with the sales at that time. It can also be used to improve management, such as maximizing sales by adopting the seat arrangement that maximizes sales.

As described above, according to the seat display system 100 according to the present embodiment, it is possible to reduce the burden required to create a store seat map and to facilitate the operation management of the system related to seat display.

It should be noted that although the present invention has been described by way of preferred embodiments of the present invention with specific examples, these specific examples may be used without departing from the broad spirit and scope of the present invention as defined in the claims. Obviously, various modifications and changes can be made to the examples. In other words, the present invention should not be construed as being limited by the details of the specific examples and the accompanying drawings.

DESCRIPTION OF SYMBOLS 10 Seat display server 20 Relay apparatus 21 Camera 30 Reference terminal 50 Network 100 Seat display system 101 Imaging part 102 Image processing part 103 Generation part 104 Storage part

Claims

A seat display device for displaying a seat map showing a layout in a store of tables and chairs arranged in a store,
Receiving means for receiving the image from an imaging device that is installed as an imaging range of the table and chair and continuously captures an image including the table and chair;
Image processing means for detecting the arrangement position of the table and chair in the store based on the image;
Generating means for generating the seat map based on the arrangement position of the table and chair;
Have
The imaging device continuously captures the image including a person during a certain business period,
When the image processing means detects a place where the presence and absence of a dish-shaped object are recognized repeatedly based on the image captured during the certain business period for a certain period of time or more, Estimating the location as the location of the table;
A seat display device.
When the arrangement position of the table is detected, the image processing means estimates the arrangement position of the chair near the arrangement position, and when the arrangement position of the chair is detected, the image processing means detects the arrangement position of the table. Estimating the placement position;
The seat display device according to claim 1.
When the image processing means detects a place where the presence of a person is recognized at a predetermined frequency for a certain time or more based on the image taken during the certain business period, the position is set as the arrangement position of the chair. To estimate,
The seat display device according to claim 1 or 2.
If the image processing means detects a place where the presence of a person is recognized at a predetermined frequency and the presence of a person is not recognized for a certain period of time or more based on the images taken during the fixed business period, Excluding the table or chair from the arrangement position;
The seat display device according to any one of claims 1 to 3.
In the seat display device that displays a seat map showing the layout of the tables and chairs placed in the store in the store,
Receiving means for receiving the image from an imaging device that is installed as an imaging range of the table and chair and continuously captures an image including the table and chair;
Image processing means for detecting the arrangement position of the table and chair in the store based on the image;
Based on the arrangement position of the table and chair, function as a generating means for generating the seat map,
The imaging device continuously captures the image including a person during a certain business period,
When the image processing means detects a place where the presence and absence of a dish-shaped object are recognized repeatedly based on the image captured during the certain business period for a certain period of time or more, A seat display program for estimating the place as an arrangement position of the table.
A seat display system that includes an imaging device and a seat display device, and displays a seat map indicating a layout in a store of tables and chairs arranged in a store,
The seat display device
Receiving means for receiving the image from the imaging device, wherein the table and the chair are installed as an imaging range, and the image including the table and the chair is continuously captured;
Image processing means for detecting the arrangement position of the table and chair in the store based on the image;
Generating means for generating the seat map based on the arrangement position of the table and chair;
Have
The imaging device continuously captures the image including a person during a certain business period,
When the image processing means detects a place where the presence and absence of a dish-shaped object are recognized repeatedly based on the image captured during the certain business period for a certain period of time or more, Estimating the location as the location of the table;
Seat display system characterized by