JP6648094B2 - Seat information processing system, seat information acquisition device and program, and seat information providing device and program - Google Patents

Description

  The present invention relates to a seat information processing system, a seat information acquisition device and a program, and a seat information providing device and a program, and involves, for example, seating at a restaurant (café or other restaurant or a library). The present invention can be applied to a system for processing seat information in a facility.

  2. Description of the Related Art Conventionally, there has been known a technology of recognizing a vacancy state of a seat in a restaurant or the like and generating and providing vacancy information (for example, providing it on the Internet).

  As a conventional technique for providing free space information, for example, there is a technique disclosed in Patent Document 1.

  Patent Literature 1 discloses a technique in which information indicating a place where a user is absent in a place where a user in a facility is available is provided as empty information. In the technology described in Patent Document 1, position information of each seat is registered in advance in a restaurant or the like, and a person (customer sensor) using a human sensor (human sensor using a temperature detection sensor) at each seat position is used. ) Is recognized, and vacant information is generated as a vacant seat for which a person (customer) cannot be recognized.

JP-A-2006-177775

  However, in the conventional technology including the technology described in Patent Literature 1, it is necessary to input information on the details of the layout of the restaurant seats on the premise of acquiring the vacant information. There is a problem that convenience is low because information must be input.

  Further, the technology described in Patent Document 1 requires a dedicated sensor (human sensor) for recognizing the seating status of each seat, so that there is a problem that the introduction cost is high.

  In view of the above problems, a seat information processing system, a seat information obtaining device and a program, and a seat information providing device and a program that realize highly convenient processing of seat information at low cost are desired. .

A seat information processing system according to a first aspect of the present invention includes: (1) an imaging unit for imaging an inside of a facility where one or a plurality of seats are arranged; and (2) a human object is detected from an image captured by the imaging unit. A seat that performs a human object detection process and obtains the number of seat users indicating the number of seats in the facility based on the result of the human object detection process without using information on the position of the seat in the facility. Information acquisition means, (3) installed seat number holding means for storing information on the installed seat number in the facility, and (4) the installed seat number and the seat number without using information on the positions of the seats in the facility. Seat information providing means for acquiring information on vacant seats in the facility as seat information based on the number of seat users, and providing the acquired seat information to an informed party; (5) the seat information acquiring means Is obtained by the human object detection process. From the detected human objects, perform a occupant detection process of detecting a occupant seated in a seat based on the movement of each human object, and use the number of the occupants detected by the occupant detection process in the seat use. It is acquired as the number of persons.

The seat information acquisition device according to the second aspect of the present invention is: (1) performing a human object detection process of detecting a human object from an image captured by an imaging unit that captures an inside of a facility where one or a plurality of seats are arranged; Seat information acquisition means for acquiring the number of seat users indicating the number of seats in the facility without using information on the positions of the seats in the facility; and (2) information on the number of seats installed in the facility. Means for holding the number of installed seats to be held; and (3) acquiring information on vacant seats in the facility based on the number of seats installed in the facility and the number of seat users without using information on the positions of seats in the facility. Transmitting means for transmitting the information including the number of seat users to an external seat information providing apparatus for providing the information to the informed person; and (4) the seat information obtaining means includes: From the human object detected by the detection process, A occupant detection process for detecting a occupant sitting in a seat based on the movement of each person object is performed, and the number of the occupants detected by the occupant detection process is obtained as the number of seat users. .

A seat information acquisition program according to a third aspect of the present invention includes: (1) a human object detection process for detecting a human object from an image captured by an imaging unit that captures an image of a facility where one or a plurality of seats are arranged; Performing, without using information on the position of the seats in the facility, based on the result of the human object detection process, seat information acquisition means for acquiring the number of seat users indicating the number of seats in the facility, (2) means for holding the number of seats installed in the facility, and (3) means for storing the number of seats in the facility and the user of the seats without using information on the positions of the seats in the facility. A function of transmitting information including the number of seat users to an external seat information providing device that obtains information on vacant seats in the facility based on the number of seats and provides the information to the information provider; 4) Hands for obtaining the seat information The tier performs a occupant detection process of detecting a occupant sitting in a seat based on the movement of each human object from the human objects detected by the human object detection process, and detects the occupant by the occupant detection process. The number of seated persons is acquired as the number of seat users.

The seat information providing device according to a fourth aspect of the present invention holds the number of seat users acquired by the seat information acquiring device of the second aspect of the present invention , further holds the number of seats installed in the facility, A seat that obtains information on vacant seats in the facility as seat information based on the number of installed seats and the number of seat users without using information on the position of seats, and provides the obtained seat information to the information provider. It is characterized by having seat information providing means for providing information-provided seat information of the facility or information based on the seat information of the facility to the information-provided person.

According to a fifth aspect of the present invention, there is provided a seat information providing program which stores a computer, the number of seat users acquired by the seat information acquiring apparatus of the second invention , the number of seat users, The number of seats in the facility is maintained, and information on vacant seats in the facility is acquired as seat information based on the number of seats and the number of seat users, without using information on the positions of the seats in the facility, and acquired. A seat information providing program for functioning as seat information providing means for providing the provided information to the information provider.

  According to the present invention, highly convenient processing of seat information can be realized at low cost.

FIG. 1 is a block diagram illustrating an overall configuration of a seat information processing system according to an embodiment. FIG. 2 is a block diagram illustrating a functional configuration of a local computer according to the embodiment. FIG. 2 is a block diagram illustrating a functional configuration of a DB server according to the embodiment. It is explanatory drawing shown about the structural example of the store information DB which concerns on embodiment. FIG. 2 is a block diagram illustrating a functional configuration of a web application server according to the embodiment. FIG. 2 is a block diagram illustrating a functional configuration of a client terminal according to the embodiment. It is a top view inside a store concerning an embodiment. It is a figure (perspective view) when the inside of a store concerning an embodiment is seen from a camera. 5 is an example (part 1) of a video image captured by the camera according to the embodiment. 9 is an example (part 2) of a video image captured by the camera according to the embodiment. FIG. 9 is an explanatory diagram illustrating an example of a circumscribed rectangle cut out from an image by the object recognition unit according to the embodiment. FIG. 10 is an explanatory diagram illustrating an example when the object recognition unit according to the embodiment obtains a motion amount in a human object tracking process. It is explanatory drawing shown about the example at the time of the object recognition part which concerns on embodiment at the time of calculating | requiring a movement amount by seat user object detection processing. It is explanatory drawing shown about the example of a structure of the store selection screen (Web screen) which the Web application server which concerns on embodiment provides. It is explanatory drawing shown about the structural example of the store information provision screen (Web screen) which the Web application server which concerns on embodiment provides. It is explanatory drawing shown about the structural example of the store selection screen (Web screen) which the Web application server which concerns on the modification of embodiment provides.

(A) Principal Embodiment Hereinafter, an embodiment of a seat information processing system, a seat information acquisition device and a program, and a seat information providing device and a program according to the present invention will be described in detail with reference to the drawings.

(A-1) Configuration of Embodiment FIG. 1 is a block diagram showing the overall configuration of a seat information processing system 1 of this embodiment.

  First, an overview of the entire seat information processing system 1 will be described.

  The seat information processing system 1 is a system that processes information (hereinafter, referred to as “seat information”) relating to seats of facilities such as one or more restaurants.

  In this embodiment, the seat information processing system 1 is described as a system that processes seat information of N (N is an arbitrary integer) stores (first store to N-th store).

  In the seat information processing system 1, the cameras 10 (10-1 to 10-N) for imaging the inside of each store (first to N-th stores), seat information in the store, and the like are processed. Local computers 20 (20-1 to 20-N) are arranged.

  Further, in the seat information processing system 1, information including seat information of each store is held from the local computer 20 (20-1 to 20-N) of each store, and the information is stored in the client terminal 50 owned by the receiver of the seat information. The web application server 30 is provided as a seat information providing device to be provided. Further, in the seat information processing system 1, the Web application server 30 includes a DB server 40 as means for managing various data. Note that the number of devices (computers) constituting the seat information providing device of the present invention and the connection configuration between the devices are not limited. In this embodiment, as shown in FIG. 1, a description will be given assuming that the Web application server 30 and the DB server 40 are installed in a data center.

  As described above, the client terminal 50 is a terminal owned by the receiver of the seat information of the store, and connects to the Web application server 30 according to the operation of the receiver to obtain the seat information of each store. Perform the process of displaying.

  It is assumed that the devices in the seat information processing system 1 can communicate with each other. In the example of this embodiment, each device in the seat information processing system 1 will be described as being configured to be mutually connectable via the Internet.

  Next, the configuration of each device constituting the seat information processing system 1 will be described.

  First, the configurations of the camera 10 (10-1 to 10-N) and the local computer 20 (20-1 to 20-N) installed in each store will be described. Although the detailed configuration (for example, hardware configuration) of the devices installed in each store may be different, the devices can be functionally shown as the same configuration as shown in FIG.

  The camera 10 installed in each store is a camera that captures an image of a predetermined area inside the installed store (an area including a seat (for example, a seat such as a chair) for which seat information is to be processed). Various digital cameras can be applied as the camera 10. The camera 10 supplies an image of the inside of the store to the local computer 20 in the same store.

  The local computer 20 is a device that processes seat information in the installed store. Specifically, the local computer 20 acquires seat information in the store based on the image supplied from the camera 10 and supplies the acquired seat information to the Web application server 30.

  In this embodiment, the seat information acquired by the local computer 20 includes, at least, vacant seats (hereinafter, referred to as “process target seats”) that are not used among the seats to be processed (hereinafter, referred to as “process target seats”) at the installation store. , And “vacant seats” (hereinafter referred to as “vacant seats”).

  Specifically, in the example of this embodiment, the seat information acquired by the local computer 20 includes the number of seats to be processed in the installation store (hereinafter, also referred to as the “number of installed seats”), The information includes the number of seats (hereinafter, also referred to as “occupied seats”) being used (for example, seated) by a person (for example, a customer of a store) (hereinafter, also referred to as “occupied seats”). Shall be Note that the local computer 20 may acquire the vacant seat number itself as seat information. When the seat information providing device (Web application server 30 and DB server 40) stores information on the number of seats installed at each store, the local computer 20 simply uses only the number of seats in use as seat information. You may make it acquire.

  The internal configuration of the local computer 20 can be functionally shown as in FIG. The local computer 20 is constructed, for example, by installing a program (including the “seat information acquisition program” according to the embodiment) in a computer (for example, a PC, a single-board computer, a microcomputer, etc.) having a memory and a processor. You may.

  As shown in FIG. 2, the local computer 20 includes a video acquisition unit 21, an object recognition unit 22, a seat information processing unit 23, and a data recording unit 24.

  The data recording unit 24 is a data recording unit that records various data used in each processing in the local computer 20.

  The video acquisition unit 21 performs a process of acquiring an image captured by the camera 10 and supplying the acquired image to the object recognition unit 22 for each predetermined period (hereinafter, this period is referred to as “T1”). The video acquisition unit 21 may control the camera 10 to capture an image every period T1, or may use a sequence of images (moving images) continuously captured and supplied by the camera 10 at intervals of the period T1. An image may be obtained. The interval of the period T1 is not limited, but may be, for example, 0.2 to 1 second.

  The object recognizing unit 22 analyzes an image for each time series (an image for each period T1) supplied from the video acquiring unit 21 and recognizes a person reflected in the installation store (in an area captured by the camera 10). An object (hereinafter, also referred to as a “person object”) is detected, and a user (hereinafter, referred to as a “seat user”) further using (for example, sitting) a processing target seat is detected from the detected human object. Perform the following processing. In this embodiment, the “seats” do not necessarily have to be provided with seats (chairs), and are places where eating and drinking can be performed while standing (use of facilities is possible) (so-called “standing seats”). It may be.

  The specific procedure of the object recognition processing by the object recognition unit 22 will be described later.

  The seat information processing unit 23 performs a process of acquiring seat information at the installation store using the recognition result of the object recognition unit 22. In this embodiment, the seat information processing unit 23 acquires at least information on the number of seats installed and the number of seats in use at the installation store as seat information.

  It is assumed that the seat information processing unit 23 of this embodiment reads and acquires data on the number of installed seats from data set in the data recording unit 24 in advance. In the local computer 20, the method of receiving the data input of the number of installed seats in the data recording unit 24 is not limited. For example, in the local computer 20, data input of the number of installed seats may be received by input means (for example, a display, a keyboard, a mouse, or the like) (not shown), or data input from an external device may be received by communication. You may.

  The seat information processing unit 23 acquires the number of seats in use based on the number of seat users currently recognized by the object recognition unit 22. That is. The seat information processing unit 23 performs a process of obtaining the number of seat users currently recognized by the object recognition unit 22 as the number of in-use seats.

  Then, the seat information processing unit 23 acquires information including the acquired number of seats and the number of seats in use as seat information, and transmits the acquired information to the Web application server 30.

  The timing at which the seat information processing unit 23 transmits the seat information to the Web application server 30 is not limited. For example, the seat information processing unit 23 may transmit the seat information (the latest acquired seat information) in accordance with the timing requested from the Web application server 30, or may transmit the seat information at predetermined intervals. It may be transmitted, or may be transmitted each time the seat information changes.

  Next, the internal configuration of each device (Web application server 30, DB server 40) constituting the seat information providing device will be described.

  FIG. 3 is a block diagram showing the internal configuration of the DB server 40.

  The DB server 40 is an apparatus that provides various data necessary for the processing of the Web application server 30. Specifically, the DB server 40 holds a database of information about each store (hereinafter, referred to as a “store information DB”), and performs data processing (for example, updating or reading data) in response to a request from the Web application server 30. Process).

  The DB server 40 has a DB processing unit 41 and a data recording unit 42.

  The internal configuration of the DB server 40 can be functionally shown as in FIG. The DB server 40 can be constructed using, for example, a computer (for example, a PC or a workstation) having a memory and a processor.

  As shown in FIG. 3, the DB server 40 has a DB processing unit 41 and a data recording unit 42.

  The data recording unit 42 is a data recording unit that records various data used in each process in the DB server 40. As shown in FIG. 3, it is assumed that at least the store information DB 421 is recorded in the data recording unit 42.

  The DB processing unit 41 performs data processing of a database (store information DB 421) recorded in the data recording unit 42. The DB processing unit 41 can be constructed using, for example, a DBMS (Database Management System) such as ORACLE (registered trademark) or MySQL (registered trademark). Therefore, the store information DB 421 recorded in the data recording unit 42 needs to have a data structure corresponding to the DB processing unit 41.

  Next, the data structure of the store information DB 421 will be described with reference to FIG.

  FIG. 4 is an explanatory diagram showing the data structure of the store information DB 421. FIG. 4 illustrates the data structure of the store information DB 421 in a table format. In FIG. 4, information of one store is illustrated in one line.

  As shown in FIG. 4, a store ID, a store name, the number of seats installed, the number of seats in use, access information, and store information are registered in the store information DB 421 for each store.

  The item of “store ID” is an item that functions as a key for identifying each store in the store information DB 421. As shown in FIG. 4, in this embodiment, it is assumed that IDs 1 to N are assigned to a first store to an N-th store, respectively.

  The item of “store name” is an item for recording the name of the store. In FIG. 4, the store name of the first store (store ID = 1) is “AAA”, the store name of the second store (store ID = 2) is “BBB”, and the third store (store ID = 3). , The store name of the Nth store (store ID = N) is “XXX dining room”.

  The information of the “number of seats installed” and the “number of seats in use” are items for recording seat information of each store.

  The access information is information for accessing the local computer 20 of each store. The access information includes an IP address for accessing the local computer 20, a host name, an FQDN (Fully Qualified Domain Name), and the like. In FIG. 4, the access information for accessing the local computers 20-1 to 20-N of each store is illustrated as host1 to hostN, respectively.

  The item of “store information” is an item representing information of each store (for example, information to be presented to an information-supplied person having the client terminal 50). FIG. 4 shows an example in which information of an address, business hours, and a homepage URL is recorded as store information of each store. In the store information DB 421, the number and combination of information items to be recorded as store information are not limited.

  The store information DB 421 is configured as described above. Then, the DB processing unit 41 performs data processing (for example, data updating and reading processing) of the store information DB 421 configured as described above in response to a request (command) of the Web application server 30.

  FIG. 5 is a block diagram showing the internal configuration of the web application server 30.

  The Web application server 30 has a Web service unit 31, a server application 32, and a data recording unit 33.

  The internal configuration of the Web application server 30 can be functionally shown as in FIG. The Web application server 30 can be constructed using, for example, a computer (for example, a PC or a workstation) having a memory and a processor.

  The data recording unit 33 is a data recording unit that records various data used in each processing in the web application server 30.

  The Web service unit 31 provides the client terminal 50 with information including seat information of each store as content (for example, a Web page). The Web service unit 31 provides the content under the control of the server application 32 to the client terminal 50.

  As the Web service unit 31, for example, middleware of various Web servers can be applied. In this case, the server application 32 is an application (server-side program) that performs a process of generating content to be provided in response to a request from the outside (for example, the client terminal 50) on middleware as the Web service unit 31. .

  The server application 32 acquires the seat information (information including the number of seats installed and the number of seats in use in this embodiment) of each store from the local computer 20 of each store, and stores the latest acquired seat information in the store of the DB server 40. Register in the information DB 421.

  Further, the server application 32 responds to a request (operation received by the client terminal 50) received from the client terminal 50 by the Web service unit 31 and stores information (data relating to some or all stores) of the store information DB 421 of the DB server 40. ) To get. Then, the server application 32 generates a content (Web screen) using the acquired information of the store information DB 421, and performs a process of providing the Web service unit 31 to the client terminal 50 of the request source.

  A configuration example of the content (Web screen) provided by the Web application server 30 will be described later.

  Next, the internal configuration of the client terminal 50 will be described.

  FIG. 6 is an explanatory diagram showing an example of the internal configuration of the client terminal 50.

  The client terminal 50 can be constructed using various terminal devices (computers; hardware) such as a smartphone, a PC, and a tablet PC. FIG. 6 illustrates an example in which a smartphone is applied as hardware of the client terminal 50.

  As shown in FIG. 6, the client terminal 50 has a touch panel display 51 and a client application 52.

  The touch panel display 51 is a device (a device having a function of a user interface) for receiving information (for example, input using a GUI screen) from the information provider to output information to the information provider possessing the client terminal 50. It is. In this embodiment, an example is shown in which the client terminal 50 is constructed using a smartphone. However, the client terminal 50 is responsible for a user interface (input means from the information provider and output means to the information provider). The type of device is not limited. For example, when the client terminal 50 is constructed using a PC, a display, a mouse, and a keyboard may be used as a user interface.

  The client application 52 uses the touch panel display 51 to receive an operation from the information provider, transmits a request corresponding to the operation of the information provider to the Web application server 30, and transmits the content supplied from the Web application server 30. (Information including seat information of one or a plurality of stores) is displayed on the touch panel display 51.

  For example, the client application 52 may be constructed using various Web browsers. In this embodiment, the client application 52 is set as a predetermined URL (URL for providing seat information of each store) on the Web application server 30 as a connection destination (connection destination URL) to the Web browser. explain.

  The client application 52 may be constructed as a dedicated application without using a Web browser.

(A-2) Operation of Embodiment Next, the operation of the seat information processing system of the embodiment having the above configuration will be described.

  First, an installation environment of the camera 10 in each store will be described.

  FIG. 7 is an example of a plan view inside the store of the first store (store ID = 1).

  Hereinafter, an example of an installation environment of the camera 10 will be described using the inside of the first store as an example.

  As shown in FIG. 7, in the first store, a square counter table R1, eight chairs C (C1 to C8) arranged side by side on one side of the counter table R1, and a cooking table R2 are provided. Are located.

  In the first store, these eight chairs C1 to C8 are assumed to be the processing target seats.

  Then, as shown in FIG. 7, a camera 10-1 is arranged at a corner (edge) in the first store.

  FIG. 8 is a diagram (perspective view) when the inside of the first store (store ID = 1) is viewed from the camera 10-1 installed in the store.

  As described above, since the chairs C1 to C8 are the processing target seats in the first store, the camera 10-1 is set in a position, an orientation (posture), and a zoom setting such that the chairs C1 to C8 are all within the field of view for imaging. Need to be installed. In FIG. 8, the frame of the visual field A captured by the camera 10-1 is shown by a dashed line. As shown in FIG. 8, all the chairs C1 to C8 are accommodated in the visual field A.

  Therefore, when no one is in the first store, the image captured by the camera 10-1 is an image as shown in FIG. In the image shown in FIG. 9, all the chairs C1 to C8 are shown. Actually, when the user of each chair C is seated, the user's face is positioned slightly above the chair, so that even the area where the user's face is actually located is included in the field of view A. It is desirable to install the camera 10-1 so that it fits.

  Next, a description will be given of a process in which the local computer 20-1 installed in the first store acquires seat information in the installed store.

  As described above, the seat information processing unit 23 acquires at least information on the number of seats installed and the number of seats in use at the installation store as seat information. As described above, the seat information processing unit 23 of this embodiment reads and acquires data on the number of installed seats from data set in the data recording unit 24 in advance. Further, the seat information processing unit 23 performs a process of obtaining the number of seat users currently recognized by the object recognition unit 22 as the number of seats being used. Then, the seat information processing unit 23 acquires information including the acquired number of seats and the number of seats in use as seat information, and transmits the acquired information to the Web application server 30.

  Next, operations of the video acquisition unit 21 and the object recognition unit 22 will be described.

  As described above, in this embodiment, the video acquisition unit 21 performs a process of acquiring an image captured by the camera 10 and supplying the acquired image to the object recognition unit 22 every predetermined period T1. The camera 10-1 installed in the first store supplies an image of the area of the field of view A including the chairs C1 to C8 to the image acquisition unit 21 of the local computer 20-1 at least for each period T1.

  The object recognizing unit 22 performs an object recognizing process on an image for each time series supplied from the video acquiring unit 21.

  The object recognizing unit 22 of this embodiment first performs a process of analyzing an image for each period T1 and recognizing a human object that can be recognized as a human (hereinafter, referred to as a “human object recognition process”). For example, assume that the image captured by the camera 10-1 is an image as shown in FIG. In the image shown in FIG. 10, the seat user is seated on the chairs C1, C5, and C6. Further, in the image shown in FIG. 10, the clerk is walking in front of the counter table R1.

  At this time, it is assumed that the object recognizing unit 22 has recognized the seat users sitting on the chairs C1, C5, and C6 as the human objects OB101, OB102, and OB103, respectively. At this time, it is assumed that the object recognizing unit 22 has recognized the clerk in front of the counter table R1 as the human object OB104.

  The object recognizing unit 22 tracks the movement of each of the human objects OB101, OB102, OB103, and OB104 during the latest predetermined period (hereinafter, this period is referred to as “T2”) (hereinafter, “human object tracking”). Process). Then, the object recognizing unit 22 recognizes the movement characteristics of each person object from the movement history of each person object obtained by the person object tracking process.

  For example, since the seat user is in a state of sitting on a chair, the movement of the human object of the seat user (for example, the human objects OB101 to OB103 shown in FIG. 10) is also small. On the other hand, a non-seat user such as a clerk (for example, the person object OB104 shown in FIG. 10) is not seated, so that the person object of the non-seat user moves more than the seat user. Further, when a photograph or a painting in which a person is drawn is placed in the store, the person is recognized as a human object in the human object recognition processing. An object erroneously recognized as such a human object (hereinafter, also referred to as a “misrecognized object”) does not move at all, and thus is in a state where normal movement is not detected. However, even a misrecognized object may be recognized as if a minute movement has occurred due to a change in an image around the misrecognized object, a quantization error in the human object recognition process, or the like. However, since the minute movement recognized for the misidentified object is smaller than at least the movement of the seat user (sitting person), the characteristics of the movement are different.

  As described above, the motion of an object recognized as a human object has different characteristics depending on its attribute. Therefore, the object recognizing unit 22 of this embodiment performs a process of tracking the movement of each person object (hereinafter, referred to as a “person object tracking process”), and records the movement history (the state of movement) of each person object. A process of acquiring and recognizing the attribute of each person object based on the history is performed. In other words, the object recognizing unit 22 detects a person object of a seat user (hereinafter, referred to as a “seat user object”) based on a history of movement of each person object (hereinafter, “seat user object”). Detection process ”).

  In the seat user object detection process of this embodiment, the non-seat user object and the misidentified object are excluded from the recognized person object (the person object recognized by the person object recognition process), and the seat user object is removed. It shall be detected.

  Next, a specific example of a human object recognition process performed by the object recognition unit 22 will be described.

  As described above, the object recognition unit 22 performs a process of recognizing a human object from a series of images (frames) captured by the camera 10 (human object recognition process).

  The specific method of the human object recognition process performed by the object recognition unit 22 is not limited. The human object recognition processing performed by the object recognition unit 22 can be realized, for example, by using an open source library related to existing image recognition processing. Here, a description will be given assuming that a human object recognition process using OpenCV (Open Source Computer Vision Library; see References 1 and 2 below) and TensorFlow (see Reference 3 below) is performed.

Reference 1: OpenCV, [Online], INTERNET, [Search November 22, 2017], <URL: opencv.jp>
Reference 2: OpenCV Samples and Documents, [Online], INTERNET, [Search November 22, 2017], <URL: opencv.jp>
Reference 3: TensorFlow, [Online], INTERNET, [Search November 22, 2017], <URL: https://www.tensorflow.org/>

  The object recognizing unit 22 of this embodiment performs a process of recognizing an object (not limited to a person) in an image (hereinafter, referred to as an “object recognition process”) using the function of OpenCV, and performs a TensorFlow of the object from the recognized object. A human object recognition process can be performed by performing a process of classifying a human object using a function (hereinafter, referred to as a “human object classification process”).

  In the object recognition process, the object recognition unit 22 performs an outline detection process on each image captured by the camera 10 and performs an object recognition process based on the outline detection result. The accuracy may be improved by performing a filter process for converting the image into a binary image of black and white as a pre-process of the outline detection process. The object recognizing unit 22 uses the OpenCV function to convert each object from an image (pre-processed image) to an image of a rectangle (circumscribed rectangle) including the outline region of each object (hereinafter, “circumscribed rectangle image”). Is cut out and held.

  FIG. 11 is an explanatory diagram illustrating an example of a circumscribed rectangle cut out from an image by the object recognition unit 22.

  FIG. 11 illustrates circumscribed rectangles SQ101 to SQ104 for the individual objects OB101 to OB104.

  Next, a detailed example of the human object classification processing by the object recognition unit 22 will be described. The object recognizing unit 22 determines whether or not an image of a part of a person (for example, a human face) is included in the circumscribed rectangular image of each object obtained by the object recognition process, using an image recognition function of tensorflow. to decide. For example, the object recognizing unit 22 can determine whether or not a human face is included in the circumscribed rectangular image of each object by using a “create_feed_dict” function included in “human_recognizer.py” of tensorflow. The create_feed_dict function is a function that, when an image is input, outputs a numerical value (a numerical value in a range from 0 to 1) of an evaluation result (Score) as to whether or not a human face is included in the image as a return value. For example, the object recognizing unit 22 inputs a circumscribed rectangular image of each object to the create_feed_dict function, and converts the object (circumscribed rectangular image) whose evaluation result (Score) is equal to or more than a threshold value (for example, 0.4 or more) into a human object You may make it classify as.

  As described above, the object recognizing unit 22 performs a process of recognizing, as a human object, an object corresponding to a circumscribed rectangular image that is evaluated as a person (a human object recognition process).

  Next, a specific example of the human object tracking processing by the object recognition unit 22 will be described.

  The object recognition unit 22 performs a process of tracking a circumscribed rectangle (circumscribed rectangle image) of each person object in the human object tracking process. Various processes can be applied to the process of tracking the circumscribed rectangle for each human object (the process of tracking the movement of each circumscribed rectangle).

  For example, if an image at an arbitrary time is F1 and a time series image after the image F1 (for example, a time series next to F1) is F2, the object recognition unit 22 determines that the distance between the image F1 and the image F2 is A circumscribed rectangle of a human object equal to or less than a predetermined value may be recognized as a circumscribed rectangle of the same human object.

  FIG. 12 is an explanatory diagram showing an example in which the object recognizing unit 22 obtains a motion amount in the human object tracking processing.

  FIG. 12 illustrates a circumscribed rectangle of the human object OB201 in the image F1 and the time-series image F2 after the image F1.

  In FIG. 12, the horizontal direction of the image is called an X direction, and the vertical direction of the image is shown as Y. In the following, the distance on the image is a unit based on the resolution (pixel). For example, a distance of one pixel (one resolution) on an image may be referred to as 1.

  In FIG. 12, the circumscribed rectangle (dotted line) of the human object OB201 in the image F1 is illustrated as SQ201, and the circumscribed rectangle (dotted line) of the human object OB201 in the image F2 is illustrated as SQ202. In FIG. 12, the distance between the sides of the circumscribed rectangle SQ201 in the X direction is X201, and the distance between the sides of the circumscribed rectangle SQ201 in the Y direction is Y201.

  The object recognition unit 22 detects the circumscribed rectangle SQ201 of the human object OB201 from the image F1 and detects the circumscribed rectangle SQ202 of the human object OB201 from the image F2 by the object recognition processing. Then, the object recognition unit 22 obtains a distance between the circumscribed rectangle SQ201 and the circumscribed rectangle SQ202.

  The method by which the object recognizing unit 22 obtains the distance between the circumscribed rectangle of the image F1 and the circumscribed rectangle of the image F2 is not limited. For example, the center points of each circumscribed rectangle or any vertex (rectangle) (Any of the four vertices) may be obtained. In FIG. 12, the positions (coordinates) of the center points of the circumscribed rectangles SQ201 and SQ202 are illustrated as positions P201 and P202, respectively.

  At this time, the object recognition unit 22 determines that the distance (difference) DX in the X direction is less than “X201 · KX1” and that the distance (difference) DY in the Y direction is less than “Y201 · KY1” ((1) ) And (2)), the circumscribed rectangle SQ201 and the circumscribed rectangle SQ202 are recognized as circumscribed rectangles of the same human object.

Arbitrary constants can be set for KX1 and KY1. Here, a description will be given assuming that KX1 = 0.5 and KY1 = 0.5. In this case, assuming that the shapes (dimensions) of the two circumscribed rectangles SQ201 and SQ202 are the same, the state where the following equations (1) and (2) are satisfied means that the areas of the two circumscribed rectangles SQ201 and SQ202 overlap each other. A state. In FIG. 12, between the position P201 and the position P202, the distance DX in the X direction is less than “X201 · 0.5”, and the distance DY in the Y direction is less than “Y201 · 0.5”. Therefore, the object recognition unit 22 can recognize (link) the circumscribed rectangle SQ201 and the circumscribed rectangle SQ202 as those of the same human object OB201.
X201 · KX1 <DX (1)
Y201 · KY1 <DY (2)

  Note that KX1 and KY1 may be different values. Regarding KX1 and KY1, the size of the circumscribed rectangle actually recognized by the object recognizing unit 22, the interval (period T1) at which the human object tracking process is performed, the experimental result (for example, an experiment of recognition accuracy), and the like are taken into consideration. It is desirable to tune to an optimal value.

  As described above, every time an image is acquired, the object recognizing unit 22 links the circumscribed rectangle of the human object with the image whose time series is earlier or later, thereby performing the human object tracking process (for each image, (A process of recognizing a circumscribed rectangular area corresponding to each person object).

  Next, a seat user object detection process performed by the object recognition unit 22 will be described.

  The object recognizing unit 22 can hold a history of movement of each person object (movement of a circumscribed rectangle) by the person object tracking process. Here, the object recognizing unit 22 determines, for each human object, the movement of the circumscribed rectangle for the most recent period T2 (for the period from the present time to the past time by the period T2) (for example, the center point or any vertex of the vertex). By analyzing the movement, the characteristic of the movement is recognized, and the seat user object is detected based on the recognition result.

  FIG. 13 is an explanatory diagram showing an example of obtaining a motion amount in the seat user object detection processing performed by the object recognition unit 22.

  Here, as shown in FIG. 13, the circumscribed rectangle of an arbitrary human object OB301 in the image F3 at the present time is SQ301. In FIG. 13, the center point of the circumscribed rectangle SQ301 is illustrated as P301. Further, in FIG. 13, the circumscribed rectangle of the human object OB301 detected from the image F-i (i is an integer of 1 to M) before the image F3 in time series is set to “SQ-i”, and the circumscribed rectangle SQ- The center point of i is illustrated as “Pi”. For example, assuming that the time at which the image F3 is supplied in the object recognition unit 22 is t1, the time at which the image Fi is supplied is "t1- (i.times.T1)". For example, if the period T1 is 1 second and the period T2 is 60 seconds, M = 60/1 = 60. Note that the object recognition unit 22 does not need to apply all the images used up to the human object tracking processing to the seat user object detection processing. For example, a part of the image used before the human object tracking processing may be thinned out and applied to the seat user object detection processing. For example, the object recognizing unit 22 extracts every other image used until the human object tracking process (for example, extracts images F-1, F-3, F-5, F-7,...) You may make it perform a seat user object detection process.

  In FIG. 13, the distance between the sides of the circumscribed rectangle SQ301 in the X direction is X301, and the distance between the sides of the circumscribed rectangle SQ301 in the Y direction is Y301. In FIG. 13, the distance (difference) in the X direction between P301 and P-i is illustrated as DX, and the distance (difference) in the Y direction is illustrated as DY.

  In the seat user object detection processing, the object recognizing unit 22 obtains the “distance DX in the X direction” and the “distance DY in the Y direction” between the circumscribed rectangle SQ301 and all of the circumscribed rectangles SQ-1 to SQ-M. . The distance (DX, DY) between the circumscribed rectangle SQ301 and all of the circumscribed rectangles SQ-1 to SQ-M is the movement history of the human object OB301.

  Then, the object recognizing unit 22 analyzes the characteristics of the motion based on the distance (DX, DY) between the circumscribed rectangle SQ301 and all of the circumscribed rectangles SQ-1 to SQ-M, and based on the analysis result, detects the human object. The attribute of the OB 301 (for example, whether or not it is a seat user object) is recognized.

  First, an example of processing performed by the object recognition unit 22 to determine whether or not the human object OB301 is a non-seat user object will be described.

  The object recognition unit 22 calculates the distance (DX, DY) between the circumscribed rectangle SQ301 and all of the circumscribed rectangles SQ-1 to SQ-M, and calculates the distance in the X direction for all of the circumscribed rectangles SQ-1 to SQ-M. (Difference) DX is less than “X301 · KX2” and the distance (difference) DY in the Y direction is less than “Y301 · KY2” (the condition that the following expressions (3) and (4) are satisfied) (Condition) is determined. Then, when all of the circumscribed rectangles SQ-1 to SQ-M satisfy the above-described conditions (when the following expressions (3) and (4) are satisfied), the object recognition unit 22 determines that the human object related to the circumscribed rectangle SQ301 The OB 301 is recognized as not a non-seat user object (recognized as a seat user object if other conditions are satisfied). In other words, if at least one of the distances obtained for the circumscribed rectangles SQ-1 to SQ-M does not satisfy the following (3) or (4), the object recognizing unit 22: The human object OB301 relating to the circumscribed rectangle SQ301 is recognized as a non-seat user object.

  Arbitrary constants can be set for KX2 and KY2. Here, a description will be given assuming that KX2 = 0.5 and KY2 = 0.5. In this case, if the shapes (dimensions) of the two circumscribed rectangles SQ301 and SQ-i are the same, the state satisfying the following equations (3) and (4) means that the two circumscribed rectangles SQ301 and SQ-i It can be said that the areas overlap each other.

Note that the object recognizing unit 22 determines that a person object for which the circumscribed rectangle SQ cannot be continuously obtained from the current time to a time before the period T2 (a person object for which all of the circumscribed rectangles SQ-1 to SQ-M cannot be obtained) is a non-seat user object. You may make it recognize. This is because the recognition accuracy of a person object for which a period T2 or more has not yet elapsed since the start of recognition may not be determined as a seat user object in some cases.
X301 · KX2> DX (3)
Y301 · KY2> DY (4)

  Note that KX2 and KY2 may have different values. Further, if KX2 and KY2 are too large, a non-seat user object is erroneously recognized as a seat user object, and if KX2 and KY2 are too small, a seat user object is erroneously recognized as a non-seat user object. Therefore, it is usually desirable to verify the amount of movement of the seat user object during the period T2 by an experiment or the like, and to set KX2 and KY2 based on the verification result. Further, it is desirable that KX2 and KY2 are tuned to optimal values in consideration of the size of the circumscribed rectangle actually recognized by the object recognition unit 22, the interval (period T2) for performing the human object recognition process, and the like. .

  Here, the distance in the X direction of the circumscribed rectangle SQ104 of the clerk's person object OB104 is assumed to be X104 and KX2 = 0.5. Then, if the clerk of the human object OB104 moves by more than “0.5 · X104” in the X direction on the image during the period T2 (60 seconds), the human object OB104 is recognized as a non-seat user object (seat). It is recognized as not being a user object). Normally, the clerk of the restaurant (the clerk of the hall) continues to move in the store to serve customers, and therefore it is unlikely that the clerk will stay in the same place for the period T2 (60 seconds) or more. Therefore, in the example of FIG. 11, the clerk person object OB104 is generally recognized as a non-seat user object (recognized as not a seat user object).

  Next, an example of a process in which the object recognizing unit 22 determines whether or not the human object OB301 is an erroneously recognized object will be described.

The object recognition unit 22 determines that the distance (difference) DX in the X direction is less than “X301 · KX3” and the distance (difference) DY in the Y direction is “Y301 · D” for all of the circumscribed rectangles SQ-1 to SQ-M. It is determined whether or not the condition that KY3 is smaller than “KY3” (the condition that the following expressions (5) and (6) are satisfied) is satisfied. Then, when all of the circumscribed rectangles SQ-1 to SQ-M satisfy the above-described conditions (when the following expressions (5) and (6) are satisfied), the object recognition unit 22 determines that the human object related to the circumscribed rectangle SQ301 The OB 301 is recognized as an erroneously recognized object (recognized as not a seat user object). In other words, if at least one of the distances determined for the circumscribed rectangles SQ-1 to SQ-M does not satisfy the following (5) or (6), the object recognizing unit 22 determines that the person associated with the circumscribed rectangle SQ301 The object OB301 is recognized as not being a misidentified object (if other conditions are satisfied, it is a seat user object).
X301 · KX3> DX (5)
Y301 · KY3> DY (6)

  As described above, an erroneously recognized object (for example, a human object based on a photograph or a painting in which a person is drawn in a store) does not actually move at all, but recognizes as if a minute movement occurred during the human object recognition process. Could be done. However, the minute movement recognized for the misidentified object is at least smaller than the movement of the seat user (sit person). This does not mean that a person does not move a part such as a face at all even when sitting in a chair. Therefore, when viewed during the period T (60 seconds), the amount of movement of the seat user object (usually, the misidentification object) This is because the maximum amount of movement) increases. Therefore, as described above, when the movement of the circumscribed rectangle SQ301 is less than or equal to a predetermined value (when the distances obtained for the circumscribed rectangles SQ-1 to SQ-M all satisfy Expressions (5) and (6)) First, the human object OB301 can be recognized (estimated) as an erroneously recognized object.

  Any constants can be set for KX3 and KY3. For example, KX3 = 0.1 and KY3 = 0.1 may be set. Note that KX3 and KY3 may have different values. Further, if KX3 and KY3 are too large, the non-seat user object is erroneously recognized as an erroneous recognition object, and if KX3 and KY3 are too small, the erroneous recognition object is erroneously recognized as a seat user object. Therefore, it is desirable to verify the amount of movement of the normal seat user object during the period T2 by an experiment or the like, and to set KX3 and KY3 based on the verification result. Further, it is desirable that KX3 and KY3 are tuned to optimal values in consideration of the size of the circumscribed rectangle actually recognized by the object recognition unit 22, the interval (period T2) for performing the human object recognition process, and the like. .

  Through the above processing, when the object recognizing unit 22 recognizes that the human object OB301 is neither a human object of a non-seat user nor a false recognition object, it recognizes (extracts) the human object OB301 as a seat user object. That is, the object recognizing unit 22 recognizes a non-seat user object or a misrecognized object as a seat user object from the individual objects subjected to the human object tracking processing. Then, the seat information processing unit 23 counts the number of seat user objects based on the result of the latest seat user object detection process performed by the object recognizing unit 22, and acquires the count result as the number of seat users. I do.

  Next, the update operation of the seat information in the Web application server 30 and the DB server 40 will be described.

  Here, it is assumed that information other than the seat information (the number of seats installed and the number of seats in use) of each store is registered as initial master data in the store information DB 421 of the DB server 40.

  When the server application 32 of the web application server 30 receives the seat information (the number of seats installed and the number of seats in use) from the local computer 20 of each store, the server application 32 stores information on the DB server 40 (store information DB 421) based on the seat information. (Received seat information (number of seats installed and number of seats in use) relating to the store) is updated. Note that, as described above, the timing at which the seat information processing unit 23 transmits the seat information to the Web application server 30 is not limited.

  As described above, the DB server 40 (store information DB 421) is always kept in the latest state (a state in which the latest seat information of each store is held) based on the seat information received from the local computer 20 of each store. become.

  Next, an operation of providing seat information in the web application server 30 and the DB server 40 will be described.

  As described above, the Web application server 30 provides the content including the seat information held by the DB server 40 in response to a request from the client terminal 50. The content (data of the web screen) provided from the web application server 30 to the client terminal 50 is displayed on the touch panel display 51 of the client terminal 50.

  For example, when the Web application server 30 (Web service unit 31) receives an access (for example, an access using a protocol such as HTTP) from the client terminal 50 at a predetermined URL (URL of a homepage for providing seat information). Then, a content (Web screen data) of a Web screen (hereinafter, referred to as a “store selection screen”) capable of accepting a store selection is generated and provided to the client terminal 50.

  FIG. 14 is an explanatory diagram showing a configuration example of the store selection screen.

  The store selection screen (Web screen) in FIG. 14 displays a button (link) to which the store name of each store is added, and allows the user possessing the client terminal 50 to select (click) a store that provides content including seat information. Touch). On the store selection screen shown in FIG. 14, a button B101 for receiving selection of the first store (store name: “AAA cafe”) and a button B101 for receiving selection of the second store (store name: “BBB restaurant”) are displayed. A state is shown in which a button B102 and a button B102 for receiving selection of a third store (store name: “CCC coffee”) are displayed.

  At this time, the server application 32 acquires a store name for each store from the DB server 40 (store information DB 421), and causes the Web service unit 31 to generate a store selection screen (Web screen) as shown in FIG. Do. The specific configuration of the store selection screen is not limited as long as it is a Web screen that can select any store.

  Here, it is assumed that the user holding the client terminal 50 has selected the button B101 for accepting the selection of the first store (store name: “AAA cafe”) on the store selection screen shown in FIG.

  When the first store (store name: “AAA cafe”) is selected on the store selection screen, the server application 32 reads the seat information of the first store from the DB server 40 (store information DB 421). Acquire the number of seats in use) and store information (address, business hours, homepage URL). Then, based on the acquired seat information and store information of the first store (store name: “AAA cafe”), the server application 32 converts the information (information including the seat information and the store information) acquired by the Web service unit 31 into the Web application. A Web screen to be presented (hereinafter, also referred to as a “store information providing screen”) is generated and provided to the client terminal 50 of the connection source.

  FIG. 15 is an explanatory diagram illustrating a configuration example of a store information providing screen.

  In the store information providing screen shown in FIG. 15, a field F101 for displaying seat information and a field F102 for displaying information about the first store (information including a store name and store information) are arranged.

  In the store information providing screen shown in FIG. 15, the number of vacant seats and the number of seats (the number of seats in use) are displayed as seat information in a field F101. In the store information providing screen, only the number of vacant seats may be displayed as seat information, or the number of seats installed and the number of seats (the number of seats in use) may be displayed.

(A-3) Effects of Embodiment According to this embodiment, the following effects can be obtained.

  In the seat information processing system 1 of this embodiment, the number of vacant seats of each store can be recognized only by registering information on the number of seats installed as seat information of each store. As a result, in the seat information processing system 1 of this embodiment, since it is not necessary to input seat position information and the like of each store, introduction work and change of the seat layout are facilitated, and convenience is improved as compared with the related art. are doing.

  Further, in the seat information processing system 1 of this embodiment, the number of seats in use (the number of seated users) can be recognized only by installing the camera 10 in each store and performing image processing. As a result, in the seat information processing system 1 of this embodiment, it is not necessary to attach a sensor for each seat or the like, so that the introduction cost can be reduced. In other words, in the seat information processing system 1 of the present embodiment, the presence or absence of a user is not recognized for each seat, and the seat information processing system 1 performs the human object tracking processing and the seat user object detection processing to display the image captured by the camera 10. Since only the number of seated users is counted, the number of seated users can be recognized with a smaller processing amount as compared with the case where image recognition processing is performed for each seat.

  Furthermore, in the seat information processing system 1 of this embodiment, when recognizing the movement of the circumscribed rectangle of each person object in the human object tracking processing and the seat user object detection processing, the dimension (for example, Since the comparison results (for example, the results of Expressions (1) to (6)) with the distances between the sides of the rectangle and the diagonal lines are used, the same accuracy is obtained regardless of the distance from the camera 10 to each seat. Can recognize the movement. On the image captured by the camera 10, the circumscribed rectangle of a distant human object is recognized as small, and the circumscribed rectangle of a nearby human object is recognized as large. Therefore, the recognition accuracy of the human object tracking processing and the seat user object detection processing can be improved by using the result of comparison with the dimensions constituting the circumscribed rectangle as in this embodiment.

(B) Other Embodiments The present invention is not limited to the above embodiments, but may include modified embodiments as exemplified below.

  (B-1) In the above embodiment, the processing performed by each local computer 20 may be performed on the server side (the Web application server 30 or a server attached to the Web application server 30).

  (B-2) In the above embodiment, the store selection screen provided by the Web application server 30 (server application 32) (see FIG. 14) only displays the stores registered in the store information DB 421. The store to be displayed according to information such as the position of each store and the seat information of the seats of each store (for example, the number of vacant seats and the number of seats installed) (hereinafter, also referred to as “search information”), and the order of the stores to be displayed are It may be changed.

  For example, the server application 32 acquires the position information (the position information of the client terminal 50) from the client terminal 50 that is the request source of the store selection screen (the request source of the Web communication), and displays the client terminal 50 on the store selection screen. The buttons related to each store may be displayed in the order of short distance from the position. For example, the server application 32 may display the buttons on the store selection screen from the top of the screen in order from the store closest to the position of the client terminal 50.

  In addition, as shown in FIG. 14, the server application 32 may be configured to receive a search for a store that satisfies predetermined conditions on the store selection screen.

  14A, a field F101 for receiving an input of the number of vacant seats and a field F102 for receiving an input of a distance from the client terminal 50 (“distance from here” in FIG. 14) are arranged on the store selection screen. I have. Then, on the store selection screen shown in FIG. 14A, a button B201 for accepting a search for a store that matches the conditions input to the fields F101 and F102 near the client terminal 50 (in FIG. "Search for a store that is located" is displayed).

  In FIG. 14A, in a default state, “2” (2 or more vacant seats) is input to F101 indicating the number of vacant seats, and “200” (within 200 m) is input to F102 indicating “Distance from here”. It is in a state of being left. Note that the input method for the fields F101 and F102 is not limited, and may be a method for receiving a numerical value input, or a method for selecting with a combo box or a radio button. The default values of the fields F101 and F102 are not limited.

  Note that the items and the number of items of the search condition on the store selection screen are not limited. For example, on the store selection screen shown in FIG. 14, only one of the number of vacant seats or the distance may be received as a search condition, or another condition may be added. Note that, in the store selection screen shown in FIG. 14, an example in which the number of vacant seats and the distance are input is described. However, the input of the number of vacant seats and the distance is not particularly input, and the default values (preset values) are set in the server application 32. Value) may be applied.

  When the button B201 is pressed on the store selection screen shown in FIG. 14A, the server application 32 acquires the position information of the client terminal 50, and sets the distance set in the field F102 from the acquired position of the client terminal 50. A store within which the number of vacant seats is equal to or greater than the number set in the field F101 is searched from the contents of the (store information item) of the store information DB 421. The method by which the server application 32 acquires the position information of the client terminal 50 is not limited. For example, the server application 32 may request the client terminal 50 to acquire the position information. For example, the client terminal 50 may acquire the position information based on communication with a GPS or a carrier network of a mobile phone company. In addition, the server application 32 may acquire the location information based on the address of each store, for example, based on a map information database on the Internet (for example, data such as Google Map (registered trademark)).

  Here, it is assumed that the matching store is only “AAA cafe” (first store). Then, the server application 32 changes the store selection screen to a state in which only the button B101 of the “AAA cafe” (first store), which is the matching store, is displayed, as shown in FIG. In addition, in FIG. 14B, a message indicating a search condition is displayed, "Stores having two or more vacant seats within 200 m from here are as follows." When there are a plurality of stores that match the search condition, the server application 32 displays a button for each store based on an order based on any of the conditions constituting the search information (for example, in the order of the shortest distance or in the order of the number of empty seats). You may do so. In the store selection screen shown in FIG. 14, the server application 32 acquires information on the client terminal 50 and each store when the button B201 is pressed on the screen in FIG. 14A, the server application 32 detects a store that has been detected based on search information (vacancy number and distance) preset from the beginning without going through the screen of FIG. It may be displayed (the screen of FIG. 14B is displayed).

  DESCRIPTION OF SYMBOLS 1 ... Seat information processing system, 10, 10-1-10-N ... Camera, 20, 20-1-20-N ... Local computer, 21 ... Video acquisition part, 22 ... Object recognition part, 23 ... Seat information processing part , 24: Data recording unit, 30: Web application server, 31: Web service unit, 32: Server application, 33: Data recording unit, 40: DB server, 41: DB processing unit, 42: Data recording unit, 421: Store Information DB, 50: client terminal, 51: touch panel display, 52: client application.

Claims (5)

Imaging means for imaging an inside of a facility where one or more seats are arranged;
Performing a human object detection process of detecting a human object from the image captured by the imaging unit, without using information on the position of the seat in the facility , based on the result of the human object detection process, based on the result of the human object detection process , Seat information acquisition means for acquiring the number of seat users indicating the number of users ;
Installed seat number holding means for holding information of the installed seat number in the facility,
Without using information on the positions of the seats in the facility, information on vacant seats in the facility is obtained as seat information based on the number of seats installed and the number of seat users, and the obtained seat information is provided by the information-provided provider. And means for providing seat information to the
The seat information acquisition means performs a occupant detection process of detecting a occupant sitting in a seat based on the movement of each person object from the person objects detected by the person object detection process. A seat information processing system , wherein the number of seats detected by processing is acquired as the number of seat users . The image pickup means for picking up the inside of the facility where one or more seats are arranged performs a human object detection process for detecting a human object from the image taken, and without using information on the positions of the seats in the facility, Seat information acquisition means for acquiring the number of seat users indicating the number of seats in the facility based on the result of the object detection process;
Installed seat number holding means for holding information of the installed seat number in the facility,
An external seat that obtains information on vacant seats in the facility based on the number of seats installed in the facility and the number of seat users without using information on the position of the seats in the facility and provides the information-provided person with the information. Transmitting means for transmitting information including the number of seat users to the information providing apparatus,
The seat information acquisition means performs a occupant detection process of detecting a occupant sitting in a seat based on the movement of each person object from the person objects detected by the person object detection process. A seat information acquiring apparatus , wherein the number of seated persons detected by processing is acquired as the number of seat users . Computer
The image pickup means for picking up the inside of the facility where one or more seats are arranged performs a human object detection process for detecting a human object from the image taken, and without using information on the positions of the seats in the facility, Seat information acquisition means for acquiring the number of seat users indicating the number of seats in the facility based on the result of the object detection process;
Installed seat number holding means for holding information of the installed seat number in the facility,
An external seat that obtains information on vacant seats in the facility based on the number of seats installed in the facility and the number of seat users without using information on the positions of seats in the facility and provides the information-provided person with the information. Causing the information providing device to function as transmission means for transmitting information including the number of seat users,
The seat information acquisition means performs a occupant detection process of detecting a occupant sitting in a seat based on the movement of each person object from the person objects detected by the person object detection process. A seat information acquisition program for acquiring the number of seated persons detected by processing as the number of seat users . The number of seat users acquired by the seat information acquisition device according to claim 2 , the number of seats installed in the facility is further maintained, and the number of seats installed in the facility is maintained without using information on the position of the seat in the facility. Seat information providing means for obtaining information on vacant seats in the facility as seat information based on the number of seats and the number of seat users, and providing the obtained seat information to the informed party; Providing device. Computer
The number of seats acquired by the seat information acquisition device according to claim 2 , the number of seats is retained, the number of seats installed in the facility is further retained, and the position of a seat in the facility is retained. Seat information providing means for acquiring information on vacant seats in the facility as seat information based on the number of installed seats and the number of seat users without using information on the seats , and providing the acquired seat information to an informed party A seat information providing program characterized by functioning as a.

Images