JP2022063323A - Information processor and system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate control of a robot that moves in a shop.

SOLUTION: An information processor comprises input means, display means, instruction input means, and output means. The input means inputs imaging data captured at a plurality of individual tables arranged in a shop. The display means displays an image on the basis of the imaging data corresponding to one of the plurality of tables. The instruction input means inputs an instruction to a robot having a moving function. In response to the instruction, the output means outputs, to the robot, an instruction for moving to the table corresponding to the image displayed by the display means.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2022,JPO&INPIT

Description

Embodiments of the present invention relate to an information processing apparatus and an information processing program.

In recent years, in restaurants and the like, the work of transporting food and drink from the kitchen to the seat where the customer is located is executed by a self-propelled robot on behalf of the clerk. After the food or drink is placed on the tray, for example, the robot starts running in response to an input operation on an employee's button. The robot travels along the route to the designated seat and carries food and drink to the seat.

The conventional robot operates in response to an input operation for a button provided on the robot. Therefore, when using a robot, the employee must be near the robot. In addition, after the robot started running, it was difficult for the employee to grasp the operating status of the robot.

Japanese Unexamined Patent Publication No. 2009-172410

An object to be solved by the present invention is to provide an information processing device and an information processing program capable of facilitating control of a robot moving in a store.

According to the embodiment, the information processing apparatus includes an input means, a display means, an instruction input means, and an output means. The input means inputs the imaging data captured in each of the plurality of tables arranged in the store. The display means displays an image based on the image pickup data corresponding to any of the plurality of tables. The instruction input means inputs an instruction to a robot having a moving function. The output means outputs to the robot an instruction to move to the table corresponding to the image displayed by the display means in response to the instruction.

The block diagram which shows the structure of the order processing system of this embodiment. The block diagram which shows the main part structure of the hall controller in this embodiment. The block diagram which shows the main part structure of the order receiving terminal in this embodiment. The block diagram which shows the main part structure of the table robot in this embodiment. The figure which shows the appearance of the order receiving terminal and the table robot in this embodiment. The flowchart which shows the processing of a hall controller and a table robot in 1st Embodiment. The figure which shows an example of the screen which shows the seat situation in 1st Embodiment. The figure which shows an example of the detail screen (seat situation detail) in 1st Embodiment. The figure which shows an example of the selection screen of the floor robot 26 in 1st Embodiment. The figure which shows an example of the screen which shows the seat situation in 1st Embodiment. The flowchart which shows the processing of the handy terminal and the floor robot in 1st Embodiment. The flowchart which shows the processing of the table robot and the hall controller in 2nd Embodiment. A flowchart showing the comparison process by the hall controller. The figure which conceptually shows the 1st image and the 2nd image. The flowchart which shows the processing of the handy terminal and the floor robot in 2nd Embodiment. The flowchart which shows the processing of the table robot in 3rd Embodiment. The flowchart which shows the processing of the handy terminal and the floor robot in 3rd Embodiment.

Hereinafter, this embodiment will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of an order processing system according to the present embodiment. The order processing system is installed and operates in a restaurant, for example.

The order processing system includes an order station 10, a POS (Point Of Sales) terminal 12, a kitchen printer 14, a hall controller 16, and at least one plurality of access points (AP) 18 (18-1, ..., 18-m). They are connected to each other via a network 19 such as a LAN (Local Area Network). In addition, the order processing system is a tablet terminal 16K, a plurality of handy terminals 20 (20-1, ..., 20-n), and a plurality of units installed for each table in the store via wireless communication with the access point 18. Order receiving terminal 22 (22-1, ..., 22-x), a plurality of table robots 24 (24-1, ..., 24-x), and at least one floor robot 26 (26-1, ..., 26). -Y) is connected to the network 19.

The order station 10 transmits a cooking instruction to the kitchen printer 14 based on the order information received from the handy terminal 20 or the order receiving terminal 22. When the serving completion is received from the handy terminal 20, the order information is transmitted to the POS terminal 12 as sales information.

The POS terminal 12 is arranged in, for example, a checkout counter and pays for the food and drink of the customer. The POS terminal 12 registers and processes sales information, and performs accounting processing for each transaction. The sales information includes information on the sales details of the product for each customer.

The kitchen printer 14 is arranged, for example, in a kitchen. The kitchen printer 14 outputs a cooking instruction by printing an order slip including an item (menu item) ordered by the customer and an identification number of a table based on the order information received from the order station 10.

The hall controller 16 is a computer that operates as an information processing device that performs processing for controlling and managing the floor robot 26. The hall controller 16 manages the situation of customers in the store and controls each device constituting the order processing system according to the situation of customers. The hall controller 16 manages, for example, the status of the table in which the customer is seated, customer information (number of people, gender, age, occupation, etc.), management of order history, usage time (elapsed time from seating, end time of course setting, etc.). (Remaining time until, etc.) is managed. The hall controller 16 displays a screen showing the situation of the customer. The store employee visually recognizes the screen of the hall controller 16. Further, the hall controller 16 inputs image pickup data captured with at least a range including a table on which products (drinks, food, etc.) are arranged as an image pickup range, and controls the operation of the floor robot 26 based on the image pickup data. Perform the processing for. The hall controller 16 inputs, for example, image pickup data captured by a camera provided in a table robot 24 (or an order reception terminal 22 may be installed) installed on a table. The captured data may be either a still image or a moving image (video). The hall controller 16 displays an image (or image) based on the captured data, and gives an instruction to the floor robot 26 to control the operation in response to an input operation by the employee who confirmed the image (or image). It can be output (first embodiment). Further, the hall controller 16 detects the state of the product, for example, the remaining amount of the product, and the state of the container (glass, dish, etc.) used for providing the product by performing image processing on the input imaging data. The hall controller 16 can output an instruction to control the operation to the floor robot 26 at an appropriate timing based on the detection result (second embodiment).

The tablet terminal 16K is used as an input / output device for the hall controller 16. The tablet terminal 16K communicates with the hall controller 16 via the access point 18 and the network 19. The tablet terminal 16K may be configured to directly communicate with the hall controller 16 without going through the access point 18. Further, the tablet terminal 16K is not limited to one, and a plurality of tablets may be provided.

Further, although FIG. 1 shows an example in which the hall controller 16 is realized by a computer connected to the network 19, the processing of the hall controller 16 described later may be executed by the tablet terminal 16K. Further, one hall controller 16 (or tablet terminal 16K) may not only execute the processing described later independently, but may also execute the processing in cooperation with another device (a plurality of computers or an order station 10). good. Therefore, in the present embodiment, the hall controller 16 may be in either a form realized by a computer connected to the network 19 or a form realized by the tablet terminal 16K.

The access point 18 is a wireless communication device that controls wireless communication with the tablet terminal 16K, the handy terminal 20, the order receiving terminal 22, the table robot 24, and the floor robot 26. At least one access point 18 is arranged, for example, on the ceiling in the store.

The handy terminal 20 is a terminal for a clerk. The handy terminal 20 inputs an order, a check-in process for registering a customer who has visited the store, an input for completing the serving of the product ordered by the customer, and outputs a message in response to a notification from the hall controller 16. The handy terminal 20 transmits to the order station 10 the order information according to the input order, the product information indicating the product whose serving is completed, and the like. Further, the handy terminal 20 can transmit various notifications to the hall controller 16 according to the operation of the clerk or the like.

The order receiving terminal 22 (22-1, ..., 22-x) is arranged, for example, on each table T (T-1, ..., Tx) in the store. The order receiving terminal 22 displays a menu and accepts an order for a product (menu item) from a customer. The order receiving terminal 22 transmits the input order information to the order station 10. The order receiving terminal 22 displays a screen of a recommended product or the like according to the recommendation information output from the hall controller 16.

The table robot 24 (24-1, ..., 24-x) is arranged, for example, on each table T (T-1, ..., Tx) in the store. The table robot 24 is a robot for communicating with customers. The table robot 24 has an output function of inputting image pickup data captured by a camera over a range including a table on which products (drinks, food, etc.) are arranged as an image pickup range and transmitting the image pickup data to the hall controller 16. Further, the table robot 24 has a function of recognizing the voice input from the microphone, and has a simple conversation with the customer, accepts an order for a product by the voice from the customer, and the like. The table robot 24 is provided with a voice output function for outputting a message to the customer from the speaker, and outputs a response message corresponding to the voice recognized by the voice recognition function for conversation with the customer, the hall controller 16 or the order station. Output a message or the like in response to the notification from 10.

Although the order receiving terminal 22 and the table robot 24 are realized as separate devices in FIG. 1, they can also be realized by one device.

Further, the function of controlling the floor robot 26 provided in the hall controller 16 can also be realized by the order receiving terminal 22 or the table robot 24. That is, the order receiving terminal 22 or the table robot 24 (further, a configuration in which the order receiving terminal 22 and the table robot 24 are combined) is operated as an information processing device that performs processing for controlling and managing the floor robot 26. Also good (third embodiment).

The floor robot 26 is a robot used for serving products to each table in a store and collecting (bashing) items such as tableware, glasses, and garbage from each table. The floor robot 26 is provided with at least one mounting table on which tableware and the like can be placed. The floor robot 26 is provided with a function of autonomously traveling, and can travel to a designated target table position according to a route set according to the arrangement of tables in the store. The floor robot 26 is provided with, for example, a collision prevention sensor, and travels while avoiding obstacles.

The floor robot 26 is provided with a wireless communication function via an access point 18, for example, an instruction input by a customer from an order receiving terminal 22 or a table robot 24, an instruction from a hall controller 16 according to the customer's situation, and the like. Alternatively, it is possible to receive an instruction input by an operation by an employee and perform an operation according to the instruction. Further, the floor robot 26 notifies the hall controller 16 of the position data indicating the current position as the vehicle travels, via the access point 18. Based on the position data received from the floor robot 26, the hall controller 16 displays information indicating the current position (operation status) on the route on which the floor robot 26 travels on a floor map displaying, for example, the table layout of a store. Can be displayed.

The floor robot 26 has an input / output device such as a touch panel, a person authentication function based on an image taken by a camera, a voice recognition function input from a microphone, and a speaker in order to communicate with customers and clerk. A voice output function that outputs a message to a customer or a clerk is provided.

FIG. 2 is a block diagram showing a main configuration of the hall controller 16 in the present embodiment.
The hall controller 16 is realized by a computer. The hall controller 16 includes a CPU (Central Processing Unit) 16A, a ROM (Read Only Memory) 16B, a RAM (Random Access Memory) 16C, a storage device 16D, a display device 16E, an input device 16F, a touch panel 16G, a communication device 16H, and the like. ..

The CPU 16A controls each configuration of the hall controller 16 by executing a program stored in the ROM 16B or the storage device 16D. The program executed by the CPU 16A includes an information processing program for operating as an information processing device in addition to a basic program (OS (Operating System)). The CPU 16A manages the situation of customers in the store by executing an information processing program. The CPU 16A has an input function for inputting image pickup data transmitted from the table robot 24 (or order reception terminal 22) through the communication device 16H, and performs image processing based on the input image pickup data to produce products (drinks, food). ), And an output function that outputs an instruction for controlling the floor robot 26 according to the detection result of the detection function are realized. As the status of the product detected by the detection function, it is possible to detect the remaining amount of the product, the state of the container (glass, plate, etc.) used to provide the product, and the like. As for the state of the container used to provide the product, it is desirable to detect a state in which the customer does not need the container, for example, the dishes are stacked or placed outside the table.

Further, the CPU 16A manages, for example, the status of the seated table, customer information (number of people, gender, age, occupation, etc.), management of order history, and usage time (elapsed time from seating, until the end time of course setting). Based on information such as (remaining time, etc.), the recommendation information extraction process for determining the recommendation information according to the customer is executed.

The ROM 16B stores a program or the like executed by the CPU 16A. The RAM 16C forms various working memory areas. The storage device 16D is, for example, an HDD (Hard disk drive), an SSD (Solid State Drive), or the like, and stores various programs and various data.

The display device 16E is, for example, an LCD (Liquid Crystal Display). The input device 16F is a pointing device such as a keyboard and a mouse. The touch panel 16G inputs by display and touch operation.

The communication device 16H controls communication with each device through, for example, the network 19. Further, the communication device 16H may include a function of wireless communication.

FIG. 3 is a block diagram showing a configuration of a main part of the order receiving terminal 22 in the present embodiment.
The order receiving terminal 22 is realized by, for example, a tablet-type computer. The order receiving terminal 22 includes a CPU 22A, a ROM 22B, a RAM 22C, a storage device 22D, a touch panel 22G, a communication device 22H, and the like.

The CPU 22A controls each configuration of the order receiving terminal 22 by executing a program stored in the ROM 22B or the storage device 22D. The program executed by the CPU 22A includes an order acceptance program for operating as an order acceptance terminal 22 in addition to a basic program (OS (Operating System)). By executing the order acceptance program, the CPU 22A displays the menu screen, accepts the customer's input operation, sends the order information according to the product selected from the menu screen by the customer's input operation to the order station 10, and halls. Processing such as notification (display or voice output) of recommended products according to the recommendation information transmitted from the controller 16 (or the order station 10) is executed. The CPU 22A displays a screen (GUI (Graphical User Interface)) corresponding to each process on the touch panel 22G.

The ROM 22B stores a program or the like executed by the CPU 22A. The RAM 22C forms various working memory areas. The storage device 22D is, for example, an HDD, an SSD, or the like, and stores various programs and various data. The touch panel 22G performs display and input by touch operation. The communication device 22H controls wireless communication with the access point 18.

The order receiving terminal 22 can be provided with the same function as the table robot 24 described later, and the installation of the table robot 24 can be omitted. In this case, the order receiving terminal 22 is provided with a camera, a microphone, and a speaker, and is provided with the same control function as the table robot 24 described later for each.

FIG. 4 is a block diagram showing a main configuration of the table robot 24 in the present embodiment.
The table robot 24 is provided with a computer function. The table robot 24 includes a CPU 24A, a ROM 24B, a RAM 24C, a storage device 24D, a microphone 24E, a speaker 24F, a camera 24G, a communication device 24H, a drive mechanism 24L, and the like.

The CPU 24A controls each configuration of the table robot 24 by executing a program stored in the ROM 24B or the storage device 24D. The program executed by the CPU 24A includes a control program for operating as a table robot 24 in addition to a basic program (OS (Operating System)). The CPU 24A controls the operation of the camera 24G by executing the control program, and realizes an output function of outputting the image pickup data captured by the camera 24G to the hall controller 16 through the communication device 24H. Further, the CPU 24A controls the operation for communicating with the customer by executing the control program. Communication with customers includes voice (conversation) and movement. The CPU 24A executes a voice recognition function of the voice input from the microphone 24E and a voice output function of outputting the voice from the speaker 24F in order to communicate with the customer by voice (conversation). The CPU 24A can recognize the voice for conversation from the customer or the voice for ordering the product by the voice recognition function. In the voice recognition function, for example, a predetermined voice command (word or sentence) can be recognized and processing according to the voice command can be executed. Further, the CPU 24A outputs a response message corresponding to the voice for conversation recognized by the voice recognition function by the voice output function, or outputs a message according to the notification from the hall controller 16 or the order station 10. can do.

The ROM 24B stores a program or the like executed by the CPU 24A. The RAM 24C forms various working memory areas. The storage device 24D is, for example, an HDD, an SSD, or the like, and stores various programs and various data.

The camera 24G executes imaging under the control of the CPU 24A and outputs imaging data. The camera 24G can capture a still image or a moving image (video). The communication device 24H controls wireless communication with the access point 18.

The CPU 24A operates the drive mechanism 24L. The table robot 24 is configured by combining a plurality of housings (for example, an upper housing and a lower housing), and a part of the housings is configured to be moved by a drive mechanism 24L. The CPU 24A drives the drive mechanism 24L in a steady state, for example, for communication with a customer, in accordance with the timing of voice conversation, product order, and output of various message voices. Further, the CPU 24A can drive the drive mechanism 24L so as to change the direction of the housing provided with the camera 24G in order to change the imaging range by the camera 24G.
The drive mechanism 24L is a mechanical structure for moving a part of the housing constituting the table robot 24.

FIG. 5 is a diagram showing the appearance of the order receiving terminal 22 and the table robot 24 in the present embodiment. FIG. 5 shows an example in which the order receiving terminal 22 and the table robot 24 are arranged side by side on the table.
The order receiving terminal 22 is basically installed in a cradle on the table, and can be removed from the cradle and used by the customer as needed.

Since the table robot 24 is used for communicating with the customer, it is desirable to install the table robot 24 at a position close to the customer seated at the table. Further, the table robot 24 performs imaging with at least a range including a table on which products (drinks, food, etc.) are arranged as an imaging range. For example, the table robot 24 may have an imaging range not only on the table on which the product is placed but also including a customer seated at the table. Therefore, the table robot 24 is installed at a position where the camera 24G can take an image in a target imaging range.

In the table robot 24 shown in FIG. 5, for example, the camera 24G can be installed in the upper housing, and the camera 24G can perform imaging while operating the upper housing left, right, up and down within a predetermined range by the drive mechanism 24L. .. This makes it possible to set a wider range as the imaging range than when the direction of the camera 24G is fixed.

Next, the operation of the order processing system in this embodiment will be described.
In the order processing system, when a customer who visits the store sits at one of the tables, a check-in process for registering the customer is executed. In the check-in process, table numbers are registered, customer information (number of people, gender, age, occupation, etc.), order course (multiple products to be offered, time, etc. are predetermined) are input. The customer information may be input by, for example, a clerk who owns the handy terminal 20 by operating the handy terminal 20 or by operating the order receiving terminal 22 by the customer.

Further, by transmitting the image (image data) taken by the camera of the table robot 24 to the hall controller 16 and displaying the image on the hall controller 16 (or tablet terminal 16K), the customer information is provided by the clerk who confirmed the image. You may be asked to enter. Further, the hall controller 16 may execute a person authentication process on the image captured by the customer to detect the number of people, gender, age, occupation, etc., and input it as customer information.

When ordering a product after sitting at the table, the customer can inform the clerk who owns the handy terminal 20 and use the order receiving terminal 22. The order receiving terminal 22 displays a menu on the touch panel 22G, and transmits the order information corresponding to the product designated by the touch operation from the menu to the order station 10. As for the menu, a normal menu for classifying and presenting all products, a recommended menu including specific recommended products prepared in advance on the store side, and the like are switched and displayed according to a customer's input operation.

The ordered product is carried to the table by the clerk and also by using the floor robot 26. In the floor robot 26, for example, a product ordered on a mounting table, such as a drink glass or a food plate, is placed in response to an input operation by an employee or an instruction from the hall controller 16 (or a table robot 24). After that, by being instructed to serve the table, it is possible to run by itself and carry it to the instructed table.

The hall controller 16 can confirm the situation of the table in which the customer is seated by the image (video) captured by the camera 24G of the table robot 24. In the present embodiment, the hall controller 16 determines the eating and drinking status of the served products (drinks, food), that is, the status of the consumption of the products by the customer based on the image (image) captured, and at an appropriate timing. In order to collect (bash) items such as tableware, glasses, and garbage, an instruction to move the floor robot 26 to the table on which the image is captured is output. Hereinafter, processes for outputting instructions for controlling the operation of the floor robot 26 (first embodiment, second embodiment, third embodiment) will be described.

(First Embodiment)
FIG. 6A is a flowchart showing the processing of the hall controller 16 in the first embodiment. FIG. 6B is a flowchart showing the processing of the table robot 24 in the first embodiment. When the order receiving terminal 22 substitutes the function of the table robot 24, the order receiving terminal 22 executes the processing of the flowchart shown in FIG. 6 (B).

In the first embodiment, the camera 24G of the table robot 24 (or the order receiving terminal 22) captures an image with an imaging range of at least on the table on which the products are arranged, and this image is captured by a store employee on the hall controller 16. To be able to confirm. The hall controller 16 outputs an instruction to control the operation to the floor robot 26 in response to an input operation by the employee who confirmed the image.

The CPU 16A of the hall controller 16 manages the situation in the store based on the inputs from the handy terminal 20, the order receiving terminal 22, the table robot 24, the floor robot 26, and the like. The CPU 16A can display a screen showing a management status in response to an input operation by an employee.

The CPU 16A displays a screen showing the seat status, including, for example, a floor map (seat map) showing the arrangement of tables in the store, in response to an input operation by the employee (ActA1).

FIG. 7 is a diagram showing an example of a screen D1 showing a seating condition displayed on the display device 16E of the hall controller 16, the touch panel 16G, or the tablet terminal 16K.
On the screen D1 showing the seat status shown in FIG. 7, for example, a switching button D11, a table layout D12, a floor robot mark D13, a table mark D14 (A01 to A10), a table robot mark D15, a communication mark D16, and a table layout D19 are displayed. Will be done. The switching button D11 is for instructing screen switching. For example, in addition to the "seat status" button corresponding to the screen shown in FIG. 7, it corresponds to each of "cooking status", "shift management", and "various tasks". Buttons are included. The table layout D12 shows the arrangement of a plurality of tables within the range to be displayed in the store. FIG. 7 shows an example in which 12 tables of table numbers A01 to A10, B10, and B12 are displayed. The table layout D12 is changed according to the display target range changed by the input operation. The floor robot mark D13 is a mark for indicating the position and operation status of the floor robot 26 traveling in the store. The example shown in FIG. 7 shows that three floor robots 26 are used in the store. The floor robot mark D13 is displayed at a position corresponding to the current position as the floor robot 26 travels in the store. The table marks D14 (A01 to A10, B10, B12) indicate individual tables, and managed customer information and the like are displayed. The table robot mark D15 indicates that the table robot 24 is arranged on the table. The communication mark D16 indicates the number of times the table robot 24 and the customer communicate with each other. Communication between the table robot 24 and the customer includes, for example, a conversation with the table robot 24 (response to an output voice message), an order for a product from the table robot 24, and the like. The table layout D19 shows the arrangement of all the tables in the store, and the range in which the table layout D12 is displayed is specified.

When the CPU 16A of the hall controller 16 detects an operation of selecting any of the table marks D14 (ActA2, Yes), the CPU 16A instructs the table robot 24 installed on the table corresponding to the detected table mark D14 to transmit a video. Is transmitted (ActA3). Further, the CPU 16A causes the display device 16E, the touch panel 16G, or the tablet terminal 16K to display the detailed screen of the corresponding table (ActA4).

On the other hand, when the table robot 24 receives the video transmission instruction from the hall controller 16 (ActB1, Yes), the table robot 24 causes the camera 24G to perform imaging with at least the table on which the product is arranged as the imaging range. The table robot 24 transmits the image pickup data to the hall controller 16 through the communication device 24H, for example, by adding data indicating the table number (ActB2).

The CPU 16A of the hall controller 16 receives an image from the table robot 24 (ActA5) and displays the received image on the detail screen (ActA6).

FIG. 8 is a diagram showing an example of the detailed screen D2 (seat situation details).
The detailed screen D2 shown in FIG. 8 includes, for example, customer management information D21, a video display area D22, and buttons D23 to D29 for selecting a plurality of functions. The customer management information D21 indicates information (order course, remaining time determined by the course) indicating the current status of the customer, including customer information. In the image display area D22, an image corresponding to the image pickup data received from the table robot 24 is displayed. The CPU 16A can change the display target range of the image by operating the image display area D22. For example, in the initial state, among the images captured by the table robot 24, the image in the range on the table is displayed. Next, for example, when there is a touch operation on the image display area D22, an image including the entire table is displayed, and when there is a touch operation, an image including a customer seated at the table is displayed. .. As a result, the employee can display an arbitrary range of images and confirm the customer's situation including the consumption status of the product from the images.

The buttons D23 to D29 for selecting a plurality of functions include, for example, a recommended menu button D23, an additional menu button D24, a last order button D25, an order history button D26, a serving button D27, a bashing button D28, and an OK button D29. The recommended menu button D23, the additional menu button D24, and the last order button D25 are buttons for outputting the recommendation information to the order receiving terminal 22 in response to an input operation by the employee. The order history button D26 is, for example, a button for displaying a list of order history for today.

The serving button D27 is a button for instructing the floor robot 26 to serve the product. The bashing button D28 is a button for instructing the floor robot 26 to move to a designated table in order to collect (bash) tableware, glasses, dust, etc. from the table. The OK button D29 is a button for instructing the end of the display of the detail screen D2.

When the CPU 16A of the hall controller 16 detects an operation for a button provided on the detail screen D2, the input operation is a button for instructing the operation of the floor robot 26, that is, one of the serving button D27 and the bashing button D28. Determine if it is. Here, when it is determined that the operation is for another button (ActA7, A9, No), the CPU 16A executes another process according to the operated button and ends the display of the detail screen D2 (the display of the detail screen D2 is finished). ActA10).

On the other hand, when the operation (cleanup instruction) for the bashing button D28 is detected (ActA9, Yes), the CPU 16A instructs the floor robot 26 to move to the table currently selected as the image display target. , Instruct movement for collection (bashing) of goods (ActA8).

When a plurality of floor robots 26 are provided, the CPU 16A selects one of the floor robots 26 based on the priority set in advance for the plurality of floor robots 26, and the selected floor robots 26. An instruction may be output to 26.

Further, it may be possible to select one from a plurality of floor robots 26 by an input operation by an employee. In this case, for example, in response to the operation of the bashing button D28, the CPU 16A causes the display device 16E, the touch panel 16G, or the tablet terminal 16K to display a selection screen for selecting the floor robot 26.

FIG. 9 is a diagram showing an example of the selection screen D3 of the floor robot 26.
In the selection screen D3 shown in FIG. 9, the floor robot mark D31 corresponding to the selectable floor robot 26 and the table mark D32 corresponding to the table to which the floor robot 26 is moved are clearly shown. Further, as the operation guide message D33, for example, "Please select a robot to go to bashing" is displayed. The floor robot mark D31 is not selectable for the floor robot 26 that is moving or has received other instructions earlier, and the floor robot mark D31 corresponding to the floor robot 26 is not displayed.

When any floor robot mark D31 is selected by an employee from the selection screen D3, the hall controller 16 moves the table and articles to be moved to the floor robot 26 corresponding to the selected floor robot mark D31. Instruct the collection (bashing) of.

When the CPU 16A outputs an instruction to control the operation to the floor robot 26, the CPU 16A notifies the handy terminal 20 possessed by the clerk that the instruction is output to the floor robot 26.

Similarly, when the operation (serving instruction) for the serving button D27 is detected (ActA7, Yes), the CPU 16A sets the floor robot 26 to the table currently selected as the image display target as the moving destination. Instruct the movement for serving the goods (ActA8).

When the serving button D27 or the bashing button D28 is selected on the detail screen D2, the CPU 16A is in a state where the floor robot 26 cannot be used with respect to the handy terminal 20 if there is no operable floor robot 26. Notify that. As a result, the store clerk can promptly respond on behalf of the floor robot 26.

Further, when the CPU 16A of the hall controller 16 displays the screen D1 (floor map) showing the seat status (ActA1), if there is an operating floor robot 26, the operating status of the floor robot 26 is displayed on the floor map. ..

FIG. 10 is a diagram showing an example of a screen D4 showing a seating situation in which the operating state of the floor robot 26 is displayed.
The screen D4 showing the seat status shows that, for example, one floor robot 26 (R3) is in operation for collecting (bashing) articles. In the floor robot mark D41, the text "bashing" indicating the purpose of operation is added in association with the floor robot 26 (R3) in operation. In addition, the text of "serving" is added during the operation for serving. Further, the route D44 to the table mark D42 of the table (A09) to be moved is displayed, and the floor robot mark D43 indicating the current position of the floor robot 26 (R3) is displayed. The display position of the floor robot mark D43 is changed based on the position data indicating the current position received from the floor robot 26 (R3), and indicates the current position of the floor robot 26 (R3) in the store.

Thereby, by checking the screen D4 showing the seat status displayed on the hall controller 16, the employee can easily grasp the operating status of the floor robot 26.

FIG. 11A is a flowchart showing the processing of the handy terminal 20 in the first embodiment. FIG. 11B is a flowchart showing the operation of the floor robot 26 in the first embodiment.

When the handy terminal 20 receives an instruction from the hall controller 16 (ActC1, Yes), the handy terminal 20 outputs according to the received content (ActC2). For example, the handy terminal 20 causes the display device to display a pop-up screen and displays the content of instructions from the hall controller 16. Further, the handy terminal 20 is not limited to the display as an output form, and the store clerk may be made to recognize the reception from the hall controller 16 by sound output, vibration generation, or the like.

The handy terminal 20 can display, for example, information for identifying the floor robot 26 in operation (for example, “R3”), a table number of a table to be moved to, a moving purpose “bashing”, and the like.

Further, when the hall controller 16 notifies that the floor robot 26 cannot be used, the handy terminal 20 outputs a message or the like requesting that the clerk respond on behalf of the floor robot 26. In this case, in the same manner as described above, the table number, the purpose of movement "bashing", etc. are displayed so that the clerk can respond promptly.

The handy terminal 20 ends the process (ActC3, Yes) when the input operation by the clerk is received for the display according to the instruction from the hall controller 16. For example, the clerk confirms the information of the movable floor robot 26 and inputs a response indicating that the confirmation has been completed. Further, when it is notified that the floor robot 26 is not available, the clerk inputs a response indicating that he / she responds or a response requesting support from another clerk (handy terminal 20). ..

On the other hand, when the floor robot 26 receives an instruction from the hall controller 16 (ActD1, Yes), the floor robot 26 sets a route from the current position to the target table based on the table number indicating the movement destination, and controls the movement. To start. The floor robot 26 notifies the hall controller 16 of the position data indicating the current position as the vehicle travels in the store via the access point 18.

When the floor robot 26 moves to the position of the target table, the floor robot 26 outputs, for example, a message according to the purpose to the customer.

For example, in the case of serving purposes, the floor robot 26 outputs, for example, a voice message "I'm sorry to have kept you waiting. Please take food (drinks)" to display the products (food, drinks) on the stand. Encourage them to take. In addition to voice messages, it is also possible to display the message on a display device. Similarly, for the purpose of collection, the floor robot 26 outputs, for example, a voice message "Please put an unnecessary glass or plate if you have one" so that the unnecessary item can be put on the mounting table. prompt.

When the floor robot 26 receives an input operation by the customer (ActD3, Yes), for example, the floor robot 26 returns to the initial position or starts processing to the next target table position (ActD4). For example, the floor robot 26 accepts an operation indicating that the food (drink) has been received at the time of serving, and receives an operation indicating that an unnecessary item at the time of collection has been placed on the mounting table. The floor robot 26 may input an operation from a customer by, for example, an operation on a touch panel, or may input by a voice command (voice recognition).

In the above description, the floor robot 26 is instructed to either serve or clean up, but it is also possible to instruct the floor robot 26 to serve and clean up at the same time. For example, the product is moved to the serving table, transported to the target table, and then moved to another table for cleaning up. It is also possible to specify multiple tables as destinations for tidying up. As a result, the floor robot 26 can be operated efficiently.

In this way, in the order processing system of the first embodiment, the employee confirms the image taken by the table robot 24 installed on each table on the hall controller 16, and the image is displayed on the floor robot 26. It is possible to instruct the table selected as the display target. In the hall controller 16, a table can be arbitrarily selected on the screen D1 showing the seat status, and an image can be displayed in the image display area D22 of the detail screen D2. Therefore, the floor robot 26 that moves in the store can be moved at an appropriate timing by checking the consumption status of the product, the state of the customer, and the like for each table. In the order processing system, the hall controller 16 outputs various instructions to the plurality of floor robots 26, and the operation status is collectively displayed on the screen D4 showing the seat status, so that management can be facilitated. can.

(Second Embodiment)
FIG. 12A is a flowchart showing the processing of the table robot 24 in the second embodiment. FIG. 12B is a flowchart showing the processing of the hall controller 16 in the second embodiment. When the order receiving terminal 22 substitutes the function of the table robot 24, the order receiving terminal 22 executes the processing of the flowchart shown in FIG. 12 (A).

In the second embodiment, the camera 24G of the table robot 24 (or the order receiving terminal 22) captures an image in which at least the table on which the products are arranged is the imaging range, and the served products (drinks, food) are eaten and eaten. The situation, that is, the situation of the consumption of the product by the customer (the state where the remaining amount is low or there is no remaining amount) is detected based on the captured image. Then, based on this detection result, an instruction for collecting (bashing) the article is output to the floor robot 26 at an appropriate timing.

After the product is served on the table, the table robot 24 executes imaging by the camera 24G (ActE1), and adds the imaging data of the captured image, for example, data indicating the table number, to the communication device 24H. It is transmitted to the hall controller 16 through (ActE2).

When the CPU 16A of the hall controller 16 receives the image pickup data transmitted from the table robot 24 (ActF1, Yes), the CPU 16A displays an image based on the image pickup data. For example, the CPU 16A is displayed in the video display area D22 of the detail screen D2 in the same manner as in the first embodiment (ActF2). The CPU 16A extracts, for example, a first image from the video received from the table robot 24 after the product is served, and stores it in the RAM 16C or the storage device 16D in association with the table number. Here, the first image captured represents the initial state of the product that has not been consumed by the customer.

Further, when it is determined that a predetermined time has elapsed after storing the first image (ActF3, Yes), the CPU 16A extracts the second image from the video received from the table robot 24 and first. A comparison process for comparing the image and the second image is executed (ActF4).

The predetermined time may be one predetermined fixed time, or may be changed according to the elapsed time from the check-in process for the customer. When changing the predetermined time, for example, when the pace of eating and drinking by the customer is fast, the predetermined time is set to a short time (for example, 3 minutes) in the time zone immediately after the start (for example, 0 to 15 minutes), and the ordering pace of the product is generally set. The closer to the end, the longer the predetermined time is. For example, the time zone of 15 to 30 minutes may be 5 minutes, and the time zone of 30 to 60 minutes may be 10 minutes. As a result, the floor robot 26 can be moved for collection at a more appropriate timing than when the glasses and plates are emptied (drinks and food are consumed), and image processing is performed during the time when the product consumption pace is slow. The processing load may be reduced by reducing the frequency of needing.

FIG. 13 is a flowchart showing a comparison process by the Hall controller 16.
First, the CPU 16A determines whether the first image is stored and the object detection described later for the first image is completed. Here, when the first image is not stored first (ActS1, No), the CPU 16A detects an object image (first object) corresponding to the product provided to the customer from the first image (ActS2), and the first image is stored. 1 The object is stored in the RAM 16C or the storage device 16D (ActS3).

FIG. 14A is a diagram conceptually showing the first image. As shown in FIG. 14A, the first image taken by the camera 24G of the table robot 24 may include an image corresponding to each of a plurality of products. The CPU 16A detects an image corresponding to each product as an object, assigns a different identification label to each object, and stores the image. As an image processing method for detecting an object in an image, for example, a method of registering feature data indicating features such as the shape and color of the object to be detected in advance and detecting the object based on the feature data, etc. Can be used. Other existing image processing methods can be used.

In the example shown in FIG. 14A, for example, objects OB1 and OB2 corresponding to drinks (mugs, glasses) and objects OB3, OB4 and OB5 corresponding to food are detected.

Further, the CPU 16A detects an object image (second object) corresponding to the product provided to the customer from the second image in the same manner as the first image (ActS4), and stores the second object in the RAM 16C or the storage device 16D. Let me.

FIG. 14B is a diagram conceptually showing the second image. The example shown in FIG. 14B shows an example in which objects OB1 to OB5 are detected in the same manner as in the first image. However, as shown in FIG. 14 (B), since the second image is extracted after a predetermined time has elapsed from the imaging of the first image, the customer eats and drinks and the product is consumed. In addition, the positions of the objects OB1 and OB2 corresponding to drinks (mugs, glasses) on the table have been changed by the customer. The second image means an image in which the same range in which the first image is captured is captured again after a predetermined time has elapsed from the time when the first image was captured.

The CPU 16A associates a plurality of first objects included in the first image with a plurality of second objects included in the second image in order to detect a change (remaining amount) in the situation for each product (ActS5). .. That is, it is possible to detect a change in the situation for each product even if the position on the table of the product is different between the time of capturing the first image and the time of capturing the second image.

The CPU 16A compares each combination of the associated first object and the second object and extracts the difference (ActS6). That is, the change between the state of the product at the time of capturing the first image and the state of the product at the time of capturing the second image is detected.

The CPU 16A compares the difference for each combination of the first object and the second object with the preset reference value (ActS7), and determines whether the difference exceeds the reference value (ActS8). The reference value is a value for determining that the consumption of the product is sufficiently high, in other words, the remaining amount of the product is less than the preset predetermined value, or there is no remaining amount. That is, it is a value for determining whether there is an empty vessel that will be emptied soon or there is an empty vessel to be collected. It should be noted that the portion corresponding to the remaining amount of the product may be detected based on the difference between the first object and the second object. In this case, a reference value for determining the remaining amount is set, and the reference value is compared with the remaining amount to determine that the remaining amount of the product is less than the predetermined value or there is no remaining amount. .. Further, the reference value may be set for each type of object (drink, food), or may be set for each product as long as the product can be specified when the object is detected.

Based on the result of the comparison process, the CPU 16A determines whether or not the notification condition for operating the floor robot 26 has been reached. That is, by comparing the first object and the second object, if there is an object (product) whose difference exceeds the reference value, it is determined that the notification condition has been reached. The CPU 16A may determine that the notification condition is reached when the difference of at least one object exceeds the reference value, or when the difference of a predetermined number of objects exceeds the reference value. It may be determined that the notification condition has been reached.

In the comparison process described above, the difference between the first image (first object) and the second image (second object) is extracted, and the product status is determined based on this difference. A method may be used to determine the status of the product. For example, for each of the plurality of products, the feature amount of the image obtained by capturing the product is registered in advance as dictionary data, and the dictionary data is compared with the image of each object extracted from the second image. Then, it may be determined whether the similarity between the image of the object and the dictionary data is below a predetermined reference value, that is, whether the product is in a sufficiently high consumption state.

Here, when it is determined that the notification condition has not been reached (ActF5, No), the CPU 16A performs the same as described above after a predetermined time has elapsed since the previous extraction of the second image (ActF3, Yes). , The second image is extracted from the video, and it is determined by comparing the first image and the second image whether the notification condition for operating the floor robot 26 is reached (Act F4, F5).

In the second and subsequent comparison processes, since the first image has already been received (FIG. 13, ActS1, Yes), the CPU 16A performs the process of detecting the first object from the first image (ActS2, S3). Omit. However, when the additional product is served, the first image is stored and the first object is extracted after that. In this case, the CPU 16A extracts the object corresponding to the added product from the first image and stores it in the same manner as the other objects. Further, regarding the second object lost from the second image, it is assumed that the goods have been consumed and collected, and the first object corresponding to the second object is also deleted.

When it is determined that the notification condition has been reached (ActF5, Yes), the CPU 16A is currently targeted for displaying an image on the floor robot 26, assuming that there is a device with a low remaining amount or no remaining amount. In addition to instructing the table selected as the move destination as the move destination, the call information for instructing the move for collecting (bashing) the article is output (ActF6). When a plurality of floor robots 26 are provided, the CPU 16A selects one of the floor robots 26 based on the priority set in advance for the plurality of floor robots 26, and the selected floor robots 26. Call information is output to 26.

When the CPU 16A outputs the call information to the floor robot 26, the CPU 16A outputs the call information to the handy terminal 20 owned by the clerk in the same manner as in the first embodiment. If there is no operable floor robot 26, the CPU 16A notifies the handy terminal 20 that the floor robot 26 cannot be used.

As a result, the hall controller 16 detects that the floor robot 26 is in a state of being sent to collect the article based on the image received from the table robot 24 without any input operation by the employee. It is possible to instruct the floor robot 26.

FIG. 15A is a flowchart showing the processing of the handy terminal 20 in the second embodiment. FIG. 15B is a flowchart showing the operation of the floor robot 26 in the second embodiment.

When the handy terminal 20 receives the call information from the hall controller 16 (ActG1, Yes), the handy terminal 20 outputs according to the reception (ActC2). For example, the handy terminal 20 causes the display device to display a pop-up screen and displays the content of instructions from the hall controller 16. Further, the handy terminal 20 is not limited to the display as an output form, and the store clerk may be made to recognize the reception from the hall controller 16 by sound output, vibration generation, or the like. It should be noted that the operation of the handy terminal 20 is performed in the same manner as at the time of collection (bashing) in the first embodiment, and detailed description thereof will be omitted.

On the other hand, when the floor robot 26 receives the call information from the hall controller 16 (ActH1, Yes), the floor robot 26 receives the position information (table number indicating the movement destination) and sets a route from the current position to the target table. , Start movement control. It should be noted that the operation of the floor robot 26 is performed in the same manner as at the time of collection (bashing) in the first embodiment, and detailed description thereof will be omitted.

In this way, in the order processing system of the second embodiment, the remaining amount of the products on the table is low or there is no remaining amount based on the images taken by the table robot 24 installed on each table. Upon detection, it is possible to output an instruction for collecting (bashing) the article to the floor robot 26. Therefore, the floor robot 26 can be moved for collection at an appropriate timing according to the consumption status of the product for each table without increasing the burden on the employee.

(Third Embodiment)
FIG. 16 is a flowchart showing the processing of the table robot 24 (or the order receiving terminal 22) in the third embodiment. FIG. 17A is a flowchart showing the processing of the handy terminal 20 in the third embodiment. FIG. 17B is a flowchart showing the operation of the floor robot 26 in the third embodiment.

In the third embodiment, the processing of the hall controller 16 described in the second embodiment is executed by the table robot 24. That is, the table robot 24 is operated as an information processing device that performs processing for controlling and managing the floor robot 26. Instead of the table robot 24, the order reception terminal 22 or the order reception terminal 22 and the table robot 24 may be combined for execution.

After the product is served on the table, the CPU 24A of the table robot 24 executes imaging by the camera 24G (ActJ1) and stores the first image in the RAM 16C or the storage device 16D in association with the table number.

When it is determined that a predetermined time has elapsed after storing the first image (ActJ2, Yes), the CPU 24A captures the second image (ActJ3) and compares the first image with the second image. The comparison process is executed (ActF4). It should be noted that the comparison process is performed in the same manner as in the second embodiment, and detailed description thereof will be omitted.

Similar to the second embodiment, when it is determined that the notification condition for operating the floor robot 26 is reached based on the processing result of the comparison processing (ActJ5, Yes), the CPU 24A has the same with respect to the floor robot 26. Then, the call information for instructing the movement for the collection (bashing) of the article is output (ActJ6). At this time, the CPU 24A instructs the table on which the own table robot 24 is installed as the moving destination. When a plurality of floor robots 26 are provided, the CPU 16A selects one of the floor robots 26 based on the priority set in advance for the plurality of floor robots 26, and the selected floor robots 26. Call information is output to 26.

When the CPU 24A outputs the call information to the floor robot 26, the CPU 24A outputs the call information to the handy terminal 20 owned by the clerk in the same manner as in the second embodiment.

Further, the table robot 24 may output the call information to the hall controller 16 instead of directly outputting the call information to the floor robot 26. In response to the call information from the table robot 24, the hall controller 16 instructs the floor robot 26 to move the floor robot 26 for collecting (bashing) the goods with the table from which the call information is transmitted as the move destination. Is output. In this case, when a plurality of floor robots 26 are provided, the hall controller 16 selects one of the floor robots 26 based on the priority set in advance for the plurality of floor robots 26, and selects one of the floor robots 26. It is possible to output call information to the floor robot 26. Further, when the hall controller 16 receives the call information from the table robots 24 installed in each of the plurality of tables, the hall controller 16 schedules the collection of the articles to the plurality of tables on the floor. You may instruct the robot 26. The hall controller 16 sets the table to be collected and the order of movement based on the arrangement of the tables so that the articles can be efficiently collected at a plurality of tables by, for example, one floor robot 26. , Instruct the floor robot 26.

When the handy terminal 20 receives the call information from the table robot 24 (or the hall controller 16) (ActK1, Yes), the handy terminal 20 outputs according to the reception (ActK2). It should be noted that the operation of the handy terminal 20 is performed in the same manner as at the time of collection (bashing) in the second embodiment, and detailed description thereof will be omitted.

On the other hand, when the floor robot 26 receives the call information from the hall controller 16 (ActM1, Yes), the floor robot 26 sets a route from the current position to the table from which the call information is transmitted, and starts the movement control (ActM2). It should be noted that the operation of the floor robot 26 is performed in the same manner as at the time of collection (bashing) in the second embodiment, and detailed description thereof will be omitted.

In this way, in the order processing system of the third embodiment, the remaining amount of the products on the table is small or the rest is based on the image on the table captured by the table robot 24 installed on each table. It is possible to detect the absence state and output an instruction for collecting (bashing) the article to the floor robot 26. Therefore, the floor robot 26 can be called for collection at an appropriate timing according to the consumption status of the product without the customer performing an input operation to the order receiving terminal 22 or the table robot 24.

In the description of each of the above-described embodiments, it is described that a camera is provided on the table robot 24 or the order receiving terminal 22 to take an image. However, apart from the table robot 24 or the order receiving terminal 22, the periphery of the table (upper part) One or a plurality of cameras may be installed in (including) to take an image on the table or in the vicinity including the customer around the table as the imaging range.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

Further, the process described in the above-described embodiment is a recording medium such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a semiconductor memory, etc., as a program that can be executed by a computer. Can be provided to various devices by writing to. It is also possible to transmit it through a communication medium and provide it to various devices. The computer reads the program recorded on the recording medium or receives the program via the communication medium, and the operation is controlled by the program to execute the above-mentioned processing.

10 ... order station, 12 ... POS terminal, 14 ... kitchen printer, 16 ... hall controller, 18 (18-1, ..., 18-m) ... access point, 20 (20-1, ..., 20-n) ... handy Terminal, 22 (22-1, ..., 22-x) ... Order reception terminal, 24 (24-1, ..., 24-x) ... Table robot, 26 (26-1, ..., 26-y) ... Floor robot ..

Embodiments of the present invention relate to information processing devices and systems .

An object to be solved by the present invention is to provide an information processing device and a system capable of facilitating control of a robot moving in a store.

According to the embodiment, the information processing apparatus includes a receiving means, a first display means, an instruction input means, an output means , and a second display means . The receiving means inputs the imaging data captured in each of the plurality of tables arranged in the store. The first display means displays an image based on the image pickup data corresponding to any of the plurality of tables. The instruction input means inputs an operation instruction corresponding to an operation purpose for a robot having a function of autonomously traveling and moving in the store . The output means outputs to the robot a movement instruction to move to the table corresponding to the image displayed by the display means in response to the operation instruction. The second display means displays information indicating the operation purpose together with the position in the store of the robot that moves in response to the movement instruction output by the output means.

In this way, in the order processing system of the first embodiment, the employee confirms the image taken by the table robot 24 installed on each table on the hall controller 16 and displays the image on the floor robot 26. It is possible to instruct the table selected as the display target. In the hall controller 16, a table can be arbitrarily selected on the screen D1 showing the seat status, and an image can be displayed in the image display area D22 of the detail screen D2. Therefore, the floor robot 26 that moves in the store can be moved at an appropriate timing by checking the consumption status of the product, the state of the customer, and the like for each table. In the order processing system, various instructions are output to a plurality of floor robots 26 by the hall controller 16 and collectively displayed on the screen D4 showing the operation status, so that management can be facilitated.

Claims (5)

An input means for inputting image pickup data captured in each of a plurality of tables arranged in the store, and an input means.
A display means for displaying an image based on the image pickup data corresponding to any of the plurality of tables, and a display means.
An instruction input means for inputting an instruction to a robot having a moving function, and an instruction input means.
An information processing device having an output means for outputting an instruction to move to a table corresponding to an image displayed by the display means to the robot in response to the instruction. The information processing device according to claim 1, wherein the output means instructs the robot to perform at least one operation of serving a product to the table or collecting an article from the table. An input means for inputting image pickup data, with the range in which the products on the table arranged in the store are arranged as the image pickup range, and
A detection means for detecting the status of the product based on the imaged data,
An information processing device having an output means for outputting an instruction to a robot having a moving function when the condition of the product corresponds to a preset condition. The detection means detects a state in which the remaining amount of the product is less than a predetermined value, and
The information processing device according to claim 3, wherein the output means outputs an instruction to the robot to move to a table in which the imaging data is captured when the remaining amount of the product is less than the predetermined value. Computer,
An input means for inputting image pickup data captured in each of a plurality of tables arranged in the store, and an input means.
A display means for displaying an image based on the image pickup data corresponding to any of the plurality of tables, and a display means.
An instruction input means for inputting an instruction to a robot having a moving function, and an instruction input means.
An information processing program for functioning as an output means for outputting an instruction to move to a table corresponding to an image displayed by the display means to the robot in response to the instruction.

Images