JP2020160781A - Voice recognition order system and voice recognition order method - Google Patents

Abstract

To provide a voice recognition order system and a voice recognition order method, in which a smart speaker is used to process orders such as food orders.SOLUTION: A voice recognition order system S is configured to include a smart speaker 31A and an order server 10 connected to the smart speaker 31A via a network so that information can be exchanged. The smart speaker 31A receives voice data obtained when a user speaks in a trigger activated state, recognizes the voice data, authenticates that a specified access keyword has been spoken, and generates text information that identifies user's order content based on the voice data. When the authentication of the access keyword having been spoken is successful, order information including the text information is transmitted to the order server 10.SELECTED DRAWING: Figure 1

Description

The present invention relates to a voice recognition order system and a voice recognition order method for performing order processing using a smart speaker.

In recent years, in restaurants such as restaurants, a system for placing an order from a touch panel type terminal device installed on a table has been adopted. From the menu displayed on the display screen of the terminal device, select the dish and quantity on the touch panel and touch the "Order" button, and the order information will be sent from the terminal device to the order receiving device in the kitchen via the wireless network. It is supposed to be sent. In the order receiving device in the kitchen, cooking is started by presenting the order information to the cook by printing out the order information or displaying it on the display screen of the order receiving device. Some of such touch panel type ordering systems are configured so that accounting processing and aggregation processing can be smoothly performed in conjunction with a conventionally known POS system. Furthermore, a store management system that directly or indirectly interacts with the robot installed at the table seat, judges the customer's situation from the content of the dialogue, and gives operation instructions to the store staff. (Patent Document 1) is also disclosed.

JP-A-2018-1617111

In the conventional touch panel type order system, the customer makes a necessary key touch using the touch panel to open the menu page of the dish, selects the dish, further selects the number, and finally decides the order.
On the other hand, recently, smart speakers that accept and execute operations such as surveys using search engines, reading online news, and playing music and videos by user's voice are on the market. With a smart speaker, the user can perform various operations by speaking without operating the touch panel.
However, in the smart speakers currently on the market and systems using smart speakers, it is not possible to open a desired menu page on the touch panel display screen or order a desired dish from the kitchen simply by placing the smart speaker in the audience seat. For example, when speaking "beer", the smart speaker connects to the AI assistant server that is normally connected, and general search results associated with "beer", for example, "beer is a type of alcoholic beverage. There are various ways to make it, but mainly the malt that has sprouted barley (starch is saccharified by an enzyme (amylase)) is alcohol-fermented with beer yeast, and the optimum solution is returned. So, it doesn't lead to ordering food.

In view of such problems and the like, an object of the present invention is to provide a voice recognition order system and a voice recognition order method for performing order processing such as ordering food using a smart speaker.

The voice recognition ordering system of the present invention includes a smart speaker including at least a voice input means and a voice output means, and a first computer connected to the smart speaker so as to be able to exchange information via a network. In the voice recognition order system, the smart speaker recognizes voice data receiving means for receiving voice data obtained by speaking by a user while the trigger is activated, and voice data received by the voice data receiving means. , An authentication means that authenticates that a predetermined access keyword has been spoken, a text information generation means that generates text information that identifies a user's order content based on the voice data received by the voice data receiving means, and the authentication means. The first computer has an order information transmitting means for transmitting order information including the text information generated by the text information generating means to the first computer when the authentication of the speech of the access keyword is successful. The computer has a reception processing means that executes an order reception process based on the received order information when the order information transmitted by the order information transmission means of the smart speaker is received.

Based on the received order information, the first computer generates instruction information generating means for generating order execution instruction information for instructing execution of processing according to the order contents, and the instruction information generating means. The second computer has an instruction information transmitting means for transmitting the order execution instruction information to a second computer connected to the first computer via a network so as to be able to exchange information. Processing according to the order contents based on the instruction information receiving means for receiving the order execution instruction information transmitted by the instruction information transmitting means of one computer and the order execution instruction information received by the instruction information receiving means. It may be configured to have an order execution means for executing.

In the voice recognition order system including the plurality of smart speakers and the terminal device associated with each of the smart speakers and capable of instructing the operation by the user, the order execution instruction information is the same. The second computer includes ID information for identifying the smart speaker that is the source of the order information, and the second computer is the terminal associated with the smart speaker specified by the ID information included in the order execution instruction information. The device may be configured to have a terminal control means for controlling the device as a control target terminal device based on the order execution instruction information.

The second computer transmits control information to the control target terminal device to instruct the display screen of the control target terminal device to display according to the order content indicated by the order execution instruction information. It may be configured as follows.

The voice recognition ordering method of the present invention is performed by a smart speaker having at least a voice input means and a voice output means, and a first computer connected to the smart speaker via a network so as to exchange information. In the ordering method, the smart speaker recognizes the voice data reception step of receiving the voice data obtained by the user speaking while the trigger is activated and the received voice data, and a predetermined access keyword is spoken. An authentication step for authenticating that the computer has been performed, a text information generation step for generating text information that identifies the user's order content based on the voice data received in the voice data reception step, and an access keyword utterance in the authentication step. The first computer has an order information transmission step of transmitting order information including the text information generated in the text information generation step to the first computer when the authentication is successful. However, when it receives the order information transmitted from the smart speaker, it has a reception processing step of executing an order reception process based on the order information.

When the first computer receives the order information transmitted from the smart speaker, instruction information for generating order execution instruction information instructing to execute a process according to the order contents based on the order information. The instruction information transmission step of transmitting the generation step and the order execution instruction information generated by the instruction information generation step to a second computer connected to the first computer via a network so as to be able to exchange information. And, the instruction information receiving step in which the second computer receives the order execution instruction information transmitted from the first computer, and the order execution instruction information received in the instruction information receiving step. Based on the above, it may be configured to have an order execution step for executing a process according to the order contents.

The order execution instruction information includes ID information for identifying the smart speaker that is the source of the order information, and the second computer corresponds to the plurality of the smart speakers and each of the smart speakers. Among the attached terminal devices that can be operated by the user, the terminal device associated with the smart speaker specified by the ID information included in the order execution instruction information is controlled. The terminal device may be configured to have a terminal control step for controlling based on the order execution instruction information.

Let's send control information to the control target terminal device that the second computer instructs the display screen of the control target terminal device to display according to the order content indicated by the order execution instruction information. It may be configured.

According to the present invention, it is possible to provide a voice recognition order system and a voice recognition order method for performing order processing such as ordering food using a smart speaker.

It is explanatory drawing which shows the configuration example of the voice recognition order system S. It is a block diagram which shows schematic structure of order server 10. It is a block diagram which shows the structure of the store server 30 schematicly. It is a block diagram which shows schematic structure of smart speaker 31A. It is a block diagram which shows the structure of the tablet terminal 31B schematicly. This is an example of the store information DB 1021. This is an example of the utterance scenario DB 1022. This is an example of the audience seat terminal management DB 3021. It is a sequence chart for explaining the processing operation example. It is a sequence chart for explaining the processing operation example. It is a sequence chart for explaining the processing operation example. It is a sequence chart for explaining the processing operation example. It is a sequence chart for explaining the processing operation example. It is a sequence chart for explaining the processing operation example. It is a sequence chart for explaining the processing operation example. This is an example of the hotel information DB1023. This is an example of the utterance scenario DB1024. It is a sequence chart for explaining the processing operation example. It is a sequence chart for explaining the processing operation example.

An embodiment of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and there is no intention of excluding the application of various modifications and techniques not specified below. That is, the present invention can be implemented with various modifications (combining each embodiment, etc.) as long as the effect is exhibited. Further, in the description of the following drawings, the same or similar parts are designated by the same or similar reference numerals. The drawings are schematic and do not necessarily match the actual dimensions and ratios. Even between drawings, parts with different dimensional relationships and ratios may be included.

FIG. 1 is an explanatory diagram showing a configuration example of the voice recognition order system S. Here, a business operator that operates a seat terminal 31 prepared for each seat in a restaurant or the like, an order server 10 as a first computer, a store server 30 as a second computer, and the voice recognition order system S. The service provider device 20 installed by the company and the AI assistant server 40 of the smart speaker 31A are connected to the network by wire or wirelessly, respectively. The voice recognition order system S can be applied to a plurality of restaurants. Each restaurant has multiple seats.

The audience seat terminal 31 includes a smart speaker 31A and an audience seat tablet terminal (hereinafter, referred to as a tablet terminal 31B) used in pairs with the smart speaker 31A. The audience terminal 31 is wirelessly connected to the network via an access point (not shown), and other system components are connected to the network by wire. An access point (not shown) is a wireless device that connects wireless terminals to each other and connects to a network such as a wired network.

In the present embodiment, the order server 10, the service provider device 20, and the AI assistant server 40 are separated from each other as the server-side configuration, but these may be configured on one computer. Each server can be further composed of a plurality of computers.

The voice recognition order system S is connected to a smart speaker 31A including at least a microphone as an example of a voice input means and a speaker as an example of a voice output means, and information can be exchanged with the smart speaker 31A via a network. It is a voice recognition order system S configured to include an order server 10 as an example of the first computer.

The smart speaker 31A recognizes the voice data receiving means for receiving the voice data obtained by the user speaking while the trigger is activated and the voice data received by the voice data receiving means, and a predetermined access keyword is spoken. The authentication means for authenticating, the text information generation means for generating text information that identifies the user's order content based on the voice data received by the voice data receiving means, and the authentication means have successfully authenticated the utterance of the access keyword. If so, it has an order information transmitting means for transmitting the order information including the text information generated by the text information generating means to the order server 10. When the order server 10 receives the order information transmitted by the order information transmitting means of the smart speaker 31A, the order server 10 has a reception processing means that executes an order reception process based on the received order information.

FIG. 2 is a block diagram schematically showing the configuration of the order server 10, FIG. 3 is a block diagram schematically showing the configuration of the store server 30, and FIG. 4 is a block diagram schematically showing the configuration of the smart speaker 31A. , FIG. 5 is a block diagram schematically showing the configuration of the tablet terminal 31B.

<Order server 10>
The order server 10 is mainly a device that receives order information from the smart speaker 31A of each audience seat of each restaurant and executes necessary order acceptance processing.
The order server 10 is a semiconductor memory element such as a control unit 101, a RAM, and a flash memory as the first computer of the present invention, which is composed of a CPU having a calculation function, a work RAM, a ROM for storing various data and programs, and the like. Alternatively, a storage unit 102 having a storage device such as a hard disk or an optical disk, a display unit 103 having a display screen such as a monitor, or an instruction signal corresponding to the instruction received from a business operator operating the voice recognition order system S. Is given to the control unit 101 via an input unit (for example, a keyboard, a mouse, an operation panel (including a touch panel), etc.) 104, various networks (including a LAN (Local Area Network)), an audience terminal 31, and a service provider. It is configured to include a communication unit 105 for communicating with the device 20 and the store server 30 and the like. The components are connected to each other via a bus.

The control unit 101 includes a reception processing means 1011 and an instruction information generating means 1012, and an instruction information transmitting means 1013, and functions as each means of the present invention in cooperation with other members.
The storage unit 102 stores the store information database (DB) 1021, the utterance scenario database (DB) 1022, and the like.

In the store information DB 1021, store information of a store (for example, a restaurant) operated by a company that has introduced the voice recognition order system S is registered (stored). FIG. 6A is an example of the store information DB 1021, and in the case of the example shown in FIG. 6A, "store information" including the destination information of the store server 30 and "access" are associated with the "store ID" that uniquely identifies the store. "Keyword" is registered. All stores are given different store IDs.

The "store information" includes a "tablet terminal ID" that uniquely identifies the tablet terminal 31B of each audience seat terminal 31 installed in each store, and a "tablet terminal ID" for transmitting necessary control information to the tablet terminal 31B. The "tablet terminal destination information", the "smart speaker ID" as an example of the ID information that uniquely identifies the smart speaker 31A, and the "smart speaker destination information" that is the destination that responds to the smart speaker 31A are associated with each other. It is registered.

The utterance scenario DB 1022 stores information on a plurality of utterance scenarios of conversations with the restaurant user through the smart speaker 31A. The utterance scenario is a story of conversations exchanged between the user and the order server 10. A user who has entered a restaurant changes a plurality of stay states between the time he / she enters the restaurant and the time he / she leaves the restaurant. Therefore, in the utterance scenario DB 1022, separate utterance scenarios are accumulated for each transition state.

Each utterance scenario includes user-side scenario data used by the user for dialogue (that is, utterance toward the smart speaker 31A or selection on the tablet terminal 31B), server-side scenario data used by the order server 10 for dialogue, and further. The "information on the next process to be performed" when each "predetermined utterance" is performed is stored in association with each other. For example, after "predetermined utterance: utterance that conveys the number of people", "send order execution instruction information so that the menu is displayed on the display screen of the tablet terminal 31B" as "the next process to be performed". The specific processing will be described later.

FIG. 6B is an example of the utterance scenario DB 1022, and in the case of the example shown in FIG. 6B, the utterance scenario DB 1022 has a “speech scenario” and a “product list” in association with the “store ID” that uniquely identifies the store. It is registered. The store information DB 1021 and the utterance scenario DB 1022 can be updated from the service provider device 20 connected via the input unit 10 or via the network from the business operator operating the voice recognition order system S. It is configured.

<Store server 30>
The store server 30 is a device that mainly receives order execution instruction information from the order server 10 and executes processing according to the order contents, such as control of the tablet terminal 31B. For example, it is installed in the backyard of a restaurant kitchen or the like. For example, it may function as a server of a POS system that manages the order status of each seat in the store.

The store server 30 is a semiconductor memory element such as a control unit 301, a RAM, and a flash memory as a second computer of the present invention, which is composed of a CPU having a calculation function, a work RAM, a ROM for storing various data and programs, and the like. Alternatively, a storage unit 302 having a storage device such as a hard disk or an optical disk, a display unit 303 having a display screen such as a monitor, or an instruction signal corresponding to the instruction received from a clerk in the store is sent to the control unit 301. The order server 10, the audience terminal 31, and the service provider device 20 via an input unit (for example, a keyboard, a mouse, an operation panel (including a touch panel), etc.) 304, and various networks (including a LAN (Local Area Network)). It is configured to include a communication unit 305 for communicating with and the like. The components are connected to each other via a bus.

The control unit 301 includes an instruction information receiving means 3011, an order executing means 3012, and a terminal controlling means 3013, and functions as each means of the present invention in cooperation with other members.
The storage unit 302 stores the audience seat terminal management database (DB) 3021 and the like. FIG. 6C is an example of the audience seat terminal management DB 3021, and in the case of the example shown in FIG. 6C, the “tablet” that uniquely identifies the tablet terminal 31B of each audience seat terminal 31 installed in the store where the store server 30 is installed. A "terminal ID", a "tablet terminal destination information" for transmitting necessary control information to the tablet terminal 31B, a "smart speaker ID" as an example of ID information that uniquely identifies the smart speaker 31A, and a smart speaker. "Smart speaker destination information", which is a destination for responding to 31A, is registered in association with each other. Are registered in association with each other.

<Audience terminal 31>
The audience seat terminal 31 is installed on a table or the like of each audience seat, and is composed of a smart speaker 31A and a tablet terminal 31B.
<Smart speaker 31A>
The smart speaker 31A is a so-called IoT (Internet of Things) device, and performs various information processing in cooperation with the AI assistant server 40. For example, the smart speaker 31A uses voice analysis technology to identify the content of words uttered by the user, provide information and content according to the specified content, and even make orders to various electronic shopping districts. It is a device that can be used.

The smart speaker 31A includes a control unit 311A, a storage unit 312A, a speaker 313A as a voice output means, a microphone 314A as a voice input means, an operation unit 315A, and a communication unit 316A. Further, the housing of the smart speaker 31A is provided with an LED (light emitting element) that visually indicates the status of the smart speaker 31A.

The communication unit 316A is realized by NIC or the like, and is connected to the network by wire or wirelessly. Then, information is transmitted and received between the order server 10, the store server 30, and the AI assistant server 40.

The operation unit 315A is an input device that receives various operations from the user. For example, the operation unit 315A is realized by an operation key or the like provided on the smart speaker 31A.
The smart speaker 31A does not have to have a physical operation unit 315A. For example, the operation unit 315A may receive the voice detected by the microphone 314A as an input instead of the physical operation keys.

The microphone 314A detects various information about the smart speaker 31A. Specifically, the microphone 314A is various sensors, detects a voice emitted by a user and an environmental sound around the smart speaker 31A, and acquires it as voice data.

The storage unit 312A uniquely identifies the connection destination information of the order server 10, the connection destination information of the AI assistant server 40, the "smart speaker ID" as an example of the ID information that uniquely identifies the smart speaker 31A, and the store where the smart speaker is installed. Stores various information such as the "store ID" specified in. The storage unit 312A is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk.

The storage unit 312A stores the voice recognition database (DB) 3121A and the like. The voice recognition DB3121A stores a language that triggers the smart speaker 31A to be activated. When the smart speaker 31A receives a voice utterance that is a trigger from the user through the microphone 314A, the smart speaker 31A goes from the standby state to the activated state. Then, the LED provided in the housing develops a color indicating the activated state.

Further, the voice recognition DB3121A stores predetermined access keyword information for performing authentication for connecting to the order server 10. When the authentication means 3112A of the control unit 311A, which will be described later, authenticates whether or not the prefix of the voice data received through the microphone 314A has an "access keyword", and the prefix of the received voice data has an "access keyword". , Sends order information including text information to the order server 10.

The "access keyword" is, for example, the store name "○○ restaurant" or the like. If there is no "access keyword", the AI assistant server 40 is connected to perform so-called conventionally known information processing. Further, for example, the storage unit 312A may store the voice data detected by the microphone 314A in association with the date and time when each information is detected.

The control unit 311A is composed of a CPU having a calculation function, a work RAM, a ROM for storing various data and programs, and the like. Further, the control unit 311A is a controller, and is realized by, for example, an integrated circuit such as an ASIC or FPGA.

The control unit 311A includes a voice data receiving means 3111A, an authentication means 3112A, a text information generating means 3113A, and an order information transmitting means 3114A, and functions as each means of the present invention in cooperation with other members.

<Tablet terminal 31B>
The tablet terminal 31B performs processing according to control information from the store server 30, for example. For example, it is a device that displays a menu or the like on a touch panel according to control information from the store server 30. In addition, it is a device that allows the user to instruct operations, and can accept user operations on the touch panel, perform screen transitions, accept orders, and send accounting instructions to the store server 30.

The tablet terminal 31B is a control unit 311B as a computer composed of a CPU having a calculation function, a work RAM, a ROM for storing various data and programs, a semiconductor memory element such as a RAM or a flash memory, or a hard disk or an optical disk. Storage unit 312 including a storage device such as, etc., display unit 313B including a display screen such as a touch panel, input unit 314B that receives a user's operation instruction via the touch panel and gives an instruction signal corresponding to the instruction to the control unit 311B. A communication unit 315B for communicating with a store server 30, an order server 10, a service provider device 20, and the like via various networks (including a LAN (Local Area Network)) is provided. The components are connected to each other via a bus.

The storage unit 312B stores the orderable product database (DB) 312B and the like. Menu list information (food photos, food information (production area, etc.), prices, etc.) of orderable dishes is registered (memorized) in the orderable product DB3121B, and menus and dishes extracted from the orderable product DB3121B. Information etc. is displayed on the display screen. The menu list information of the orderable product DB3121B is configured to be updatable from the store server 30 based on the control information.

<Processing operation example 1 of voice recognition order system S>
7 to 12 show a case where a user enters a store, transmits various order information to the order server 10 via the smart speaker 31A, and executes an order acceptance process based on the order information received by the order server 10. It is a sequence chart for explaining the processing operation example.

<Entering the store-menu display>
The smart speaker 31A receives a voice utterance that triggers the user, and changes from the standby state to the activated state (step 1). The user makes some utterance following the trigger utterance, and the voice data receiving means 3111A of the control unit 311A cooperates with the microphone 314A to receive those utterances and receive them as voice data (step S2).

Next, the authentication means 3112A of the control unit 311A recognizes the received voice data and authenticates whether or not the access keyword is spoken at the prefix of the utterance (step S3). Authentication is performed by collating with the access keyword stored in advance in the voice recognition DB of the storage unit 312A. FIG. 7 shows an example in which "○○ restaurant" is used as an access keyword. In the example of FIG. 7, since the access keyword “○○ restaurant” is spoken as a prefix, the authentication is successful (OK), and the text information generation means 3113A orders the user based on the voice data received in step S2. Generate text information to specify (step S4).

Then, with reference to the connection destination information of the order server 10 stored in the storage unit 312A, the order information including the text information generated in step S4 is transmitted to the order server 10 (step S5). At this time, the "store ID" and the "smart speaker ID" stored in the storage unit 312A are transmitted together.

Next, when the reception processing means 1011 of the order server 10 receives the order information transmitted by the smart speaker 31A (step S6), the reception processing means 1011 executes the order reception processing based on the received order information (step S7). Specifically, the order acceptance process is executed based on the text information included in the order information.

In the example shown in FIG. 7 the previous stage, it has received the order information, including the text information of the content of "○○ Restaurant Hello" from the smart speaker 31A. Then, the control means 101 is out of the text information included in the order information from the "utterance scenario" associated with the "store ID" received together with the order information in the utterance scenario DB 1022 (FIG. 6B) of the storage unit 102. , extracts the response of conversation for the "Hello" except for the access keyword, to respond to smart speaker 31A, which is the source of the order information. The "store ID" identifies the sender's store from among a plurality of stores, and the "smart speaker ID" identifies the sender's smart speaker 31A from among the smart speakers 31A of the plurality of audience terminals 31. be able to.

In the example in the first part of Fig. 7, the reply is "Welcome to XX restaurant! How many people?". Specifically, the reception processing means 1011 refers to the store information DB 1021 of the storage unit 102, and refers to the smart speaker 31A stored in association with the "store ID" and the "smart speaker ID" transmitted together with the order information. Reply to the destination information. In step S5, the destination information of the smart speaker 31A may be included in the order information and transmitted to the order server 10, and in step S7, the destination information of the smart speaker 31A included in the order information may be returned. Good.

When the smart speaker 31A receives the response from the order server 10, the smart speaker 31A outputs the response from the speaker 313A (step S8).
Next, when the user utters "4 XX restaurants", the control unit 311A receives voice data (step S9), authenticates access keywords (step S10), generates text information (step S11), and orders including text information. Information transmission (step S12) is performed. Since the processes of steps S9 to S12 are the same as the processes of steps S2 to S5, the description thereof will be omitted.

The authentication status based on the access keyword can be set to be maintained for a predetermined time. Therefore, if the authentication state performed in step S3 continues after step S9, the authentication of the access keyword in step S10 is unnecessary. If the smart speaker 31A is not in the activated state before receiving the utterance in step S9 and receiving the voice data, it receives the voice utterance that triggers the user, changes from the standby state to the activated state, and then steps S9 and thereafter. Perform the processing of. Whether the smart speaker 31A is in the standby state, activated state, or authenticated state can be distinguished by the light emitting color of the LED (light emitting element) indicating the status, so that the light emitting color of the LED (light emitting element) of a different color corresponding to each state can be distinguished. Show to the user.

When the reception processing means 1011 of the order server 10 receives the order information transmitted by the smart speaker 31A (step S13), the reception processing means 1011 executes the order reception processing based on the received order information (step S14). In the example shown in the latter part of FIG. 7, order execution instruction information for instructing execution of processing according to the order contents is generated and transmitted to the store server 30 in which the smart speaker 31A of the order information transmission source is installed.

With reference to the utterance scenario DB 1022 (FIG. 6B) of the storage unit 102, the utterance excluding the access keyword is received from the text information, and the “next process” is executed. For example, as in the case of receiving order information including text information of "○○ restaurant 4 people" shown in the latter part of FIG. 7, after the utterance that tells the number of people, it is paired with the smart speaker 31A that tells the number of people. Displaying the menu on the display screen of the tablet terminal 31B is the "next process". Therefore, the order execution instruction information indicating the instruction is generated by the instruction information generating means 1012, and the instruction information transmitting means 1012 transmits the order execution instruction information to the store server 30 in which the smart speaker 31A of the order information transmitting source is installed (FIG. 8: step). S15).

Specifically, referring to the store information DB 1021 (FIG. 6A) of the storage unit 102, the “store information” stored in association with the “store ID” and the “smart speaker ID” transmitted together with the order information in step S12 is stored. The order execution instruction information is transmitted to the destination information of the store server 30 included in the above. At this time, the "smart speaker ID" which is the ID information for identifying the smart speaker 31A which is the transmission source of the order information in step S12 is also transmitted.

The instruction information receiving means 3011 of the store server 30 receives the order execution instruction information from the order server 10 (step S16). Then, the order execution means 3012 of the store server 30 executes processing according to the order contents based on the order execution instruction information.

In the example of FIG. 8, the terminal control means 3013 of the store server 30 uses the tablet terminal 31B associated with the smart speaker 31A specified by the smart speaker ID included in the order execution instruction information as the control target terminal device for order execution instruction. Control based on information.

Specifically, referring to the audience terminal management DB 3021 (FIG. 6C) of the storage unit 302, the “tablet terminal destination information” stored in association with the “smart speaker ID” transmitted together with the order execution instruction information in step S16. The control information for displaying the menu on the display screen of the tablet terminal 31B is transmitted (step S17).

The tablet terminal 31B receives the control information from the store server 30 (step S18) and performs processing according to the control information (step S19). In the example of FIG. 8, the cover of the menu is displayed on the display screen of the display unit 313B of the tablet terminal 31B. Specifically, it is extracted from the menu list information stored in the orderable product DB of the storage unit 312B of the tablet terminal 31B and displayed.

In step S15, the "smart speaker ID" of the smart speaker 31A, which is the source of the order information, is transmitted, but the content of the order execution instruction information generated by the instruction information generation means 1012 is the content related to the tablet terminal 31B. If there is, the store information DB 1021 (FIG. 6A) is referred to, and the "smart speaker ID" of the smart speaker 31A of the order information transmission source is changed to the "tablet terminal ID" associated with the "smart speaker ID" or You may send "tablet terminal address information". For example, when the "tablet terminal destination information" is received, in step S17, the terminal control means 3013 of the store server 30 does not need to refer to the audience seat terminal management DB 3021 (FIG. 6C).

When various displays such as a drink menu page, a meal menu page, a dessert menu page, and an order status display page are displayed from the cover of the menu displayed on the display screen of the tablet terminal 31B, the display screen of the tablet terminal 31B is used. The user can touch on the touch panel to turn the page or change the display.
In the present embodiment, a case of ordering a product by changing the display of the tablet terminal 31B through the smart speaker 31A will be described.

<Display of desired page>
The smart speaker 31A receives a voice utterance that triggers the user, and changes from the standby state to the activated state (step 20). The user makes some utterance following the trigger utterance, and the voice data receiving means 3111A of the control unit 311A cooperates with the microphone 314A to receive those utterances and use them as voice data (“Open the XX restaurant drink menu”). Receive (step S21).

Next, the control unit 311A performs access keyword authentication (step S22), text information generation (step S23), and order information transmission including text information (step S24). Since the processes of steps S21 to S24 are the same as the processes of steps S2 to S5, the description thereof will be omitted.

When the reception processing means 1011 of the order server 10 receives the order information transmitted by the smart speaker 31A (step S25), the reception processing means 1011 executes the order reception processing based on the received order information (step S26). Specifically, the order acceptance process is executed based on the text information included in the order information.
In the example shown in FIG. 9, order execution instruction information for instructing execution of processing according to the order contents is generated and transmitted to the store server 30 in which the smart speaker 31A of the order information transmission source is installed.

With reference to the utterance scenario DB 1022 (FIG. 6B) of the storage unit 102, the utterance excluding the access keyword is received from the text information, and the “next process” is executed. For example, as in the case of receiving order information including the text information "Open the XX restaurant drink menu" shown in FIG. 9, after the utterance requesting to open the predetermined menu page, the requesting source smart Displaying the requested menu page on the display screen of the tablet terminal 31B paired with the speaker 31A is the "next process". Therefore, the order execution instruction information indicating the instruction is generated by the instruction information generating means 1012, and the instruction information transmitting means 1012 transmits the order information transmitting means to the store server 30 in which the smart speaker 31A of the order information transmitting source is installed (step S27). Since the specific processing is the same as the processing in step S15, the description thereof will be omitted.

The instruction information receiving means 3011 of the store server 30 receives the order execution instruction information from the order server 10 (step S28). Then, the order execution means 3012 of the store server 30 executes processing according to the order contents based on the order execution instruction information.

In the example of FIG. 9, the terminal control means 3013 of the store server 30 uses the tablet terminal 31B associated with the smart speaker 31A specified by the smart speaker ID included in the order execution instruction information as the control target terminal device for order execution instruction. Control based on information.

Specifically, referring to the audience terminal management DB 3021 (FIG. 6C) of the storage unit 302, the "tablet terminal destination information" stored in association with the "smart speaker ID" received together with the order execution instruction information in step S28. Control information to instruct the display screen of the tablet terminal 31B to display the drink menu page is transmitted (step S29).

The tablet terminal 31B receives the control information from the store server 30 (step S30) and performs processing according to the control information (step S31). In the example of FIG. 9, the drink menu page is displayed on the display screen of the display unit 313B of the tablet terminal 31B. Specifically, it is extracted from the menu list information stored in the orderable product DB of the storage unit 312B of the tablet terminal 31B and displayed.

From the drink menu page displayed on the display screen of the tablet terminal 31B, the user can touch the touch panel to order a desired drink.
Subsequently, in the present embodiment, a case where a desired product is ordered through the smart speaker 31A will be described.

<Product order>
Next, the voice data receiving means 3111A of the control unit 311A cooperates with the microphone 314A and receives the user's utterance as voice data (“○○ restaurant draft beer 4!”) (Step S32).

Next, the control unit 311A performs access keyword authentication (step S33), text information generation (step S34), and order information transmission including text information (step S35). If the authentication status performed in step S21 continues, the access keyword authentication in step S33 is unnecessary. Since the processes of steps S32 to S35 are the same as the processes of steps S2 to S5, the description thereof will be omitted.

When the reception processing means 1011 of the order server 10 receives the order information transmitted by the smart speaker 31A (step S36), the reception processing means 1011 executes the order reception processing based on the received order information (step S37). Specifically, the order acceptance process is executed based on the text information included in the order information.

In the example shown in the first part of FIG. 10, order information including text information of "○○ restaurant draft beer 4!" Is received from the smart speaker 31A. Then, the control means 101 is out of the text information included in the order information from the "utterance scenario" associated with the "store ID" received together with the order information in the utterance scenario DB 1022 (FIG. 6B) of the storage unit 102. , "Conversation response" to "4 draft beers!" Excluding access keywords and "Next processing" will be executed.

As a reply to the conversation, the smart speaker 31A, which is the sender of the order information, is replied. The next process to be performed is to display the order contents on the display screen of the tablet terminal 31B.
First, in the example in the first part of FIG. 10, the reply "Are you sure you want to do this?" Is given. Since the specific processing is the same as the processing in step S7, the description thereof will be omitted.
When the smart speaker 31A receives the response from the order server 10, the smart speaker 31A outputs the response from the speaker 313A (step S38).

In step S39, the instruction information generating means 1012 generates order execution instruction information indicating an instruction to display the order contents on the display screen of the tablet terminal 31B, and the instruction information transmitting means 1012 causes the smart speaker 31A of the order information transmitting source to generate the order execution instruction information. It is transmitted to the installed store server 30 (step S39). Since the specific processing is the same as the processing in step S15, the description thereof will be omitted.

The instruction information receiving means 3011 of the store server 30 receives the order execution instruction information from the order server 10 (step S40). Then, the order execution means 3012 of the store server 30 executes processing according to the order contents based on the order execution instruction information.

In the example of the first stage of FIG. 10, the terminal control means 3013 of the store server 30 orders execution using the tablet terminal 31B associated with the smart speaker 31A specified by the smart speaker ID included in the order execution instruction information as the control target terminal device. Control based on the instruction information.

Specifically, referring to the audience terminal management DB 3021 (FIG. 6C) of the storage unit 302, the “tablet terminal destination information” stored in association with the “smart speaker ID” received together with the order execution instruction information in step S40. Control information for displaying the order details on the display screen of the tablet terminal 31B is transmitted (step S41).

The tablet terminal 31B receives the control information from the store server 30 (step S42) and performs processing according to the control information (step S43). In the example of the first stage of FIG. 10, the order contents are displayed on the display screen of the display unit 313B of the tablet terminal 31B. Specifically, it is extracted from the menu list information stored in the orderable product DB of the storage unit 312B of the tablet terminal 31B and displayed.
Upon receiving the utterance in step S38, the user may continue to place an order (steps S32 to S43).

The voice data receiving means 3111A of the control unit 311A cooperates with the microphone 314A and receives the user's utterance as voice data (“at or above”) (step S44).
Next, the control unit 311A performs text information generation (step S45) and order information transmission including text information (step S46).

Since the processes of steps S44 to S46 are the same as the processes of steps S2, S4, and S5, respectively, the description thereof will be omitted. If the smart speaker 31A is not in the activated state before receiving the utterance in step S44 and receiving the voice data, it receives the voice utterance that triggers the user, changes from the standby state to the activated state, and then steps S44 and thereafter. Perform the processing of. If the authentication status performed in step S33 does not continue, the access keyword is authenticated following step S44. If the user does not speak the access keyword in the non-authenticated state, the utterance is directed to the AI assistant server 40. The communication with the AI assistant server 40 will be described in detail later with reference to FIG.

Next, when the reception processing means 1011 of the order server 10 receives the order information transmitted by the smart speaker 31A (step S47), the reception processing means 1011 executes the order reception processing based on the received order information (step S48). Specifically, the order acceptance process is executed based on the text information included in the order information.

In the example shown in the latter part of FIG. 10, order execution instruction information for instructing execution of processing according to the order contents is generated and transmitted to the store server 30 in which the smart speaker 31A of the order information transmission source is installed.

With reference to the utterance scenario DB 1022 (FIG. 6B) of the storage unit 102, the utterance excluding the access keyword is received from the text information, and the “next process” is executed. For example, as in the case of receiving the order information including the text information of "above" shown in the latter part of FIG. 10, after the utterance requesting the confirmation of the order, the smart speaker 31A of the requesting source is paired. Displaying the order confirmation on the display screen of the tablet terminal 31B is the "next process". Therefore, the order execution instruction information indicating the instruction is generated by the instruction information generating means 1012, and the instruction information transmitting means 1012 transmits the order information transmitting means to the store server 30 in which the smart speaker 31A of the order information transmitting source is installed (step S49). Since the specific processing is the same as the processing in step S15, the description thereof will be omitted.

The instruction information receiving means 3011 of the store server 30 receives the order execution instruction information from the order server 10 (step S50). Then, the order execution means 3012 of the store server 30 executes processing according to the order contents based on the order execution instruction information.

In the example in the latter part of FIG. 10, the terminal control means 3013 of the store server 30 orders execution using the tablet terminal 31B associated with the smart speaker 31A specified by the smart speaker ID included in the order execution instruction information as the control target terminal device. Control based on the instruction information.

Specifically, referring to the audience terminal management DB 3021 (FIG. 6C) of the storage unit 302, the “tablet terminal destination information” stored in association with the “smart speaker ID” received together with the order execution instruction information in step S50. Control information to instruct the order confirmation display is transmitted to the display screen of the tablet terminal 31B (step S51).

The tablet terminal 31B receives the control information from the store server 30 (step S52) and performs processing according to the control information (step S53). In the latter example of FIG. 10, the order confirmation display is displayed on the display screen of the display unit 313B of the tablet terminal 31B.

A confirmation button is displayed along with the order confirmation display, and when the user selects the confirmation button, it is assumed that a formal order has been placed, and the order information is sent to the order terminal device in the kitchen and displayed on the monitor of the order terminal device. The order may be displayed or printed out and presented to the cook to start cooking. The store server 30 may function as an order terminal device. Alternatively, when the order execution instruction information is received in step S50, it may be confirmed that the formal order has been placed. Whether or not to confirm with the user can be changed as appropriate.

<Various orders>
Not limited to the above-mentioned food order, the voice recognition order system S can handle various orders.
As shown in the sequence chart of FIG. 11, when the user orders to introduce "this month's recommendation" in steps S100 to S104, the reception processing means 1011 of the order server 10 receives the order information transmitted by the smart speaker 31A. Then (step S105), the order acceptance process is executed based on the received order information (step S106).

Then, the recommended moving image is reproduced and displayed on the display screen of the tablet terminal 31B paired with the requesting smart speaker 31A via the store server 30 (steps S107 to S111). The recommended moving image is extracted from the menu list information stored in the orderable product DB of the storage unit 312B of the tablet terminal 31B and displayed.

As shown in the sequence chart of FIG. 12, when the user orders to explain "fresh fish heap" in steps S120 to S124, the reception processing means 1011 of the order server 10 receives the order information transmitted by the smart speaker 31A. (Step S125), the order acceptance process is executed based on the received order information (step S126).

Then, the control means 101 refers to the utterance scenario DB 1022 (FIG. 6B) of the storage unit 102, and explains the “fresh fish assortment” from the “speaking scenario” associated with the “store ID” received together with the order information. It is extracted and replied to the smart speaker 31A, which is the source of the order information (step S127).

In addition, the control means 101 refers to the utterance scenario DB 1022 (FIG. 6B) of the storage unit 102, and there is no product requested to be explained in the "product list" associated with the "store ID" received together with the order information. In that case, reply "There is no such product."
As described above, the voice recognition order system S can handle various orders. Not limited to the above-mentioned cases, it is possible to cancel an order or place an order for an accounting request.

<If authentication is not possible>
In the above-described embodiment, when the authentication means 3112A of the control unit 311A of the smart speaker 31A authenticates whether or not a predetermined access keyword is spoken for the voice data received by the voice data receiving means 3111A, and the authentication is successful. However, the case of authentication failure (authentication not possible) will be described with reference to the sequence chart of FIG.

The smart speaker 31A receives a voice utterance that triggers the user, and changes from the standby state to the activated state (step 130). The user makes some utterance following the trigger utterance, and the voice data receiving means 3111A of the control unit 311A cooperates with the microphone 314A to receive the utterances and voice data (“What time is the last train at Ebisu Station on the □□ line?””. ) (Step S131).

Next, the control unit 311A authenticates the access keyword (step S132). Authentication is performed by collating with the access keyword stored in advance in the voice recognition DB of the storage unit 312A, and if the authentication is not successful, that is, if the authentication is not possible (NG), the received voice data is sent to the AI assistant server. It is transmitted to 40 (step S133). Conventionally known information processing is performed on the AI assistant server 40. That is, the voice data is received (step S134), voice analysis is performed (step S135), and the optimum solution is returned to the smart speaker 31A (steps S136 and S137).

As described above, according to the smart speaker 31A of the present embodiment, if the access keyword authentication is successful, the connection is made to the order server 10, and if the access keyword is not authenticated (authentication is not possible), the normal AI assistant server 40 is connected. Be connected. Therefore, it can be incorporated into the configuration of the voice recognition order system S while maintaining the function as a conventionally known smart speaker.

<Other usage examples>
In the above-described embodiment, the case where the voice recognition order system S is applied to a restaurant has been described as an example, but the voice recognition order system S of the present invention can also be applied to other situations. Next, a case where the voice recognition order system S of the present invention is applied to a hotel, an inn, or the like will be described.

The configuration of the audience seat terminal 31 and the store server 30 is the same as that described above, except that the names are changed. That is, a guest room terminal 31 prepared in each guest room of each hotel, an order server 10 as a first computer, a front server 30 as a second computer, and a business operator operating the voice recognition order system S are installed. The service provider device 20 and the AI assistant server 40 of the smart speaker 31A are connected to the network by wire or wirelessly, respectively.

The order server 10 is a device that mainly receives order information from the smart speaker 31A installed in each guest room of each hotel and executes necessary order acceptance processing.
The storage unit 102 stores the hotel information database (DB) 1023, the utterance scenario database (DB) 1024, and the like. FIG. 14A is an example of the hotel information DB 1023, and in the case of the example shown in FIG. 14A, “hotel information” including the destination information of the front server 30 and “access” are associated with the “hotel ID” that uniquely identifies the hotel. "Keywords" are registered. All hotels are given different hotel IDs.

In the "hotel information", the "tablet terminal ID" that uniquely identifies the tablet terminal 31B of the guest room terminal 31 installed in each guest room in the hotel and the necessary control information are transmitted to each end 31B of the tablet end. "Tablet terminal destination information" for this purpose, "smart speaker ID" as an example of ID information that uniquely identifies the smart speaker 31A, and "smart speaker destination information" that is the destination that responds to the smart speaker 31A. It is registered in association with each other.

The utterance scenario DB 1024 stores information on a plurality of utterance scenarios of conversations with the user staying at the hotel through the smart speaker 31A. The utterance scenario is a story of conversations exchanged between the user and the order server 10. A user in a hotel guest room transitions between a plurality of stay states from entering the room to leaving the room for check-out. In the utterance scenario DB 1024, separate utterance scenarios are accumulated for each transition state.

FIG. 14B is an example of the utterance scenario DB 1024, and in the case of the example shown in FIG. 14B, the utterance scenario DB 1024 has a “speech scenario” and a “request list” registered in association with the “hotel ID”.

The front server 30 operates in the same manner as the store server 30. This is a device that mainly receives order execution instruction information from the order server 10 and executes processing according to the order contents, such as control of the tablet terminal 31B. For example, it is installed at the front desk of a hotel or in a concierge room.

The guest room terminal 31 is installed in each guest room and is composed of a smart speaker 31A and a tablet terminal 31B.
The storage unit 312B of the tablet terminal 31B stores a request content database (DB) (not shown) and the like. Request details DB3121B registers (memorizes) requestable information such as room service details, dinner reservations, dinner time reservations / changes, breakfast reservations, breakfast time reservations / changes, and private bath reservations. The request content read from the request content DB is displayed on the display screen.

<Processing operation example 2 of voice recognition order system S>
15 and 16 are sequence charts for explaining an example of processing operation when the voice recognition order system S of the present invention is applied to a hotel.

When the user orders the "supper reservation" in steps S200 to S204, the reception processing means 1011 of the order server 10 receives the order information transmitted by the smart speaker 31A (step S205), and based on the received order information. The order acceptance process is executed (step S206).

Then, the reservable time is displayed on the display screen of the tablet terminal 31B paired with the requesting smart speaker 31A via the front server 30 (steps S207 to S211). For example, in step S209, the front server 30 transmits control information including reservation time information of each restaurant in the hotel. Then, the tablet terminal 31B extracts the restaurant information stored in the request content DB of the storage unit 312B and displays it in association with the received reservable time information.

Subsequently, when the user orders "Japanese restaurant reservation from 17:00" in steps S212 to S215, the reception processing means 1011 of the order server 10 receives the order information transmitted by the smart speaker 31A (step S216) and receives the order information. The order acceptance process is executed based on the ordered order information (step S217).

In the example of the first stage of FIG. 16, the control means 101 is included in the order information from the “utterance scenario” associated with the “hotel ID” received together with the order information in the utterance scenario DB1024 (FIG. 14B) of the storage unit 102. Of the text information provided, the response to the conversation to "Reservation for Japanese restaurant from 17:00" excluding the access keyword and the processing to be performed next will be executed.

As a reply to the conversation, the smart speaker 31A, which is the sender of the order information, is replied. The next process to be performed is to display the order contents on the display screen of the tablet terminal 31B.
First, in the example in the first stage of FIG. 16, the reply "Do you have any other needs?" Is replied and output from the smart speaker 31A (step S218).

Further, the reservation content is displayed on the display screen of the tablet terminal 31B paired with the requesting smart speaker 31A via the front server 30 (steps S219 to S223).

Further, when an order is received from the user through the smart speaker 31A in steps S224 to S226, the reception processing means 1011 of the order server 10 receives the order information transmitted by the smart speaker 31A (step S227) and receives the order. The order acceptance process is executed based on the order information (step S228). Hereinafter, since the processing of steps S229 to S234 is the same as the processing described above, the description thereof will be omitted.

As described above, according to the voice recognition order system S in the present embodiment, it is authenticated whether or not a predetermined access keyword is spoken, and if the authentication is successful, the order information is transmitted to the order server 10 to place an order. Since the server 10 can execute the order reception process based on the order information, it is possible to realize the voice recognition order system S that performs the order process using the smart speaker 31A in a restaurant, a hotel, or the like.

When the order reception process is executed by replying the conversation with the order contents, a predetermined reply can be made from the order server 10 to the smart speaker 31A. In addition, if there is a "process to be performed next" corresponding to the order contents, the process can be executed.

Further, if the process to be performed as the order acceptance process based on the order information is the process to be instructed to the store server 30, the store server 30 can change the order contents by transmitting the order execution instruction information to the store server 30. It is possible to execute the corresponding processing. Then, the store server 30 identifies the tablet terminal 31B of the tablet terminal ID associated with the smart speaker ID (ID information) included in the order execution instruction information in the control information of the tablet terminal 31B, and controls the tablet terminal 31B. It can be controlled as a terminal device.

The scope of application of the present invention is not limited to the above-described configuration. The present invention can be widely applied to a voice recognition ordering system and a voice recognition ordering method performed by using a smart speaker.

10 Order server (first computer)
101 Control unit 1011 Reception processing means 1012 Instruction information generation means 1013 Instruction information transmission means 102 Storage unit 1021 Store information database (DB)
1022 Speech scenario database (DB)
103 display unit, 104 input unit, 105 communication unit 30 Store server (second computer)
301 Control unit 3011 Instruction information receiving means 3013 Terminal control means 302 Storage unit 3021 Audience terminal management database (DB)
303 Display unit, 304 Input unit, 305 Communication unit 31 Audience terminal 31A Smart speaker 311A Control unit 3111A Voice data receiving means 3112A Authentication means 3113A Text information generating means 3114A Order information transmitting means 312A Storage unit 3121A Voice recognition database (DB)
313A speaker (audio output means),
314A microphone (voice input means),
315A operation unit, 316A communication unit 31B tablet terminal (terminal device, controlled terminal device)
311B Control unit 312B Storage unit 3121B Orderable product database (DB)
313B display unit, 314B input unit, 315B communication unit 40 AI assistant server

Claims (8)

A voice recognition ordering system including a smart speaker having at least a voice input means and a voice output means, and a first computer connected to the smart speaker via a network so as to exchange information.
The smart speaker
A voice data receiving means for receiving voice data obtained by a user speaking while the trigger is activated.
An authentication means that recognizes the voice data received by the voice data receiving means and authenticates that a predetermined access keyword has been spoken.
A text information generating means that generates text information that specifies a user's order content based on the voice data received by the voice data receiving means, and a text information generating means.
It has an order information transmitting means for transmitting the order information including the text information generated by the text information generating means to the first computer when the authentication of the utterance of the access keyword is successful by the authentication means.
The first computer is
A voice recognition order system comprising a reception processing means that executes an order reception process based on the received order information when the order information transmission means of the smart speaker receives the order information transmitted. The first computer is an instruction information generation means for generating order execution instruction information instructing to execute a process according to the order contents based on the received order information.
It has an instruction information transmitting means for transmitting the order execution instruction information generated by the instruction information generating means to a second computer connected to the first computer via a network so as to exchange information.
The second computer
An instruction information receiving means for receiving the order execution instruction information transmitted by the instruction information transmitting means of the first computer, and
The voice recognition order system according to claim 1, further comprising an order execution means that executes a process according to the order contents based on the order execution instruction information received by the instruction information receiving means. The voice recognition order system according to claim 2, wherein the plurality of smart speakers and a terminal device associated with each of the smart speakers and including the terminal device capable of instructing an operation by a user.
The order execution instruction information includes ID information for identifying a smart speaker that is a source of the order information.
The second computer is a terminal that controls the terminal device associated with the smart speaker specified by the ID information included in the order execution instruction information as a control target terminal device based on the order execution instruction information. A voice recognition ordering system characterized by having a control means. The second computer transmits control information to the control target terminal device to instruct the display screen of the control target terminal device to display according to the order content indicated by the order execution instruction information. The voice recognition order system according to claim 3, wherein the voice recognition order system is characterized. A voice recognition ordering method executed by a smart speaker having at least a voice input means and a voice output means, and a first computer connected to the smart speaker via a network so as to exchange information.
The smart speaker
A voice data reception step that receives voice data obtained by the user speaking while the trigger is activated, and
An authentication step that recognizes the received voice data and authenticates that the specified access keyword has been spoken.
A text information generation step for generating text information that specifies a user's order content based on the voice data received in the voice data receiving step, and a text information generation step.
When the authentication of the utterance of the access keyword is successful in the authentication step, the order information transmission step of transmitting the order information including the text information generated in the text information generation step to the first computer is used. Have,
The first computer
A voice recognition ordering method comprising: a reception processing step of executing an order reception process based on the order information upon receiving the order information transmitted from the smart speaker. The first computer
Upon receiving the order information transmitted from the smart speaker, an instruction information generation step of generating order execution instruction information instructing to execute a process according to the order contents based on the order information, and an instruction information generation step.
It has an instruction information transmission step of transmitting the order execution instruction information generated by the instruction information generation step to a second computer connected to the first computer via a network so as to exchange information. And
The second computer
An instruction information receiving step for receiving the order execution instruction information transmitted from the first computer, and
The voice recognition order method according to claim 5, further comprising an order execution step for executing a process according to the order content based on the order execution instruction information received in the instruction information receiving step. .. The order execution instruction information includes ID information for identifying the smart speaker that is the source of the order information.
The second computer is a plurality of smart speakers and a terminal device associated with each of the smart speakers, and is included in the order execution instruction information among the terminal devices that can be operated by the user. The sixth aspect of claim 6 is characterized in that the terminal device associated with the smart speaker specified by the ID information is controlled as a control target terminal device based on the order execution instruction information. Voice recognition ordering method. The second computer transmits control information to the control target terminal device to instruct the display screen of the control target terminal device to display according to the order content indicated by the order execution instruction information. The voice recognition ordering method according to claim 7, characterized in that.