JP7465075B2 - Calculation device, recording medium, voice input device - Google Patents

Description

The present invention relates to a computing device, a recording medium, and a voice input device.

Due to the declining birthrate and aging population, as well as work style reforms, there is a demand to provide a large number of robots capable of interacting with humans at low cost. In order for a robot to provide a highly satisfying voice dialogue to humans, it is necessary to have a fast response speed and a high recognition rate. To achieve this, a method of starting a voice recognition process when a user speaks to a robot can be considered. However, the voice recognition process has a large-capacity language model for converting spoken voice into characters with high accuracy, and it takes several seconds to start up. Therefore, if the voice recognition process is started in response to a user's speech, a waiting time occurs before the user's voice recognition starts, resulting in a delayed response to the user, which causes a problem of lowering user satisfaction.
Patent document 1 discloses a communication robot equipped with an audio output unit, which includes an imaging unit that images a subject and generates an image, a target person identification means that identifies a target person to be spoken to based on the image obtained by the imaging unit, a congestion degree determination means that determines the degree of congestion in the area around the position where the communication robot is placed, and a speech means that performs a predetermined process depending on the determination result by the congestion degree determination means and outputs speech data addressed to the target person identified by the target person identification means from the audio output unit.

JP 2018-149625 A

The invention described in Patent Document 1 leaves room for improvement in the timing of starting the voice recognition process.

A computing device according to a first aspect of the present invention includes resource control information indicating the possibility of a voice dialogue with a user, a server communication unit that communicates with a plurality of robots that transmit voice information, a voice recognition unit that is capable of executing a plurality of recognition processes that convert the voice information received from the robots into text information, and a process control unit that uses the resource control information received from the robots to manage the number of recognition processes that are activated and starts and stops the recognition processes.
A recording medium according to a second aspect of the present invention is a computer-readable recording medium having the following program recorded thereon, the program causing a computer having resource control information indicating the possibility of a voice dialogue with a user and a server communication unit that communicates with a plurality of robots that transmit voice information to execute a voice recognition process capable of executing a plurality of recognition processes that convert the voice information received from the robots into text information, and a process control process that uses the resource control information received from the robots to manage the number of recognition processes that are activated and to start and stop the recognition processes.
A voice input device according to a third aspect of the present invention is a voice input device comprising: a robot communication unit capable of communicating with a calculation device having a voice recognition unit capable of executing multiple recognition processes for converting received voice information into text information; a start-up request unit that transmits a start-up request to the calculation device to request the start-up of the recognition process based on the distance from the user; and a transmission unit that records the user's speech and transmits it to the calculation device as the voice information, and further comprises a distance determination unit that calculates the product of the start-up time of the recognition process and the relative speed between the user and the voice input device as a monitoring distance, and the start-up request unit transmits the start-up request when the distance between the user and the voice input device is less than or equal to the monitoring distance.

According to the present invention, the voice recognition process is initiated when the user approaches the robot, thereby shortening the time between when the user speaks and when voice recognition begins.

Overview of the speech recognition system Robot configuration diagram A diagram showing the relationship between the arithmetic unit and the input/output unit. FIG. 1 shows the configuration of a computing device. A diagram showing the hardware configuration of a computer that realizes a calculation device. FIG. 2 is a diagram showing an example of the configuration of a voice recognition unit; FIG. 13 is a diagram illustrating an example of a standby management table. FIG. 13 is a diagram showing an example of a number management table. FIG. 13 is a diagram showing an example of an allocation management table. Flowchart showing the process of the start request unit A flowchart showing details of the user detection process of FIG. 10. Flowchart showing the process of the allocation request unit Flowchart showing the processing of the process control unit A flowchart showing a process executed when a positive determination is made in step S1301 of FIG. Flowchart showing the processing of the process stopping unit FIG. 1 is a configuration diagram of a robot according to a second embodiment. FIG. 1 is a block diagram of a computing device according to a second embodiment. 13 is a flowchart showing the process of the distance determination unit in the third embodiment.

-First embodiment-
A first embodiment of a voice recognition system will be described below with reference to FIGS.

Figure 1 is a schematic diagram showing the entire voice recognition system in the first embodiment. The voice recognition system S is configured to include multiple robots 200 and a calculation device 400 equipped with a voice recognition unit 600, which will be described in detail later. In the following, in order to distinguish between the multiple robots 200, they will be given sub-numbers and referred to as robot 200-1, robot 200-2, etc. As it is sufficient for there to be more than one robot 200, Figure 1 shows up to robot 200-n as being made up of "n" robots.

Each of the multiple robots 200 communicates with the computing device 400. For example, this communication is aggregated by the router 120 to improve communication efficiency. The router 120 and the computing device 400 are connected via the network 110. However, the router 120 and the computing device 400 may communicate without going through the network 110.

The robot 200 and the router 120 are installed in a tenant 100. The tenant 100 is a building, a commercial facility, or the like. The computing device 400 is installed in a cloud 300 on a network. The cloud 300 is an abstract concept that refers to one or more points connected by a network. In this embodiment, the configuration shown in FIG. 1 is used as a premise for explanation, but the robot 200 and the computing device 400 may be present in the same location, for example, in the same building or in the same room. Also, the tenant 100 and the cloud 300 are merely examples.

In this embodiment, humans around the robot 200 are called "users." The users approach the robot 200 and speak, and the robot 200 behaves in response to the user's utterance. In more detail, the robot 200 records the user's utterance as voice data, converts the voice data into text data using the computing device 400, and interprets the text data to behave in response to the user's utterance.

Figure 2 is a configuration diagram of robot 200-1. Robots 200-1 to 200-n have the same configuration, so here the configuration of robot 200-1 will be described as a representative. However, in Figure 2, the hardware configuration, functional configuration, and information are described together. Robot 200-1 has a communication interface unit 201, a distance sensor 202, a microphone 203, and a speaker 204 as a hardware configuration. Robot 200-1 has a motion instruction analysis unit 205, a robot motion control unit 206, a data transmission/reception unit 207, a distance determination unit 208, an activation request unit 209, and an allocation request unit 210 as a functional configuration. Robot 200-1 has sensor information 211, voice information 212, human walking speed information 213, and tenant map information 214.

The hardware configuration of the robot 200-1 will now be described. The communication interface unit 201 is a communication module that communicates with the computing device 400. The robot 200-1 acquires distance information from surrounding users using the distance sensor 202, and stores this as sensor information 211. The distance sensor 202 is, for example, an RGB-D sensor that acquires a three-dimensional point cloud that is color with added depth information, i.e., RGB information. However, the distance sensor 202 is not limited to this. In addition, the sensors are not limited to those included inside the robot, and information may be acquired using, for example, a surveillance camera installed outside as a sensor.

The microphone 203 records the voice spoken by the user to the robot 200-1 and stores it as voice information 212. The robot 200-1 communicates with the arithmetic device 400 via the data transmission/reception unit 207, and transmits acquired data or receives data sent from the arithmetic device 400. The speaker 204 plays the spoken voice information sent from the arithmetic device 400. This concludes the description of the hardware configuration of the robot 200-1.

The functional configuration of the robot 200-1 will be described. The functions of the robot 200-1, namely, the motion instruction analysis unit 205, the robot motion control unit 206, the data transmission/reception unit 207, the distance determination unit 208, the activation request unit 209, and the allocation request unit 210, are realized, for example, by expanding a program stored in a ROM (not shown) into a RAM and executing it. However, the program may be stored in a non-volatile storage device (not shown). The robot 200-1 may also have an input/output interface (not shown), and the program may be read from another device when necessary via the input/output interface and a medium available to the robot 200-1. Here, the medium refers to, for example, a storage medium 291 that is detachable from the input/output interface, or a communication medium 292, i.e., a network such as a wired, wireless, or optical network, or a carrier wave or digital signal that propagates through the network.

The motion instruction analysis unit 205 analyzes the motion instructions sent from the calculation device 400, converts them into a format that can be interpreted by the robot 200, and outputs them to the robot motion control unit 206. The robot motion control unit 206 performs an action according to the received motion instruction, such as changing facial expression or moving. The data transmission/reception unit 207 realizes transmission and reception of data with the calculation device 400 using the communication interface unit 201. The distance determination unit 208 determines a monitoring distance, which is a distance used as a threshold value in the processing described below. This monitoring distance is used as follows. That is, by starting the recognition process 601 when the user gets closer to the robot 200 than the monitoring distance, voice recognition can be started immediately when the user starts speaking.

The distance determination unit 208 calculates the monitoring distance using the process startup time, which is the time required to start the recognition process 601 received from the computing device 400, and the relative speed between the user and the robot 200. However, in this embodiment, since the robot 200 does not move, human walking speed information 213 stored in advance in the robot 200 is used instead of the relative speed. For example, if the process startup time of the recognition process 601 is 5 seconds and the user's walking speed is 1 meter per second, the distance determination unit 208 determines the monitoring distance to be 5 meters. Note that the startup time required for the recognition process 601 may be determined using data stored in advance in the robot.

The start request unit 209 compares the monitoring distance obtained from the distance determination unit 208 with the distance information from the robot 200-1 to surrounding users obtained by the distance sensor 202. When the start request unit 209 determines that a user has entered within the monitoring distance of the robot 200-1, it sends a start request for the recognition process 601 to the calculation device 400. The start request can also be called information indicating the possibility of a voice dialogue with the user. Furthermore, since the calculation device 400 that receives the start request secures resources for starting the recognition process 601, the start request can also be called resource control information.

In addition, the start request unit 209 sends a request to the calculation device 400 to stop the recognition process 601 when it determines that a user who was within the monitoring distance has moved farther away from the robot 200-1 than the monitoring distance. The allocation request unit 210 sends a request to the calculation device 400 to allocate the recognition process 601 when the user starts speaking, and sends a request to the calculation device 400 to deallocate the recognition process 601 when the user finishes speaking. This concludes the explanation of the functional configuration of the robot 200-1.

The information stored in the robot 200-1 will now be described. The sensor information 211 is distance information from surrounding users, obtained using the distance sensor 202. The voice information 212 is a recording of the voice spoken by the user to the robot 200-1, obtained using the microphone 203.

The human walking speed information 213 is information about the speed at which the user walks, and is stored in advance. The human walking speed information 213 may be not only data stored in advance, but also newly acquired values, such as the results of analyzing the output of the distance sensor 202. The tenant map information 214 is information about a three-dimensional point cloud of fixed objects such as walls, pillars, and shelves in the tenant 100. The tenant map information 214 is used when the activation request unit 209 determines the distance between the user and the robot 200. This concludes the explanation of the information stored in the robot 200-1.

Figure 3 is a diagram showing the relationship between the arithmetic device 400 and the input/output devices. The cloud 300 is equipped with a communication interface unit 301 that is connected to the network 110. An input device 302 and an output device 303 are connected to the network 110. The input device 302 is, for example, a keyboard, a mouse, or a touch panel. The output device 303 is, for example, an LCD display. An operator can use the input device 302 to give operational commands to the arithmetic device 400 and use the output device 303 to check the results of calculations performed by the arithmetic device 400.

Figure 4 is a diagram showing the configuration of the arithmetic device 400. However, Figure 4 shows the functional configuration of the arithmetic device 400 and the information stored in the arithmetic device 400. The arithmetic device 400 has, as its functions, a server communication unit 401, a voice recognition unit 600, a dialogue processing unit 403, a voice synthesis unit 404, an instruction unit 405, a time measurement unit 406, a process control unit 407, a process stopping unit 408, a task reception unit 409, and a task control unit 410. The arithmetic device 400 stores dialogue definition information 412, operation scenario information 413, voice recognition processing information 414, and lifespan information 415.

The functions of the computing device 400 will be described. The server communication unit 401 communicates with the robot 200 via the communication interface unit 301 and the network 110, and exchanges voice data and the like. The voice recognition unit 600 processes data received from the robot 200 and converts the voice data into text data. The detailed configuration of the voice recognition unit 600 will be described later.

The dialogue processing unit 403 uses the dialogue definition information 412 to output text that responds to the text data obtained by the voice recognition unit 402. The voice synthesis unit 404 converts the response text output by the dialogue processing unit 403 into voice data. The instruction unit 405 generates instructions to cause the robot to speak or perform an action according to the data generated by the voice synthesis unit 404 and the action scenario information 413. Furthermore, the instruction unit 405 transmits the generated instructions to the robot 200 via the server communication unit 401.

The time measurement unit 406 measures the time from the start instruction to the recognition process 601 to the time when the recognition process 601 can start processing the voice recognition process. This time will be explained again later using a concrete example. For example, the time measurement unit 406 measures several times and transmits the longest time as the process start time to the robot 200. Note that instead of the time measurement unit 406 measuring the process start time, a value measured in advance by the operator of the computing device 400 may be input from the input device 302 and used.

The process control unit 407 controls the recognition process 601 in accordance with requests to start, stop, allocate, and deallocate the recognition process 601 received from the robot 200. Details of the operation of the process control unit 407 will be described later. The process stop unit 408 stops the recognition process 601 that has been stopped for a predetermined time or more. The task reception unit 409 receives instructions from the user to the robot 200 via the input device 302, and the task control unit 410 executes the received processing. This concludes the description of the functions of the calculation device 400.

The information stored in the computation device 400 will be described. The dialogue definition information 412 is a set of sentences for establishing a dialogue between the user and the robot 200. The dialogue definition information 412 is, for example, a combination of questions and answers, and the computation device 400 outputs answers to questions uttered by the user. The action scenario information 413 describes the actions that the robot 200 should perform for each situation. The action scenario information 413 describes, for example, that when the user utters a specific series of utterances, the robot 200 will perform a specific action, such as moving and taking a predetermined pose.

The voice recognition processing information 414 stores a standby management table 700, a number management table 800, and an allocation management table 900. These tables will be described later. The lifespan information 415 is a startup time threshold that the process stopping unit 408 refers to when stopping the recognition process 601. The lifespan information 415 is registered in advance by the administrator.

Figure 5 is a diagram showing the hardware configuration of a computer 500 that realizes the arithmetic device 400. The arithmetic device 400 is composed of one or more computers 500. The computer 500 includes an arithmetic device 501 represented by a CPU (Central Processing Unit), a memory 502 such as a RAM (Random Access Memory), an input device 503, an output device 504, a memory controller 505, and an I/O (Input/Output) controller 506. The arithmetic device 501, the memory 502, the input device 503, the output device 504, and the I/O controller 506 are connected to each other via the memory controller 505. The input device 503 is, for example, a keyboard, a mouse, a touch panel, etc. The output device 504 is, for example, a video graphics card connected to an external display monitor, etc.

In the computer 500, each program of the arithmetic device 400 is read from an external storage device 508 such as an SSD or HDD via an I/O controller 506. The arithmetic device 501 and memory 502 cooperate to execute these programs, thereby realizing the functions of the arithmetic device 400. The computer 500 may also have an input/output interface (not shown), and programs may be read from other devices when necessary via the input/output interface and a medium available to the computer 500. The medium here refers to, for example, a storage medium 591 that is detachable from the input/output interface, or a communication medium 592, i.e., a network such as a wired, wireless, or optical network, or a carrier wave or digital signal that propagates through the network.

Figure 6 is a diagram showing an example of the configuration of the voice recognition unit 600. The voice recognition unit 600 comprises an arbitrary number of recognition processes 601, a language model 602, a load balancer 603, and a controller 604. The controller 604 starts and stops the recognition processes 601 in accordance with the operational instructions of the process control unit 407 and the process stopping unit 408. The recognition process 601 is a general term for processes that perform processing to convert voice data received from each robot 200 into text data. In this embodiment, each recognition process 601 is assigned a subnumber and is referred to as recognition process 601-1, recognition process 601-2, ..., etc.

The language model 602 acts as a dictionary for converting voice data into text data. The load balancer 603 transmits the voice data sent from the robot 200 to one of the recognition processes 601 by referring to the allocation management table 900 described later.

The recognition process 601 will be described in detail using the startup of the recognition process 601-1 as an example. When the controller 604 is instructed to start a calculation process, it reads the speech recognition processing program from ROM or the like, secures an area in the memory 502 and expands it, mounts the language model 602, and executes various initialization processes to operate as the recognition process 601-1. Once the data reading and various initialization processes are complete, the recognition process 601-1 has completed startup and can be assigned to any robot 200.

When the recognition process 601-1 is assigned to one of the robots 200, it receives the voice output from that robot 200 and executes voice recognition processing. When the controller 604 decides to stop the recognition process 601-1 by processing described below, it releases the area reserved for the recognition process 601-1 in the memory 502.

The time measured by the time measurement unit 406 is the time from when the controller 604 receives an instruction to start the recognition process 601 to when the start of the new recognition process 601 is completed. When the controller 604 is instructed to start the calculation process again, it reads the same voice recognition processing program as before, expands it in a different area of the memory 502 from before, and operates it as the recognition process 601-2.

In other words, each of the recognition processes 601-1, 601-2, ... is a different instance of a voice recognition program generated using the same program code. In this embodiment, voice data from different robots 200 is input to each recognition process 601, so after the voice data is input, the information in the memory of each recognition process 601 does not match.

The recognition process 601-1 processes voice data transmitted from one robot 200. Once the recognition process 601-1 has completed processing of the voice data transmitted from one robot 200, it can process the voice data transmitted from another robot 200. However, the recognition process 601-1 cannot process voice data transmitted from two robots 200 at the same time.

The activation of each recognition process 601 by the controller 604 can also be called "creation" of each recognition process 601. Furthermore, the stopping of each recognition process 601 by the controller 604 can also be called "deletion" of each recognition process 601. Each recognition process 601 is also called a deployed program, an instance, a process, a container, etc. The recognition process 601 can be realized using Linux (registered trademark) container technology, or may be realized using a virtual machine. Furthermore, the configuration shown in FIG. 6 is merely a conceptual configuration, and each recognition process 601 may be started as an individual instance by using public cloud Software-as-a-Service (hereinafter referred to as "SaaS").

(data)
7 is a diagram showing an example of a standby management table 700 that manages each running recognition process 601. The standby management table 700 is composed of a plurality of records, and each record has fields for a process ID 701, a status 702, and a standby time 703. A unique ID assigned to each recognition process 601 is stored in the process ID 701. For example, when the recognition process 601 is realized by executing a program, the value stored in the process ID 701 may be the process ID number of the process. The status 702 stores the allocation status of each recognition process 601, that is, information indicating whether or not the process is actually being used for speech recognition processing at the present time.

The process control unit 407 transitions one of the available recognition processes 601 to an allocated state in order to have it occupied by a certain robot 200, and transitions it to a released state when speech recognition processing by a certain robot 200 is no longer necessary. The field of standby time 703 stores information indicating the elapsed time since the status 702 transitioned to the standby state in each recognition process 601. In the example shown in FIG. 7, the elapsed time itself is stored in the field of standby time 703, but it is also possible to store the time of transition to the standby state and compare the current time with the value in the field of standby time 703 when necessary.

Figure 8 is a diagram showing an example of a number management table 800 used by the process control unit 407. The number management table 800 has only one record having fields for activation number 801, allocation number 802, and reservation number 803. The activation number 801 field stores the number of each recognition process 601 that is currently activated. The allocation number 802 field stores the number of recognition processes 601 that have been allocated in accordance with an allocation request issued by the robot 200. The reservation number 803 field stores the number of robots 200 that have not yet been assigned a recognition process 601 and are expected to require voice recognition processing within a short period of time.

Note that below, the "value stored in the field of activation count 801" may be referred to as activation count 801, the "value stored in the field of allocation count 802" may be referred to as allocation count 802, and the "value stored in the field of reservation count 803" may be referred to as reservation count 803.

Figure 9 shows an example of an allocation management table 900 that manages allocation information for the recognition process 601 for a robot. The allocation management table 900 is composed of multiple records, and each record has fields for an allocation request ID 901, a robot ID 902, and a process ID 701. The process ID 701 field stores information similar to that of the process ID 701 in the waiting management table 700 illustrated in Figure 7, so a description of this will be omitted here.

In the allocation request ID 901, a unique ID is stored that is set for each allocation request issued to each robot 200 when the robot 200 starts speaking. In the robot ID 902, a unique ID is stored that is set for each robot 200. When the process control unit 407 receives an allocation request, it links the requesting robot 200 with the recognition process 601 to be allocated, assigns an allocation request ID, and registers them in the allocation management table 900.

(Flowchart showing the robot's processing)
The processing of each function commonly provided to the robots 200 will be described in detail with reference to Figures 10 to 12. However, in the description of Figures 10 to 12, the description will be made on the assumption that the robot 200-1, which is a specific robot 200, executes each flowchart.

Figure 10 is a flowchart showing the processing of the activation request unit 209 provided in the robot 200. The activation request unit 209 starts processing when the activation of the robot 200-1 is completed. First, in step S1001, the activation request unit 209 acquires monitoring distance information, for example "5 m", from the distance determination unit 208. Next, in step S1002, the activation request unit 209 acquires distance information acquired by the distance sensor 202. In the following step S1091, a user detection process described below is performed to detect a user present in the monitoring range acquired in step S1001.

In the following step S1003, if the start request unit 209 determines that a user is present within the monitoring distance range from the robot 200-1, the process proceeds to step S1004. If the start request unit 209 determines that a user is not present within the monitoring distance range from the robot 200-1, the process returns to step S1002. In step S1004, the start request unit 209 transmits a start request for the recognition process 601 to the computing device 400. In the following step S1005, the start request unit 209 acquires distance information acquired by the distance sensor 202 in the same manner as in step S1002. In the following step S1092, the start request unit 209 performs a user detection process similar to step S1091, and proceeds to step S1006.

In step S1006, the start request unit 209 determines whether or not a user is present within the monitoring distance range of the robot 200-1, in other words, whether or not the user determined to be present within the monitoring range in step S1003 is still remaining within the monitoring range. If the start request unit 209 determines that the user is present within the monitoring range, it returns to step S1005, and if it determines that the user is not present within the monitoring range, it proceeds to step S1007. In step S1007, the start request unit 209 sends a request to stop the recognition process 601 to the computing device 400. In the following step S1008, the start request unit 209 determines whether or not the robot 200-1 is in the process of stopping, and if it determines that the robot 200-1 is in the process of stopping, it ends the process shown in FIG. 10, and if it determines that the robot 200-1 is not in the process of stopping, it returns to step S1002.

Figure 11 is a flowchart showing the details of the user detection process shown in steps S1091 and S1092 of Figure 10. In step S1101, the start request unit 209 acquires point cloud information from the tenant map information 214. In step S1102, the start request unit 209 identifies the current position of the robot 200-1 based on the tenant map information 214. If the robot 200-1 does not move, fixed position coordinates may be used. Next, in step S1103, the start request unit 209 deletes point clouds located outside the monitoring distance from the three-dimensional point cloud information acquired by the distance sensor 202.

In the following step S1104, the start request unit 209 compares the tenant map information 214 with the three-dimensional point cloud information, and deletes the point cloud corresponding to fixed objects such as walls and pillars. The processing of steps S1103 and S1104 deletes the information of fixed objects contained in the point cloud acquired by the robot 200-1, so that if a user is present within the monitoring range, only that point cloud remains. In step S1105, if the start request unit 209 determines that a point cloud remains after the processing of step S1104, it proceeds to step S1106, and if it determines that no point cloud remains, it proceeds to step S1107.

In step S1106, the start request unit 209 outputs a detection result indicating that a user was detected within the monitoring range, and ends the process shown in FIG. 11. In step S1107, the start request unit 209 outputs a detection result indicating that a user was not detected within the monitoring range, and ends the process shown in FIG. 11.

Figure 12 is a flowchart showing the processing of the allocation request unit 210. When the allocation request unit 210 detects the start of a user's speech, it executes the processing shown in Figure 12. The start of speech can be detected, for example, when the volume of the sound input to the microphone 203 becomes larger than a predetermined threshold. In step S1202, the start request unit 209 sends an allocation request for the recognition process 601 to the computing device 400. In step S1203, the start request unit 209 determines whether or not the end of the user's dialogue has been detected.

The end of the dialogue can be detected, for example, by using a sensor to detect the movement of the user who was in front of the robot 200-1, or by detecting that the user does not make a next utterance even after a certain time has passed since the user finished speaking. If the start request unit 209 determines that the end of the user's dialogue has been detected, it proceeds to step S1204, and if it determines that the end of the user's dialogue has not been detected, it remains in step S1203. In step S1204, the start request unit 209 sends a request to the computing device 400 to deallocate the recognition process 601.

In steps S1202 and S1204, the information sent by the activation request unit 209 to the computing device 400 includes information that can identify the sender, robot 200-1. For example, when TCP/IP is used as the communication protocol, the computing device 400 can identify the sender, robot 200, because the sender's IP address included in the header of the IP packet indicates robot 200-1. In addition, in the case of a communication protocol such as CAN (registered trademark) that does not include information that identifies the sender, robot 200-1 adds an identifier that indicates robot 200-1 to the payload.

(Flowchart showing the processing of the arithmetic device)
The processing of the functions of the arithmetic device 400 will be described in detail with reference to Figures 13 to 15. However, in the description of Figures 13 to 15, it is assumed that the robot 200 that has transmitted the allocation request, release request, start request, and stop request is the robot 200-1.

Figures 13 and 14 are flowcharts showing the processing of the process control unit 407 included in the computing device 400. The computing device 400 starts the processing shown in Figure 13 when it receives a request from the robot 200-1, specifically, an allocation request, a release request, a start request, or a stop request.

In step S1301, if the computing device 400 determines that the received request is either an allocation request or a release request, it proceeds to "A" and executes the process shown in FIG. 14. If the computing device 400 determines that the received request is neither an allocation request nor a release request, it proceeds to step S1302. In step S1302, the process control unit 407 determines whether the received request is a startup request, and if it is determined that it is a startup request, it proceeds to step S1303, and if it is determined that it is not a startup request, i.e., that it is a stop request, it proceeds to step S1308.

In step S1303, the process control unit 407 reads the number management table 800 and calculates the number of free processes, which will be described next. The number of free processes is the value obtained by subtracting the number of allocations 802 and the number of reservations 803 from the number of activations 801. In the example of the number management table 800 shown in FIG. 8, the number of activations 801 is "3", the number of allocations 802 is "1", and the number of reservations 803 is "1", so the number of free processes is "1" obtained by subtracting "2" from "3". In the following step S1304, if the process control unit 407 determines that the number of free processes is 1 or more, the process proceeds to step S1305, and if the process control unit 407 determines that the number of free processes is zero, the process proceeds to step S1306. In step S1305, the process control unit 407 increases the number of reservations 803 in the number management table 800 by "1" and ends the process shown in FIG. 13.

In step S1306, the process control unit 407 starts the recognition process 601 and registers information about the started recognition process 601 in the standby management table 700. In the following step S1307, the process control unit 407 increases the start number 801 and the reservation number 803 in the number management table 800 by "1", and ends the processing shown in FIG. 13. In step S1308, the process control unit 407 decreases the reservation number 803 in the number management table 800 by "1", and ends the processing shown in FIG. 13.

Figure 14 is a flowchart showing the processing of "A" that is executed when a positive judgment is made in step S1301 in Figure 13. First, in step S1402, the process control unit 407 judges whether the received request is an allocation request or not, and if it is judged to be an allocation request, the process proceeds to step S1403, and if it is judged not to be an allocation request, i.e., a deallocation request, the process proceeds to step S1408. In step S1403, the process control unit 407 refers to the standby management table 700 to acquire data.

In the next step S1404, the process control unit 407 selects a record from the standby management table 700 in which the value of the status 702 field is "standby" and the value of the standby time 703 field is the smallest, in other words, the standby time is the shortest. The process control unit 407 then changes the value of the status 702 field of that record to "allocated". In the next step S1405, the process control unit 407 instructs the load balancer 603 to send the voice information sent from the robot 200-1 to the recognition process 601 identified by the value of the process ID 701 of the record selected in step S1404.

In the next step S1406, the process control unit 407 adds a new record to the allocation management table 900 and records the robot ID of the robot 200-1 that sent the allocation request and the ID of the selected recognition process 601. Finally, in step S1407, the process control unit 407 increases the value of the allocation number 802 in the number management table 800 by "1" and decreases the reservation number 803 by "1", and ends the processing shown in FIG. 14.

In step S1408, which is executed if a negative judgment is made in step S1402, the process control unit 407 identifies a record in the allocation management table 900 that includes the robot ID of the robot 200-1 that sent the release request for allocation. In the following step S1409, the process control unit 407 changes the value of the status 702 field in the record identified in step S1408 to "waiting" and changes the value of the wait time 703 field to "0". In the following step S1410, the process control unit 407 deletes the corresponding entry from the allocation management table 900. Finally, in step S1411, the process control unit 407 decrements the allocation number 802 in the number management table 800 by "1" and increments the reservation number 803 by "1", and ends the processing shown in FIG. 14.

Figure 15 is a flowchart showing the processing of the process stopping unit 408. The process stopping unit 408 executes the processing shown in Figure 15 at predetermined intervals, such as every 10 seconds, 1 minute, or 10 minutes. First, in step S1501, the process stopping unit 408 reads the values in the wait time 703 field of all records in the wait management table 700. In the following step S1502, the process stopping unit 408 identifies, from the records read in step S1501, records in which the value of the wait time 703 field is equal to or greater than a preset threshold value. Furthermore, the process stopping unit 408 stops the recognition process 601 identified by the process ID 701 written in the identified record.

In the next step S1503, the process stopping unit 408 reduces the value of the activation count 801 in the count management table 800 by the number of recognition processes 601 stopped in step S1502. Finally, in step S1504, the process stopping unit 408 deletes the record in which the recognition process 601 stopped in step S1502 is written from the standby management table 700, and ends the processing shown in FIG. 15.

According to the above-described first embodiment, the following advantageous effects can be obtained.
(1) The arithmetic device 400 includes a server communication unit 401 that communicates with a plurality of robots 200 that transmit resource control information indicating the possibility of a voice dialogue with a user and voice information, a voice recognition unit 600 that can execute a plurality of recognition processes 601 that convert voice information received from the robots 200 into text information, and a process control unit 407 and a process stop unit 408 that manage the number of recognition processes 601 that are activated using the resource control information received from the robots 200 and start and stop the recognition processes 601. Therefore, the arithmetic device 400 activates the recognition process 601 when the user approaches the robot 200, thereby shortening the time from when the user speaks to when voice recognition by the recognition process 601 is started.

As explained in this embodiment, the monitoring distance is determined using the walking speed of a human, so in most cases, the activation of the recognition process 601 is completed when the user arrives near the robot 200. However, if the user approaches the robot 200 by running, or if the user speaks to the robot 200 from a distance, the recognition process 601 may be in progress when the speech is made. However, even in such a case, the recognition process 601 has already started to be activated, so this has the effect of shortening the time from when the user speaks to when voice recognition by the recognition process 601 begins.

It is also possible to start the recognition process 601 in advance regardless of whether the user is present, and when voice information is received, start the voice recognition process immediately while also starting a new recognition process 601. In this case, however, at least one recognition process 601 is always started in an idle state, which wastes resources. If the computer 500 that executes the recognition process 601 is also used for an application other than the recognition process 601, there is a disadvantage that the resources available to that application are reduced. Also, if the recognition process 601 uses SaaS, there is a disadvantage that the usage fee for the recognition process 601 that is kept waiting in an idle state is an additional expense. Therefore, the method of this embodiment, which does not cause such disadvantages, is advantageous.

(2) The resource control information includes a request to start a recognition process that the robot 200 outputs based on the distance from the user. The process control unit 407 starts the recognition process 601 based on the start request sent by the robot, and converts the voice information sent by the robot into text information by assigning the started recognition process 601 to the robot 200. Therefore, since the recognition process 601 that is already running can be assigned to the robot 200-1, voice recognition processing can be started immediately even if the user speaks from a position far away from the robot 200-1, such as near the monitoring distance.

(3) When the process control unit 407 receives a request to allocate the recognition process 601 from the robot 200-1 (S1402: YES in FIG. 14), it allocates the recognition process 601 to the robot 200-1 by processing steps S1403 to S1407. This is managed using the allocation management table 900.

(4) When the process stop unit 408 receives a request to deallocate the recognition process 601 from the robot 200-1 (S1402: NO in FIG. 14), it deallocates the recognition process 601 to the robot 200-1 by processing steps S1408 to S1411.

(5) Each robot 200, which is also a voice input device, is equipped with a data transmission/reception unit 207 capable of communicating with a calculation device 400 having a voice recognition unit 600 capable of executing multiple recognition processes 601 that convert received voice information into text information, an activation request unit 209 that transmits an activation request to the calculation device 400 to request activation of the recognition process 601 based on the distance from the user, and the data transmission/reception unit 207 that records the user's speech and transmits it to the calculation device 400 as voice information.

(6) Robot 200-1 is equipped with distance determination unit 208 that calculates the product of the start-up time of recognition process 601 and the relative speed between the user and robot 200-1 as the monitoring distance. However, in this embodiment, robot 200 does not move, so human walking speed information 213 stored in advance in robot 200 is used instead of the relative speed. As shown in steps S1091, S1003, and S1004 of FIG. 10, start-up request unit 209 transmits a start-up request when the distance between the user and robot 200-1 is equal to or less than the monitoring distance. Therefore, when the user reaches robot 200-1, start-up of recognition process 601 has been completed, and voice recognition can start immediately.

(7) As shown in steps S1092, S1006, and S1007, the start request unit 209 of the robot 200-1 transmits a stop request to stop the recognition process 601 when the distance between the user and the robot 200-1 becomes greater than the monitoring distance. This allows the resources used for the recognition process 601 in the computing device 400 to be appropriately saved.

(Variation 1)
In the first embodiment described above, each robot 200 transmits an allocation request and a deallocation request to the arithmetic device 400. However, each robot 200 does not have to transmit at least one of an allocation request and a deallocation request. If the robot 200 does not transmit an allocation request, the arithmetic device 400 refers to the allocation management table 900 each time voice information is transmitted from the robot 200, and determines whether or not a recognition process has been allocated to the robot 200 that transmitted the voice information. If the arithmetic device 400 determines that a recognition process has not been allocated to the robot 200 that transmitted the voice information, it performs the same processing as when an allocation request is received.

If the robot 200 does not send a release request, the calculation device 400 performs the following process. That is, the calculation device 400 refers to the allocation management table 900, identifies a robot 200 to which the recognition process 601 is assigned and which has not transmitted voice information for a predetermined period of time, and releases the allocation of the recognition process 601 to that robot 200.

According to the first modification, the following effects can be obtained.
(8) When the process control unit 407 receives voice information from the robot 200 to which the recognition process 601 is not assigned, the process control unit 407 assigns a recognition process to the robot. Therefore, the arithmetic device 400 can reduce the processing load of the robot 200 by voluntarily assigning the recognition process 601.

(9) The process control unit 407 releases the allocation of the recognition process to the robot if it does not receive voice information from the robot 200 to which the recognition process 601 is assigned for a period longer than a predetermined time. Therefore, the computing device 400 can reduce the processing load of the robot 200 by voluntarily releasing the allocation of the recognition process 601.

(Variation 2)
In the above-described first embodiment, the arithmetic device 400 assigns only one recognition process to each robot 200. However, the arithmetic device 400 may assign multiple recognition processes to each robot 200. In this case, each robot 200 transmits an activation request according to the number of users existing within a monitoring distance from the robot 200. The robot 200 may output voice information to the arithmetic device 400 by assigning a unique identifier for each direction from which the voice is emitted.

For example, when robot 200-1 receives voices from the left and right directions, it assigns the identifier "200-1L" to the voice information of the voice from the left direction and the identifier "200-1R" to the voice information of the voice from the right direction and transmits the same. The computing device 400 sets the load balancer 603 so that the voice information with the identifiers "200-1L" and "200-1R" is assigned to different recognition processes 601. This second modification has the advantage that each robot 200 can respond immediately even when multiple users are speaking to it at the same time.

(Variation 3)
In the above-described first embodiment, when the arithmetic device 400 receives a request to stop the recognition process 601 from the robot 200, it only reduces the reservation number 803 as shown in step S1308 of Fig. 13, but does not stop the recognition process 601. However, the process stopping unit 408 of the arithmetic device 400 may stop the recognition process 601 when it receives a request to stop the recognition process 601 from the robot 200. In this case, it is preferable that the process stopping unit 408 refers to the standby management table 700 and stops the recognition process 601 with the longest standby time 703.

According to the third modification, the following advantageous effects can be obtained.
(10) The process stopping unit 408 stops the recognition process 601 when a request to stop the recognition process 601 is received from the robot 200. Therefore, unnecessary recognition processes 601 can be stopped early to further save resources.

--Second embodiment--
A second embodiment of the recognition system will be described with reference to Figures 16 and 17. In the following description, the same components as in the first embodiment are given the same reference numerals, and differences will be mainly described. Points that are not particularly described are the same as in the first embodiment. This embodiment differs from the first embodiment mainly in that the robot transmits sensor information directly to the arithmetic unit. The hardware configuration of the voice recognition system in the second embodiment is the same as in the first embodiment, so description will be omitted.

Figure 16 is a configuration diagram of robot 200-1 in the second embodiment, and corresponds to Figure 2 in the first embodiment. The configuration shown in Figure 16 differs from the first embodiment in that it does not include distance determination unit 208, activation request unit 209, and allocation request unit 210.

Figure 17 is a configuration diagram of the computing device 400 in the second embodiment, and corresponds to Figure 4 in the first embodiment. The configuration shown in Figure 17 differs from the first embodiment in that it further includes a distance determination unit 208, a start request unit 209, an allocation request unit 210, and tenant map information 214.

In this embodiment, each robot 200 transmits the sensor information directly to the computing device 400. The computing device 400 receives the sensor information and manages the recognition process 601 by operating the distance determination unit 208, the activation request unit 209, and the allocation request unit 210, as in the first embodiment. That is, in the second embodiment, the computing device 400 controls the recognition process 601 using the received sensor information, so the sensor information can be called "resource control information."

According to the above-described second embodiment, the following advantageous effects can be obtained.
(11) The resource control information includes sensor information that is the output of a sensor mounted on the robot. The process control unit 407 starts the recognition process 601 when it determines, using the sensor information, that a user who is speaking is present within a predetermined monitoring distance from the robot 200. Therefore, the processing load on the robot 200 can be reduced compared to the first embodiment, and even hardware with few resources and low computing power can be used in the recognition system.

(12) The resource control information includes sensor information, which is the output of a sensor mounted on the robot 200. The process control unit 407 stops the recognition process when it determines, using the sensor information, that a user who was present within a predetermined monitoring distance from the robot 200 has moved farther than the monitoring distance. Therefore, the processing load on the robot 200 can be reduced compared to the first embodiment, and even hardware with few resources and low computing power can be used in the recognition system.

(Modification of the second embodiment)
The process stopping unit 408 of the robot 200-1 may stop the recognition process 601 when it determines, using the sensor information, that the distance between the user and the robot 200-1 is farther than the monitoring distance or the stopping distance. For example, when the monitoring distance is 10 m and the stopping distance is 15 m, when the user approaches the robot 200-1 from a distance and comes within 10 m, one recognition process 601 is newly started, and when the user moves farther than 15 m, one recognition process 601 is stopped. The recognition process 601 stopped at this time is the recognition process 601 with the longest waiting time.

--Third embodiment--
A third embodiment of the recognition system will be described with reference to Fig. 18. In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and differences will be mainly described. Points that are not particularly described are the same as those in the first embodiment. This embodiment differs from the first embodiment mainly in the movement of the robot.

The hardware configuration and functional configuration of the voice recognition system S are the same as those of the first embodiment, except for the distance determination unit 208. Therefore, only the operation of the distance determination unit 208 will be explained.

Figure 18 is a flowchart showing the processing of the distance determination unit 208 in the third embodiment. The processing of the distance determination unit 208 executed by the robot 200-1 will be described below. In step S1801, the distance determination unit 208 reads human walking speed information 213, which is information stored in advance. In the following step S1802, the distance determination unit 208 acquires information on the process startup time from the computing device 400. In the following step S1803, the distance determination unit 208 calculates the movement speed of the robot 200-1 at the time when the monitoring distance calculation request is generated. The movement speed of the robot 200-1 can be calculated using various means, but as one example, it can be calculated from the rotation speed of the wheels attached to the robot 200-1.

In step S1804, the distance determination unit 208 calculates the monitoring distance using the process startup time, the human walking speed, and the robot moving speed. If the process startup time is 5 seconds, the human walking speed is 1 m per second, and the robot moving speed is 0.5 m per second, then the monitoring distance is (1 m per second + 0.5 m per second) x 5 seconds, and the monitoring distance is 7.5 m. However, in this case, the vector sum of the user and robot 200-1 may be taken as the relative speed taking into account their moving directions, or, to simplify the calculation and prevent a delay in starting the recognition process 601, the sum of their speeds may be taken as the relative speed without taking into account their moving directions.

According to the above-described third embodiment, the following advantageous effects can be obtained.
(13) The robot 200-1 includes a distance determination unit 208 that calculates the product of the activation time of the recognition process 601 and the relative speed between the user and the robot 200-1 as a monitoring distance. Therefore, by taking into consideration the moving speed of each robot 200, the voice recognition process can be brought into a complete activation state at the time the user speaks even if each robot 200 is moving.

In each of the above-mentioned embodiments and variants, the functional block configurations are merely examples. Several functional configurations shown as separate functional blocks may be configured together, or a configuration shown in a single functional block diagram may be divided into two or more functions. Also, some of the functions of each functional block may be provided by other functional blocks. In particular, the process control unit 407 and the process stopping unit 408 may be integrated into a single functional block that combines the functions of both.

In the above-mentioned embodiments and variants, the programs executed by the robot and the computing device are stored in a ROM (not shown), but the programs may be stored in a non-volatile storage area. The robot and the computing device may also have an input/output interface (not shown), and the programs may be loaded from another device when necessary via the input/output interface and a medium available to the robot and the computing device. Here, the medium refers to, for example, a storage medium that is detachable from the input/output interface, or a communication medium, i.e., a network such as a wired, wireless, or optical network, or a carrier wave or digital signal that propagates through the network. Some or all of the functions realized by the programs may be realized by a hardware circuit or an FPGA.

The above-mentioned embodiments and modifications may be combined with each other. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other aspects that are conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention.

200... Robot 208... Distance determination unit 209... Start request unit 210... Allocation request unit 213... Human walking speed information 400... Calculation device 401... Server communication unit 402... Voice recognition unit 407... Process control unit 408... Process stop unit 601... Recognition process 700... Waiting management table 800... Number management table 900... Allocation management table

Claims (10)

a server communication unit that communicates with a plurality of robots that transmit resource control information indicating the possibility of a voice dialogue with a user and voice information;
a voice recognition unit capable of executing a plurality of recognition processes for converting the voice information received from the robot into text information;
a process control unit that uses the resource control information received from the robot to manage the number of recognition processes being activated and activates and stops the recognition processes. 2. The computing device according to claim 1,
the resource control information includes a request to start the recognition process, which is output by the robot based on a distance between the robot and a user;
The process control unit is a calculation device that activates the recognition process based on the activation request transmitted by the robot, and converts voice information transmitted by the robot into text information by assigning the activated recognition process to the robot. 3. The computing device according to claim 2,
The process control unit is a computing device that assigns the recognition process to the robot when at least one of a request for assignment of the recognition process is received from the robot and when voice information is received from the robot to which the recognition process is not assigned. 4. The computing device according to claim 3,
The process control unit is a computing device that releases the allocation of the recognition process to the robot in at least one of the following cases: when a request to release the recognition process from the robot is received, and when no voice information is received from the robot to which the recognition process is allocated for a predetermined period of time. 2. The computing device according to claim 1,
the resource control information includes sensor information that is an output of a sensor mounted on the robot;
The process control unit is a computing device that initiates the recognition process when it determines, using the sensor information, that a user speaking is present within a predetermined monitoring distance from the robot. 2. The computing device according to claim 1,
The process control unit is a computing device that stops the recognition process when a request to stop the recognition process is received from the robot. 2. The computing device according to claim 1,
the resource control information includes sensor information that is an output of a sensor mounted on the robot;
The process control unit is a computing device that stops the recognition process when it determines using the sensor information that a user who was within a predetermined monitoring distance from the robot has moved farther than the monitoring distance. A computer having a server communication unit that communicates with a plurality of robots that transmit resource control information indicating a possibility of a voice dialogue with a user and voice information,
a voice recognition process capable of executing a plurality of recognition processes for converting the voice information received from the robot into text information;
A computer-readable recording medium having recorded thereon a program for executing a process control process that manages the number of recognition processes started using the resource control information received from the robot and starts and stops the recognition processes. a robot communication unit capable of communicating with a computing device having a voice recognition unit capable of executing a plurality of recognition processes for converting received voice information into text information;
an activation request unit that transmits an activation request to the arithmetic device to request activation of the recognition process based on a distance to a user;
A voice input device comprising: a transmitting unit that records an utterance of the user and transmits the utterance as the voice information to the arithmetic device,
a distance determination unit that calculates a monitoring distance by multiplying an activation time of the recognition process by a relative speed between the user and the voice input device;
The activation request unit is a voice input device that transmits the activation request when the distance between the user and the voice input device is equal to or less than the monitoring distance. 10. The voice input device according to claim 9,
The voice input device, wherein the start request unit transmits a stop request to stop the recognition process when the distance between the user and the voice input device becomes greater than the monitoring distance.

Images