JP6858334B2 - Electronic devices and their control methods - Google Patents

Description

An embodiment of the present invention relates to an electronic device for controlling a plurality of devices by voice and a control method thereof in the field of home automation in homes, offices, and small businesses.

Conventionally, in the field of home automation, there are voice recognition devices and methods for operating and controlling various devices in homes, offices, and small businesses by voice input.

This voice recognition device and method analyzes the voice input from the user to determine whether or not the input voice is the voice that turns on the function of the device, or the function of the device. When it is determined that the voice is turned on, the content of the continuous voice is analyzed and processing is performed based on the analysis result. In addition, by recognizing the characteristics of the voice input from the user, the user who has emitted the voice may be identified and processing suitable for that user may be performed.

International Publication No. 2015/029379 International Publication No. 2015/033523

As a form of a home automation system, each device is connected to each other by a network in the home, and a host device that totally controls a plurality of connected devices is connected to the network. In this case, the host device controls the operation of each device connected by the network, and collects information about each device and manages it so that the user can view it in a unified manner.

By instructing the host device by voice, for example, the user can control each device connected to the host device via a network, and can centrally browse information about each connected device. it can.

In such a form of home automation system, the devices to be controlled can be easily connected to the network, so that the number and types of connected devices tend to be large. In addition, there is a tendency that new participation in the network, setting change, and withdrawal from the network occur frequently due to addition, change, version upgrade, movement and disposal of the installation location, etc. of the device to be controlled. In addition, since there are many types of operating contents and specifications of connected devices, there is a tendency to use home automation systems in homes and offices regardless of age or sex. This tendency is becoming more and more prominent, especially with the recent miniaturization of devices and sensors with a wide variety of functions.

However, conventional home automation systems are not sufficient for controlling a wide variety of devices and supporting a wide range of users. For example, when a home automation system is used in a home, it cannot be said that the devices that match the lifestyle of each family member are sufficiently controlled.

This embodiment has been made in view of the above problems, and proposes an electronic device and a control method thereof for controlling a wide variety of devices connected by a network so as to match the individual lifestyle of the user. The purpose.

The electronic device of the embodiment is one or a plurality of devices based on the content of the first voice input from the outside and the content of the second voice input after the first voice is input. In the electronic device for determining the execution of the control of, the determination voice data for determining that the first voice is a desired voice is created and managed by the voice input from the outside a plurality of times, and is created and managed. The management means for determining that the first voice is a desired voice using the determination voice data, and the control of the one or a plurality of devices are executed based on the content of the second voice. When it is determined by the management means that the first voice is a desired voice by using the determination voice data, the control means adjusts the content of the second voice. Based on this, it has an output unit that executes control of the one or a plurality of devices and outputs a determination result by the management means by voice, and the first voice outputs the determination voice data by the management means. When it is determined that the voice is desired by using the data, the output unit outputs to that effect, and the management means uses the judgment voice data to determine that the first voice is the desired data. At the time of determination, the content to be output from the output unit is changed according to one of the plurality of criteria of the determination criterion 2 which has the determination criterion 2 having a plurality of criteria and the determination result satisfies.

FIG. 1 is a diagram showing an example of an overall image of a home automation system according to an embodiment. FIG. 2 is a list showing another example of the sensor according to one embodiment. FIG. 3 is a diagram showing an example of a host device according to an embodiment. FIG. 4 is a functional block diagram of the host device according to the embodiment. FIG. 5A is a diagram showing an example of a processing sequence in the registration of reserved words according to one embodiment. FIG. 5B is a diagram showing an example of a processing sequence in the registration of reserved words according to the embodiment. FIG. 6A is a diagram showing an example of a processing sequence in the registration of reserved words according to one embodiment. FIG. 6B is a diagram showing an example of a processing sequence in the registration of reserved words according to one embodiment. FIG. 7A is a diagram showing an example of a processing sequence in the registration of reserved words according to one embodiment. FIG. 7B is a diagram showing an example of a processing sequence in the registration of reserved words according to the embodiment. FIG. 8A is a diagram showing an example of a processing sequence in recognizing a reserved word according to an embodiment. FIG. 8B is a diagram showing an example of a processing sequence in recognizing a reserved word according to an embodiment. FIG. 9A is a diagram showing an example of a processing sequence in recognizing a reserved word according to an embodiment. FIG. 9B is a diagram showing an example of a processing sequence in recognizing a reserved word according to an embodiment. FIG. 10A is a diagram showing an example of a processing sequence for controlling the corresponding device or sensor based on the words continuously issued by the user after recognizing the reserved word according to the embodiment. is there. FIG. 10B is a diagram showing an example of a processing sequence for controlling the corresponding device or sensor based on the words continuously issued by the user after recognizing the reserved word according to the embodiment. is there. FIG. 11A is a diagram showing an example of a processing sequence in which a word for controlling a device or a sensor continuously emitted by a user is continued within a certain period of time after recognizing a reserved word according to an embodiment. FIG. 11B is a diagram showing an example of a processing sequence in which a word for controlling a device or a sensor continuously emitted by a user is continued within a certain period of time after recognizing a reserved word according to an embodiment. FIG. 12A is a diagram showing an example of a processing sequence when a word for continuously controlling a device or a sensor issued by a user is continued for a certain period of time after recognizing a reserved word according to an embodiment. .. FIG. 12B is a diagram showing an example of a processing sequence in which the words that continuously control the device or sensor issued by the user are continued for a certain period of time after the reserved words according to the embodiment are recognized. .. FIG. 13 is a list concretely showing the contents of the control information used when controlling the device or the sensor after recognizing the reserved word according to the embodiment. FIG. 14 is a list showing an example of the operation content to be changed according to a plurality of reserved words according to the embodiment. FIG. 15A is a diagram showing an example of a processing sequence in which, in the registration of a plurality of reserved words according to one embodiment, the operation contents to be changed according to each reserved word are also registered. FIG. 15B is a diagram showing an example of a processing sequence in which, in the registration of a plurality of reserved words according to one embodiment, the operation contents to be changed according to each reserved word are also registered. FIG. 16A is a diagram showing an example of a processing sequence for setting an operation content according to each reserved word in recognizing a reserved word according to an embodiment. FIG. 16B is a diagram showing an example of a processing sequence for setting an operation content according to each reserved word in recognizing a reserved word according to an embodiment. FIG. 17 is a list showing an example of an operation content set according to a word following the reserved word in the reserved word according to the embodiment. FIG. 18A is a diagram showing an example of a processing sequence in which an operation content is set according to a word following the reserved word in the recognition of the registered reserved word according to the embodiment. FIG. 18B is a diagram showing an example of a processing sequence in which an operation content is set according to a word following the reserved word in the recognition of the registered reserved word according to the embodiment. FIG. 18C is a diagram showing an example of a processing sequence for setting an operation content according to a word following the reserved word in the recognition of the registered reserved word according to the embodiment. FIG. 18D is a diagram showing another example of a processing sequence for setting an operation content according to a word following the reserved word in the recognition of the registered reserved word according to the embodiment. FIG. 18E is a diagram showing another example of a processing sequence for setting an operation content according to a word following the reserved word in the recognition of the registered reserved word according to the embodiment. FIG. 19A is a diagram showing an example of a processing sequence in which an operation content is set according to a word following the recognized reserved word in the recognition of the reserved word according to the embodiment. FIG. 19B is a diagram showing an example of a processing sequence in which an operation content is set according to a word following the recognized reserved word in the recognition of the reserved word according to the embodiment. FIG. 20 is a list showing an example of the types of speech recognition dictionaries used according to the reserved words in the recognition of a plurality of reserved words according to the embodiment. FIG. 21A is a diagram showing an example of a processing sequence for changing the type of the speech recognition dictionary used according to the reserved words in the recognition of a plurality of reserved words according to the embodiment. FIG. 21B is a diagram showing an example of a processing sequence for changing the type of the speech recognition dictionary used according to the reserved words in the recognition of a plurality of reserved words according to the embodiment. FIG. 22 is a list showing an example of changing the words following the reserved words, the operation contents to be set according to the reserved words, and the type of the voice recognition dictionary to be used in the recognition of the plurality of reserved words according to the embodiment. FIG. 23 is a list showing an example of changing the type of the voice recognition dictionary according to the embodiment according to the contents other than the reserved words. FIG. 24 is a diagram showing a sequence of processes for registering the type of the voice recognition dictionary to be changed according to the contents other than the reserved words in the change of the type of the voice recognition dictionary according to the embodiment. FIG. 25 is a diagram showing a processing sequence when the type of the voice recognition dictionary to be registered is changed according to the contents other than the reserved words in the change of the type of the voice recognition dictionary according to the embodiment. FIG. 26 shows a reserved word (for relief) for displaying a reserved word when the user forgets the registered reserved word in the process according to the embodiment, and a reserved word corresponding to the reserved word. A list showing examples of ranges. FIG. 27 is a functional block diagram of the host device according to the embodiment. FIG. 28 shows the host device 332 when a scene for registering a reserved word, an additional word, or additional information occurs, or when a scene for recognizing a reserved word or an additional word occurs in the process according to the embodiment. It is a figure which shows an example of the lapse of time in the case of recording or recording a registered scene or a recognition scene. FIG. 29 is a diagram showing an example of how the data to be reproduced when reproducing each data of the recorded or recorded scene according to the embodiment is displayed.

FIG. 1 is a diagram showing an example of the overall configuration of the home automation system according to the present embodiment. The home automation system includes a cloud server 1 consisting of a group of servers placed in the cloud, various sensors 310 and various equipment 320 connected to each other via a network 333 via a host device 332 having an HGW (HomeGate Way) function. It consists of a home 3 in which various home appliances 340 are arranged, and an Internet 2 that connects a cloud server 1 and a host device 332.

The home 1 is a small-scale home or office where various sensors 310, various equipment 320s, and various home appliances 340 connected to each other by a home network 333 are arranged via a host device 332 having an HGW function. It is a business establishment, and its scale does not matter.

The host device 332 controls the devices and sensors connected by the network 333 based on the preset information and the information notified from the sensors connected by the network 333, and also relates to each device and the sensor. It has a function to centrally manage information.

Further, the host device 332 is provided with a microphone and can capture the words uttered by the user 331. When the host device 332 recognizes a predetermined keyword (hereinafter referred to as a reserved word) from the words uttered by the user 331, the host device 332 captures the word uttered by the user 331 following the reserved word, and the captured word is used. By analyzing the contents, it has a function of returning a response according to the analysis result to the user 331, or controlling a device or a sensor connected by the network 333 according to the analysis result.

On the contrary, the host device 332 does not continuously capture the words uttered by the user 331 unless the reserved words are recognized from the words uttered by the user 331. As a result, the host device 332 is prevented from picking up unnecessary voices around it and operating.

The reserved word is recognized in the host device 332, the reserved word is continuously captured by the user 331, and the content of the captured word is analyzed in the cloud server 1. Details of the functions of the host device 332 will be described later.

For convenience of explanation, the various equipment 320 and the various home appliances 340 mean the equipment in which the equipment 320 is not so easy to move, and the various home appliances 340 mean the equipment in which the equipment 340 is relatively easy to move. The names of the illustrated equipment and home appliances do not limit the capabilities and functions of individual equipment.

Specific examples of the various sensors 310 include a security camera 311, a fire alarm 312, a motion sensor 313, and a temperature sensor 314. Further, specific examples of various equipment 320320 include an intercom 325, a lighting 326, an air conditioner 327, and a water heater 328. Further, specific examples of various home appliances 340 include a washing machine 341, a refrigerator 342, a microwave oven 343, an electric fan 344, a rice cooker 345, and a television 346.

FIG. 2 shows other examples of the various sensors 310 shown in FIG.

FIG. 3 shows various examples of the host device 332 shown in FIG.

The host device 332-1 is the host device 332 shown in FIG. 1, and is an example of a stationary type having a built-in HGW function. The host device 332-1 is connected to other devices and sensors arranged in the home 1 through the network 333, and is also connected to the cloud server 1 through the Internet 2. Since the host device 332-1 is a stationary type, it is an example in which a means for autonomous movement such as a motor is not mounted.

The host device 332-2 is an example of a stationary type that does not have a built-in HGW function. Therefore, the host device 332-2 is connected to the HGW 330 through the network 333. The host device 332-2 is connected to other devices and sensors arranged in the home 1 through the network 333 via the HGW 330, and is connected to the cloud server 1 via the Internet 2 via the HGW 330. Since the host device 332-2 is a stationary type, it is an example in which a means for autonomous movement such as a motor is not mounted.

The host device 332-3 is a movable example having a built-in HGW function. The host device 332-3 is connected to other devices and sensors through the network 333, and is also connected to the cloud server 1 through the Internet 2. Since the host device 332-3 is a movable type, it is an example in which a means for autonomous movement such as a motor is mounted.

The host device 332-4 is an example of a movable type that does not have a built-in HGW function. Therefore, the host device 332-4 is connected to the HGW 330 through the network 333. The host device 332-4 is connected to other devices and sensors via the network 333 via the HGW 330, and is connected to the cloud server 1 via the Internet 2 via the HGW 330. Since the host device 332-4 is a movable type, it is an example in which a means for autonomous movement such as a motor is mounted.

FIG. 4 shows the functional blocks of the host device 332 shown in FIG. The host device 332 connects to the system controller 402 that controls the entire internal processing, and the control management unit 401, the trigger setting unit 403, the trigger recognition unit 405, the input management unit 420, and the network 333 that control each function by the system controller 402. It has a network I / F427 of. The control management unit 401 internally manages a plurality of applications for controlling various operations of the host device 332, such as APP-Mg401-1, initial setting of each functional block of the host device 332, various state settings, and operation settings. It is composed of CONF-Mg401-2 that manages the setting contents of.

Further, the host device 332 serves as an interface (I / F) with the user 331 for the microphone 421 for capturing the words uttered by the user 331, the speaker 423 for outputting the response to the user 331 by voice, and the user 331. It also has a display unit 425 for notifying the status of the host device 332.

The microphone 421 is connected to the input management unit 420. The input management unit 420 controls whether to send the voice data input from the microphone 421 to the trigger setting unit 403, the trigger recognition unit 405, or the voice processing unit 407 according to the state of internal management. The display unit 425 notifies the user 331 of the status of the host device 332, and is, for example, an LED (LightEmittingIDE) or an LCD (Liquid Crystal Display).

The memory 410 is divided into three areas: an operation mode storage area 410-1, a reserved word storage area 410-2, and a voice storage area 410-3. The contents of the information stored in each area will be described later.

As described above, when the function of the host device 332 recognizes the reserved word from the words uttered by the user 331, it captures the word uttered by the user 331 following the reserved word and captures the content of the captured word. By analyzing, it has a function of returning a response according to the analysis result to the user 331 and controlling the operation of devices and sensors connected through the network 333.

In order to realize these functions, the host device 332 performs four major processes. The first process is the registration of reserved words. The second process is the recognition of reserved words. The third process is the registration of the control contents of the devices and sensors that control the operation. The fourth process is the control of devices and sensors whose control contents are registered.

First, the registration of reserved words, which is the first process, will be described.
The host device 332 has a function of registering a reserved word in the host device 332. In order to register the reserved words, the host device 332 has a mode for registering the reserved words (hereinafter referred to as a reserved word registration mode).

5A and 5B show an example of a processing sequence of the host device 332 from the start of registration of the reserved word to the completion of registration in the state where the host device 332 has transitioned to the "reserved word registration mode" in order to register the reserved word. Is shown.

The host device 332 may be able to change the mode by recognizing the words uttered by the user 331 in a predetermined order for changing the mode. Alternatively, a menu screen may be displayed on the display unit 425, and the mode may be changed by the user 331 operating the menu screen. Alternatively, the mode may be changed by the user 331 operating the menu screen for changing the mode of the host device 332 displayed on the smartphone or tablet connected via the network I / F 427.

When the user 331 issues a word to be registered as a reserved word, the host device 332 captures the voice data input from the microphone 421 into the input management unit 420 (S501). The input management unit 420 has a function of determining a transfer destination of input voice data according to a state managed internally. When the mode of the host device 332 is the setting mode, the input management unit 420 transfers the received voice data to the trigger setting unit 403 (S502). The trigger setting unit 403 saves the received voice data in the voice storage area 410-3 of the memory 410 (S503), and confirms whether the number of times the user 331 has captured the voice has reached the specified number (S504). ..

As a result of checking whether the number of times the user 331 has captured the voice has reached the specified number of times, the trigger setting unit 403 displays a display prompting the user 331 to say a word to be registered when it is determined that the specified number of times has not been reached. At the same time as performing (S507), an input continuation notification is sent to the input management unit 420 (S506). Upon receiving the input continuation notification, the input management unit 420 shifts the internal state to the state of waiting for input of voice from the microphone (S500).

The display prompting the user 331 to input the word to be registered is a display in which the trigger setting unit 403 sends a registration incomplete notification to the display device 425 (S505) and receives the registration incomplete notification. It is desirable that the device 425 blinks the light emitting diode (LED) in red (S507) in a display method that can be recognized by the user 331. Further, the user 331 may be prompted to input the word to be registered by using the voice method instead of the display method. In this case, the trigger setting unit 403 sends a registration incomplete notification to the speaker 423, and the speaker 423 receiving the registration incomplete notification announces, for example, "Please enter again" to the user 331. But it may be. Alternatively, the trigger setting unit 403 may use both a display method and a voice method for prompting the user 331 to input a word to be registered. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may issue an instruction to a moving means not described so that the host device 332 repeatedly rotates and moves, for example, with a certain angle width.

When the trigger setting unit 403 determines that the specified number of times has been reached as a result of checking whether the number of times the user 331 has captured the voice has reached the specified number of times, the trigger setting unit 403 stores the voice in the voice storage area 410-3 by then. A certain voice data is read out (S508) and sent to the recognition data conversion unit 101-1 in the voice recognition cloud 101 in the cloud server 1 via the Internet 2 (S509).

The recognition data conversion unit 101-1 converts the voice data sent from the trigger setting unit 403 into recognition data for recognizing as a reserved word (S510). When the conversion to the recognition data is completed, the recognition data conversion unit (101-1) sends the recognition data to the trigger setting unit 403 (S511) via the Internet 2. The trigger setting unit 403 that has received the recognition data saves the received data in the reserved word storage area 410-2 of the memory 410 (S512).

The trigger setting unit 403 displays (S514) notifying the user 331 that the registration of the reserved word is completed. As for the display notifying the user 331 that the registration of the reserved word is completed, the trigger setting unit 403 sends a registration completion notification to the display device 425 (S514), and the display device 425 receiving the registration completion notification sends the registration completion notification. For example, it is desirable to use a display method that can be recognized by the user 331, such as turning on the LED in green. Alternatively, the trigger setting unit 403 may use a voice method instead of the display method to notify the user 331 that the registration of the reserved word is completed. In this case, the trigger setting unit 403 may send a registration completion notification to the speaker 423, and the speaker 423 receiving the registration completion notification may, for example, announce to the user 331 that "registration is completed". .. Alternatively, the trigger setting unit 403 may use both the display method and the voice method to notify the user 331 that the registration of the reserved word is completed. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may issue an instruction to a moving means not described so that the host device 332 repeatedly moves linearly with a certain movement width, for example.

As described above, the trigger setting unit 403 has a role of managing the data flow in the registration of reserved words.

6A and 6B show another example of the sequence from the start of registration of the reserved word to the completion of registration. It may be insufficient to register the voice data captured by the host device 332 as a reserved word. An example of processing when the captured data is insufficient is shown.

The processes of S600 to S615 shown in FIGS. 6A and 6B are the same as the processes of S500 to S515 shown in FIGS. 5A and 5B, respectively. The difference between the processes in FIGS. 5A and 5B and the processes in FIGS. 6A and 6B is that the process of S616 is added to the process of S619 to the process of FIG. 6B.

When the trigger setting unit 403 confirms (S604) whether the number of times the user 331 has taken in the words has reached the specified number of times, and determines that the number of times has reached the specified number of times, the voice storage area by then. The voice data stored in 410-3 is read out (S608) and sent to the recognition data conversion unit 101-1 in the voice recognition cloud 101 in the cloud server 1 via the Internet 2 (S609).

When the trigger setting unit 403 determines that the number of times the word uttered by the user 331 has been captured has not reached the specified number of times, the trigger setting unit 403 displays a display prompting the user 331 to utter the word to be registered (S607) and an input management unit. An input continuation notification is sent to 420 (S606). Upon receiving the input continuation notification, the input management unit 420 shifts the internal state to the state of waiting for input of voice from the microphone (S600).

The display prompting the user 331 to input the word to be registered is a display in which the trigger setting unit 403 sends a registration incomplete notification to the display device 425 (S605) and receives the registration incomplete notification. It is desirable that the device 425 blinks the LED in red (S607) in a display method that can be recognized by the user 331. Further, the user 331 may be prompted to input the word to be registered by using the voice method instead of the display method. In this case, the trigger setting unit 403 sends a registration incomplete notification to the speaker 423, and the speaker 423 receiving the registration incomplete notification announces, for example, "Please enter again" to the user 331. But it may be. Alternatively, the trigger setting unit 403 may use both a display method and a voice method for prompting the user 331 to input a word to be registered. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may issue an instruction to a moving means not described so that the host device 332 repeatedly rotates and moves, for example, with a certain angle width.

The recognition data conversion unit 101-1 determines whether or not the sent voice data can be converted into recognition data when converting all the voice data sent from the trigger setting unit 420 into recognition data. (S616). When it is determined that some of the sent voice data cannot be converted into recognition data, the recognition data conversion unit 101-1 transmits a voice data addition request to the trigger setting unit 403 via the Internet 2 (S617). To do. Upon receiving the voice data addition request, the trigger setting unit 403 sets the number of times the user 331 is additionally input with the word to be registered as a reserved word (S618), and notifies the input management unit 420 of the input continuation (S619). Notify.

At the time when the trigger setting unit 403 sets the additional number of times for the user 331 to additionally input (S618), for example, the LED of the display unit 425 remains lit in red. According to this display, the user 331 issues a word to be registered as a reserved word for the number of times additionally set in S618.

When the input management unit 420 receives the input continuation notification (S619), the input management unit 420 shifts the internal state to the input waiting state (S600), and enters the input waiting state of the word uttered by the user 331.

In the processes shown in FIGS. 5A and 5B and the processes shown in FIGS. 6A and 6B, after the number of times the input management unit 402 has captured the voice emitted by the user 331 reaches the specified number of times, the captured voice data is collectively collected. This is an example of transmitting to the recognition data conversion unit 101-1 in the cloud server 1, but each time the input management unit 420 captures the voice emitted by the user 331, the captured voice data is captured by the recognition data conversion unit 101-. It may be transmitted to 1. 7A and 7B show that each time the input management unit 420 captures the voice emitted by the user 331, the captured voice data is sequentially sent to the recognition data conversion unit 101-1 in the cloud server 1 for recognition. This is an example of a sequence when converting to data.

The processes of S700 to S702 shown in FIG. 7A are the same as the processes shown in S500 to S502 shown in FIG. 5A, respectively. The processing of S703 and S704 shown in FIG. 7A is the same as the processing of S505 and S507 shown in FIG. 5A, respectively.

When the user 331 issues a word to be registered as a reserved word, the host device 332 captures the voice data input from the microphone 421 into the input management unit 420 (S701). Since the mode of the host device 332 is the reserved word registration mode, the input management unit 420 transfers the received voice data to the trigger setting unit 403 (S702). The trigger setting unit 403 sequentially transmits the received voice data to the recognition data conversion unit 101-1 in the cloud server 1 each time it is received (S706). When the recognition data conversion unit 101-1 converts the voice data sent from the trigger setting unit 403 into the recognition data, the recognition data conversion unit 101-1 determines whether or not the sent voice data can be converted into the recognition data ( S707).

When it is determined that the transmitted voice data cannot be converted into the recognition data, the recognition data conversion unit 101-1 transmits a voice data addition request to the trigger setting unit 403 via the Internet 2 (S708). Upon receiving the voice data addition request, the trigger setting unit 403 (S708) confirms (S714) whether the number of times the voice of the user 331 has been captured has reached the specified number of times. As a result of checking whether the number of times the user 331 has captured the voice has reached the specified number of times, the trigger setting unit 403 displays a display prompting the user 331 to say a word to be registered when it is determined that the specified number of times has not been reached. By continuing and sending an input continuation notification to the input management unit 420 (S715), the input management unit 420 is transitioned to a state of waiting for input of voice from the microphone (S700). When the input management unit 420 receives the input continuation notification (S715), the input management unit 420 shifts the internal state to the input waiting state (S700), and enters the input waiting state of the word uttered by the user 331.

When the recognition data conversion unit 101-1 determines that the sent voice data can be converted into the recognition data (S707), the recognition data conversion unit 101-1 converts the voice data into the recognition data (S709). As a result of converting the recognition data conversion unit 101-1 into the recognition data (S709), the recognition data conversion unit 101-1 uses all the recognition data including the data already converted into the recognition data, and uses the voice data input from the microphone 421. It is determined whether or not the accuracy that can be recognized as a reserved word is secured (S710).

When it is determined from all the recognition data that the voice data input from the microphone 421 is sufficiently accurate to be recognized as a reserved word, the user 331 issues a word to be registered as a reserved word. In order to have the user stop, the trigger setting unit 403 is notified of the recognition data (with the recognition data satisfaction notification) to which the information that the recognition data is sufficient is added through the Internet 2 (S711). The trigger setting unit 403 that has received the recognition data (satisfaction of the recognition data) is the recognition data received up to this point and is sufficient for recognizing the voice data input from the microphone 421 as a reserved word. Is recognized, and even if the number of times the user 331 has captured the voice has not reached the specified number of times, the user 331 is stopped from being prompted to input the words to be registered (S712). The trigger setting unit 403 stores all the recognition data received up to this point in the reserved word storage area 410-2 (S716), and also in the input management unit 420, the display unit 425, and the recognition data conversion unit 101-1. A registration completion notification is sent (S717) (S718) (S719). As a result, due to the accuracy of the converted recognition data, it is possible to stop the user 331 from issuing a word to be registered as a reserved word when the number of times the user 331 has captured the voice does not reach the specified number of times. , It is possible to register reserved words with more freedom. The specified number of times can be changed by the user 331 as a set value of the host device 332, and can be changed as one of the additional information described later.

When the recognition data conversion unit 101-1 determines that the recognition data created up to this point does not ensure sufficient accuracy to recognize the voice data input from the microphone 421 as a reserved word. , Only the converted recognition data is sent to the trigger setting unit 403 (S713). The trigger setting unit 403 that has received the recognition data confirms (S714) whether the number of times the voice of the user 331 has been captured has reached the specified number of times. When the trigger setting unit 403 determines that the specified number of times has not been reached as a result of checking whether the specified number of times has been reached, the trigger setting unit 403 continues the display prompting the user 331 to say the word to be registered, and the input management unit 420 By sending the input continuation notification (S715), the input management unit 420 is changed to the state of waiting for the input of the voice from the microphone (S700).

The display prompting the user 331 to input the word to be registered is a display in which the trigger setting unit 403 sends a registration incomplete notification to the display device 425 (S703) and receives the registration incomplete notification. It is desirable that the device 425 blinks the LED in red (S704) in a display method that can be recognized by the user 331. Further, the user 331 may be prompted to input the word to be registered by using the voice method instead of the display method. In this case, the trigger setting unit 403 sends a registration incomplete notification to the speaker 423, and the speaker 423 receiving the registration incomplete notification announces, for example, "Please enter again" to the user 331. But it may be. Alternatively, the trigger setting unit 403 may use both a display method and a voice method for prompting the user 331 to input a word to be registered. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may issue an instruction to a moving means not described so that the host device 332 repeatedly rotates and moves, for example, with a certain angle width.

When the trigger setting unit 403 that has received the recognition data determines that the specified number of times has been reached as a result of checking whether the specified number of times has been reached (S714), the registration completion notification is sent to the input management unit 420, the display unit 425, and the recognition unit. A registration completion notification is sent to the data conversion unit 101-1 for use (S717) (S718) (S719). The recognition data conversion unit 101-1 that has received the registration completion notification (S718) clears the converted recognition data temporarily stored for processing S710.

Next, the recognition of reserved words, which is the second process of the host device 332, will be described.

When the host device 332 recognizes a reserved word from the words uttered by the user 331, the host device 332 analyzes the content of the words uttered by the continuing user 331 and controls the device or the sensor based on the analysis result. It has a function. In order to recognize the reserved word and control the device or sensor after recognizing the reserved word, the host device 332 has a mode for recognizing the reserved word and controlling the device or sensor (hereinafter referred to as an operation mode). ing.

8A and 8B show an example of a processing sequence of the host device 332 until the word uttered by the user 331 is recognized as one of the registered reserved words in the operation mode.

When the user 331 speaks a word, the host device 332 captures the voice data input from the microphone 421 into the input management unit 420 (S801). When the mode of the host device 332 is the operation mode, the input management unit 420 transfers the received voice data to the trigger recognition unit 405 (S802). When the trigger recognition unit 405 receives the voice data transferred from the input management unit 420, the trigger recognition unit 405 determines whether or not the transferred voice data is a reserved word, so that the reserved word storage area 410-2 of the memory 410 is used. It is compared with the recognition data read from (S803) (S804).

When the trigger recognition unit 405 determines that the input voice data cannot be recognized as a reserved word (S805), the trigger recognition unit 405 displays a display prompting the user 331 to issue the reserved word (S808), and inputs the input to the input management unit 420. A continuation notice is sent (S807). As for the display prompting the user 331 to issue the reserved word, the trigger recognition unit 405 transmits a recognition incomplete notification to the display unit 425 (S806), and the display unit 425 receiving the recognition incomplete notification is, for example. It is desirable to use a display method that can be recognized by the user 3313, such as blinking the LED in red (S808). Further, the trigger setting unit 403 may prompt the user 331 to input the voice by using the voice method instead of the display method. In this case, the trigger recognition unit 405 also transmits a recognition incomplete notification to the speaker 423, and the speaker 423 receiving the recognition incomplete notification may announce to the user 331, for example, "I did not hear". Good. Alternatively, the trigger recognition unit 405 may use both the display method and the voice method to prompt the user 331 to input the voice. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may issue an instruction to a moving means not described so that the host device 332 repeatedly rotates and moves, for example, with a certain angle width.

When the input voice data can be recognized as a reserved word (S805), the trigger recognition unit 405 displays that the voice uttered by the user 331 is recognized as a reserved word (S810). The display indicating that the voice emitted by the user 331 is recognized as a reserved word is displayed by the trigger recognition unit 403 transmitting a recognition completion notification to the display device 425 (S809) and receiving the recognition completion notification. It is desirable that the display method be recognizable by the user 331, for example, the 425 turns on the LED in green (S810). Further, the trigger recognition unit 405 may notify that the voice uttered by the user 331 is recognized as a reserved word by using the voice method instead of the display method. In this case, the trigger recognition unit 405 also transmits a recognition completion notification to the speaker 423, and the speaker 423 receiving the recognition completion notification may announce to the user 331, for example, "Yes Yes" or "I heard". Good. Alternatively, the trigger recognition unit 405 may use both the display method and the voice method to indicate that the voice emitted by the user 331 is recognized as a reserved word. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may issue an instruction to a moving means not described so that the host device 332 repeatedly moves linearly with a certain movement width, for example.

9A and 9B are other examples of the processing sequence of the host device 332 until the word uttered by the user 331 is recognized as one of the registered reserved words in the operation mode.

The difference between the sequence examples of FIGS. 9A and 9B and the sequence examples of FIGS. 8A and 8B is that the recognition probability is taken into consideration in the process of recognizing the reserved word. The recognition probability means that the recognition data and the characteristic points such as the frequency component and the strength of the voice data transferred from the input management unit 420 are compared, and the two are at the same level. The processes of S900 to S912 shown in FIGS. 9A and 9B are the same as the processes shown in S800 to S812, respectively, and the processes in FIGS. 9A and 9B differ from the processes of FIGS. 8A and 8B in S913 to S916. The point is that processing has been added.

When the trigger recognition unit 405 receives the voice data transferred from the input management unit 420, the trigger recognition unit 405 reads the recognition data from the reserved word storage area 410-2 of the memory 410 (S903) and transfers the data from the input management unit 420. Comparison with audio data is performed (S904).

When the trigger recognition unit 405 determines that the input voice data can be recognized as a reserved word (S905), the trigger recognition unit 405 moves to the recognition probability determination process (S913).

Here, the voice recognition process performed by the trigger recognition unit 405 is characterized by characteristics such as frequency components and strengths of the recognition data read from the reserved word storage area 410-2 of the memory 410 and the voice data transferred from the input management unit 420. When the points are compared with each other and they match at a certain level or more, it is determined that the voice data transferred from the input management unit 420 is the recognition data.

When the host device 332 compares the recognition data with the feature points such as the frequency component and the strength of the voice data transferred from the input control unit 420, the host device 332 sets a threshold value for determining a level at which the two match. It is possible to provide more than one. By doing so, when the host device 332 recognizes the reserved word from the words uttered by the user, it is not determined in two ways that the reserved word can be recognized / the reserved word cannot be recognized. For example, it is possible to add a judgment that the reserved word is close to the reserved word but is not the correct reserved word, such as the reserved word cannot be recognized / the reserved word cannot be recognized / the reserved word cannot be recognized. By providing a plurality of recognition probability threshold values in this way, when the user 331 does not remember the reserved word accurately, for example, the user 331 repeatedly utters a word close to the reserved word to capture the word uttered by the user 331. However, the host device 332 responds according to the determination result that "it cannot be said that the reserved word can be recognized", and the user 331 who sees the response content can approach the correct reserved word.

The examples of FIGS. 9A and 9B are examples in which two threshold values of recognition probability are provided. Assuming that the threshold value at which the reserved word can be recognized is the threshold value 1 and the threshold value at which the reserved word cannot be recognized is the threshold value 0, the comparison result in S904 shows that if the recognition probability is the threshold value 1 or more, the reserved word can be recognized. It becomes. Further, when the recognition probability is equal to or more than the threshold value and less than the threshold value 1, it is determined that the reserved word cannot be recognized. If the recognition probability is less than the threshold value 0, it is determined that the reserved word cannot be recognized. Therefore, the process of S905 is a process of comparing the recognition probability with the threshold value of 0. Further, the process of S913 is a process of comparing the recognition probability with the threshold value 1.

When the host device 332 determines that the recognition probability is equal to or more than the threshold value and less than the threshold value 1 (S913), the host device 332 displays a display prompting the user 331 to issue a reserved word (S915), and continues input to the input management unit 420. A notification is sent (S916). As for the display prompting the user 331 to issue the reserved word, the trigger recognition unit 405 sends a recognition insufficiency notification to the display unit 425 (S914), and the display unit 425 receiving the recognition insufficiency notification is, for example. It is desirable to use a display method that can be recognized by the user 331, such as blinking the LED in green (S915).

In this way, when the recognition probability is low, the display prompting the user 331 to issue a reserved word is changed from the display when recognition fails (S908) and the display when recognition succeeds (S910). The user 331 can recognize that the word he / she has spoken is close to the reserved word but does not correctly speak the reserved word.

Further, the trigger setting unit 403 may prompt the user 331 to input the voice by using the voice method instead of the display method. In this case, the trigger recognition unit 405 transmits an insufficient recognition notification to the speaker 423 (S914), and the speaker 423 receiving the insufficient recognition notification asks the user 331, for example, "Did you call something?" It may be an announcement method. Alternatively, the trigger recognition unit 405 may use both the display method and the voice method to prompt the user 331 to input the voice. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may issue an instruction to a moving means not described so that the host device 332 repeatedly rotates and moves, for example, with a certain angle width.

Next, regarding the registration of the control content of the device or sensor that controls the operation, which is the third process of the host device 332, and the control of the device or sensor whose control content is registered, which is the fourth process. explain.

First, the overall picture of the control of the device and the sensor using the host device 332 will be described.

When the host device 332 recognizes a reserved word from the words uttered by the user 331, the host device 332 continuously captures the words uttered by the user after recognizing the reserved words, and analyzes the contents of the captured words. It has a function to control devices and sensors.

10A and 10B show a case where the host device controls the device or sensor based on the content of voice data including the control content of the device or sensor captured from the microphone 421 after the recognition of the reserved word is completed. An example of the processing sequence of is shown. The internal state of the input management unit 420 has changed to recognized (S1000) because the recognition of the reserved word has been completed.

When the user 331 utters a word including the content for controlling the device or the sensor, the host device 332 captures the voice data (control content) into the input management unit 420 (S1002) through the microphone 421 (S1001). Since the internal state has been recognized, the input management unit 420 transfers the input voice data (control content) to the voice processing unit 407 (S1002). The voice processing unit 407 sends the transferred voice data (control content) to the voice text conversion unit 101-2 in the voice recognition cloud 101 in the cloud server 1 via the Internet 2.

The voice-text conversion unit 101-2 performs a process of converting voice data sent through the Internet 2 into text data (S1004). By this process, the voice originally emitted by the user 331 captured through the microphone 421 is converted into text data.

When the conversion to the text data is completed, the voice text conversion unit 101-2 saves the converted text data internally and sends a conversion completion notification to the voice processing unit 407 (S1005).

Upon receiving the conversion completion notification, the voice processing unit 407 transmits a text analysis request to the voice text conversion unit 101-2 (S1006). When the voice-text conversion unit 101-2 receives the text analysis request, the voice-text conversion unit 101-2 sends the text analysis request to the text analysis unit 102-1 together with the data converted into the text stored inside (S1007). When the text analysis unit 102-1 receives the text analysis request (S1007), it analyzes the content of the accompanying text data (S1008). When the analysis of the contents of the sent text data is completed, the text analysis unit 102-1 sends the analysis result to the response / action generation unit 102-2 as a text analysis result notification (S1009). When the response / action generation unit 102-2 receives the text analysis result (S1009), it generates a target device and a command to control the device based on the content (S1010), and responds / actions with the generated command. It is sent to the voice processing unit 407 as a generation result notification (S1011).

When the voice processing unit 407 receives the response / action generation result notification (S1011), the voice processing unit 407 identifies the device or sensor to be controlled and its control content from the content of the response / action generation result notification (S1012). The voice processing unit 407 converts the specified device or sensor to be controlled and its control content into a format that can be recognized by the device or sensor to be controlled, and notifies the target device or sensor of the action at the required timing through the network 333. Is transmitted as (S1013).

When the device or sensor to be controlled, which is the notification destination of the action notification, receives the action notification (S1013), it operates based on the control content included in the action notification (S1014).

When the user 331 continuously emits voice, the host device 332 determines the continuous voice as a series of voices, and does not require the user 331 to emit a reserved word in the middle of the series of voices. Can be imported. On the contrary, the host device 332 requests the input of the reserved word again when the user 331 emits a voice after a certain period of time. Each case will be described with reference to FIGS. 11A and 11B and FIGS. 12A and 12B.

11A and 11B are examples of processing sequences in which the user 331 continuously utters words within the time T0 after the recognition of the reserved word is completed. When the host device 332 captures the voice data (control content) input from the microphone 421 into the input management unit 420 (S1101), the input management unit 420 activates the input interval confirmation timer T. When the next voice data (control content) emitted by the user 331 through the microphone 421 is taken into the input management unit 420 at the time T1 before the time (= T0) when the input interval confirmation timer T expires (S1121), the input management The unit 420 transfers the captured voice data (control content) to the voice processing unit 407 (S1122). At the same time, the input interval confirmation timer T being activated is restarted. The voice processing unit 407 sends the transferred voice data (control content) to the voice text conversion unit 101-2 in the voice recognition cloud 101 in the cloud server 1 via the Internet 2 (S1123). After that, the processing of the voice data (S1123) sent in the voice recognition cloud 101 is continued in the same manner as the processing of S1104 to S1110.

The input interval confirmation timer T is activated at the timing when the input management unit 420 captures the voice data input from the microphone 421. Not limited to this, for example, the input management unit 420 is sent from the microphone 421. It may be activated at the timing of transferring the data to the trigger setting unit 403 or the voice processing unit 407. Further, it may be activated at the timing when the internal state of the input management unit 420 transitions to recognized (S1100).

12A and 12B are examples in which the user 331 does not continuously emit voice within the time T0. When the host device 332 captures the voice data (control content) input from the microphone 421 into the input management unit 420 (S1201), the input management unit 420 activates the input interval confirmation timer T. When the time (= T0) at which the input interval confirmation timer T expires has passed, the input management unit 420 shifts the internal state to the input wait (S1220).

When the host device 332 captures the next audio data input from the microphone 421 after the time (= T0) at which the input interval confirmation timer T expires (S1224), the host device 332 captures the device based on the captured audio data. The user 331 is urged to issue a reserved word without executing the process of controlling the user or the sensor.

When the input interval confirmation timer T expires, the input management unit shifts the internal state to input waiting (S1220) and notifies the voice processing unit 407 of the timeout notification (S1221). The voice processing unit 407 that received the time-out notification sends a recognition incomplete notification to the display unit 425 (S1222), and the display unit 425 that receives the recognition incomplete notification sends the reserved word to the user 331. The prompting display, for example, the LED blinks in red (S1223).

When the next voice data input from the microphone 421 is fetched after the input interval confirmation timer T expires (S1224), the input management unit 420 makes a transition during recognition of the internal state (S1225) and the fetched voice. The data is transferred to the trigger recognition unit 405 (S1226). After that, the host device 332 performs the processes of S803 to S812 of FIGS. 8A and 8B or the processes of S903 to S916 of FIGS. 9A and 9B, and recognizes the reserved words again.

Next, the registration of the control content for controlling the device or sensor using the host device 332 and the control of the device or sensor based on the registered control content will be described.

FIG. 13 shows control information used when the host device 332 recognizes the reserved word and then controls various sensors 310, various equipment 320, and various home electric appliances 340 as shown in the sequence diagrams of FIGS. 10A and 10B. It shows a concrete example of the contents.

Item 1 is information that controls various sensors 310, various equipment 320, and various home appliances 340, which is included in the response / action generation result notification transmitted from the response / action generation unit 102-2 (hereinafter, response / action). This is a specific example of (called information). This response / action generation information includes "objects" such as devices and sensors controlled by the device 332, and "instructions" representing the contents of controlling the controlled objects. When the host device 332 receives the response / action generation result notification, it extracts the action information included in the response / action generation result notification, and controls the target device based on the content of the action information.

Examples of "commands" are "start command" to start (operate) the device to be controlled, "stop command" to end (stop), and "operation change command" to change the content (mode) during operation. , "Setting change command" to change the contents (mode) preset in the target device.

In order for the response / action generation unit 102-2 to generate the response / action information included in the response / action generation result notification, the user 331 preliminarily controls the device to be controlled, its control content, and the host device to control the device. It is necessary to register the combination of the words uttered to the 332 in the response / action generation unit 102-2 as the initial setting of the host device 332. Hereinafter, registration of response / action information in the initial setting of the host device 332 will be described with reference to the example of FIG.

Item 2 is a “target” which is a device controlled through the host device 332. This "target" is an identification name of a device or sensor included in various sensors 310, various equipment devices 320, and various home electric appliances 340, and the air conditioner 1 is described as a specific example.

Item 3 is an "instruction" which is a control content of the device shown in "item 2". This "command" describes the command of the air conditioner 1 listed in item 2 as a specific example, and includes a "start command" for operating the air conditioner, a "stop command" for stopping the air conditioner, and a "operation change" for changing the operation content of the air conditioner. "Command" and "Setting change command" to change the setting contents of the air conditioner are described as examples.

The product specifications of the devices and sensors of items 2 and 3 are stored in advance in the product specification cloud server in which the information of the product specifications not described is stored. The user 331 obtains information on the product specifications of items 2 and 3 of the target device and the target sensor to be controlled through the host device 332 from the product specification cloud server.

Next, the user 331 determines item 4 = "phrase", which is a word to be emitted to the host device 332 when executing the control contents of the items 2 and 3 through the host device 332. It is desirable that this "phrase" corresponds to the command of the air conditioner 1 mentioned in item 3. For example, for the "start command" to operate the air conditioner, "turn on the air conditioner" and to stop the air conditioner "stop command". For "Keep the air conditioner", change "Cooling", which is the operation content of the air conditioner, to "Dry", and "Dry" for the "Operation change command", which is the setting content of the air conditioner. For the "setting change command" that changes to "start operation at 10 pm", "turn on the air conditioner at 10 pm" is described as an example.

The user 331 creates a combination of (objects, instructions, phrases) determined from the above as the initial setting of the host device 332. The user 331 performs the same creation for all the devices to be controlled through the host device 332, and finally generates a response / action information list that summarizes (targets, commands, phrases) related to all the devices to be controlled. To do. The created response / action information list is registered in the response / action generation unit 102-2 through the host device 332.

When the response / action information list is registered in the response / action generation unit 102-2, as shown in FIGS. 10A and 10B, the host device 332 continues to issue words from the user 331 after the recognition of the reserved word is completed. Devices and sensors can be controlled by capturing and analyzing.

For example, in the case where the word uttered by the user 331 = turn on the air conditioner, the voice text conversion unit 101-2 converts the input voice data into the text "Eakon tsuke", and the text analysis unit 102-1 , Analyze the text data "Eakon Tsukete" as "Turn on the air conditioner". Based on this analysis result, the response / action generation unit 102-2 refers to the already registered response / action information list, and the response / action information corresponding to the analysis result of the "phrase" of "turn on the air conditioner". To search for. As a result, the response / action information (target = air conditioner 1, command = start of operation) is extracted, and the response / action information of (target = air conditioner 1, command = start of operation) is set in the response / action generation result notification. Notifies the voice processing unit 407.

The voice processing unit 407 refers to the response / action information set in the received response / action generation result notification, and selects the corresponding device or sensor in the various sensors 310, various equipment devices 320, and various home appliances 340. Control.

Next, when the device or sensor is controlled by using the host device 332, the control content for controlling the device or sensor according to various conditions and the case where the operation content of the host device 332 is changed will be described.

FIG. 14 shows an operation in which, when a plurality of reserved words are registered in the host device 332, the host device 332 recognizes the word uttered by the user 331 as one of the reserved words, and performs the operation according to the recognized reserved word. A list of example content.

The host device 332 can register a plurality of reserved words, and when each of the plurality of reserved words is recognized, the operation content according to the recognized reserved words (hereinafter referred to as additional information 1). Can be set.

As shown in FIG. 14, it is assumed that the host device 332 has registered three reserved words, for example, "Iroha", "Ore-sama", and "Sonya". When the host device 332 recognizes the word uttered by the user 331 as the reserved word "Iroha", the operation content already set is not changed, but the word uttered by the user 331 is regarded as the reserved word "Ore-sama". If it is recognized, the operation is changed so that when the word uttered by the user 331 is recognized, the operation is always announced through the speaker 423 as "master pleased". Further, when the word uttered by the user 331 is recognized as the reserved word "son", the host device 332 determines that the user 331 is a senior user, and the senior user tends to talk slowly. The setting is changed so that the expiration time T0 of the input interval confirmation timer shown in FIGS. 11A and 11B is longer than the normal set time.

The example of FIG. 14 shows an example in which the host device 332 changes the operation content of the host device itself, but is not limited to this, and controls the operation of the device and the sensor connected to the host device 332 by the network 333. You may.

In order to change the operation of the host device 332 according to a plurality of reserved words, the host device 332 needs to register additional information 1 for each reserved word in the host device 332 in advance.

The host device 332 has a mode (hereinafter referred to as a reserved word registration (additional information 1) mode) in which when the reserved word is registered in the host device 332, the additional information 1 corresponding to the reserved word to be registered is also registered. ing.

15A and 15B show the reserved words in the state where the host device 332 is in the "reserved word registration (additional information 1) mode" in order to register the reserved words and the corresponding additional information 1 together. An example of the processing sequence of the host device 332 from the start of registration to the completion of registration of the additional information 1 is shown. The processes of S1500 to S1514 shown in FIGS. 15A and 15B are the same as the processes of S500 to S514 shown in FIGS. 5A and 5B, respectively. The difference in processing between FIGS. 5A and 5B in FIGS. 15A and 15B is that S1515 is different from S515 and S1516 to S1523 are added.

The trigger setting unit 403 displays (S1514) notifying the user 331 that the registration of the reserved word is completed. The display (S1515) notifying the user that the registration of the reserved word is completed is a display device in which the trigger setting unit 403 sends a registration completion notification to the display device 425 (S1514) and receives the registration completion notification. It is desirable that the display method be recognizable by the user 331, for example, the 425 blinks the LED in green. As a result, the trigger setting unit 403 can urge the user 331 to register the additional information 1.

The user 331 who recognizes that the LED is blinking green (S1515) can set the additional information 1 corresponding to the reserved word whose registration is completed in S1511.

The method of setting the additional information 1 may be such that the voice emitted by the user 331 is captured by the host device 332 through the microphone 421 and can be registered by analyzing the captured voice data. Alternatively, the display device 425 may display a menu for setting the additional information 1, and the user 331 may be able to register by operating according to the menu. Alternatively, using an external device connected via the network I / F427 shown in FIG. 4, for example, a smartphone or tablet, a menu for setting additional information 1 corresponding to the reserved word on the display screen of the smartphone or tablet is displayed. It may be displayed so that the user 331 can register by operating according to the displayed menu screen. 15A and 15B are examples of a processing sequence in which the menu for setting the additional information 1 displayed on the display unit 425 is displayed and the user 331 registers the additional information 1 by operating according to the menu.

When the LED blinks green (S1515) to prompt the user 331 to input the additional information 1, a menu for registering the additional information 1 is displayed on the display unit 425. The user 331 creates the additional information 1 by operating according to the displayed menu screen. The created additional information 1 is taken into the input management unit 420 (S1517). The input management unit 420 transfers the captured additional information 1 to the trigger setting unit 403. The trigger setting unit 403 stores the transferred additional information 1 in the reserved word storage area 410-2 of the memory 410 (S1519).

When the additional information 1 is saved in the reserved word storage area 410-2 of the memory 410, the trigger setting unit 403 saves the additional information 1 in association with the reserved word registered in S1513.

In addition, the voice processing unit 407 displays (S1522) notifying the user 331 that the registration of the additional information 1 is completed. As for the display (S1522) notifying the user 331 that the registration of the additional information 1 is completed, the voice processing unit 407 sends a registration completion notification to the display device 425 (S1520) and receives the registration completion notification. It is desirable that the display device 425 use a display method that can be recognized by the user, such as turning on the LED in green.

16A and 16B show the recognition of reserved words from the words uttered by the user 331 when the additional information 1 is saved in the reserved word storage area 410-2 of the memory 410 by the processing shown in FIGS. 15A and 15B. Then, this is an example of a sequence in which the recognized additional information 1 of the reserved word is read from the reserved word storage area 410-2 and the operation is set for the host device 332.

The processes of S1600 to S1612 shown in FIGS. 16A and 16B are the same as the processes of S800 to S812 shown in FIGS. 8A and 8B, respectively. The difference between the processes of FIGS. 16A and 16B and the processes of FIGS. 8A and 8B is that the processes of S1613 and S1614 are added.

When the word uttered by the user 331 is recognized as a reserved word (S1605), the trigger recognition unit 405 reads the additional information 1 corresponding to the reserved word from the reserved word storage area 410-2 of the memory 410. The trigger recognition unit 405 that has read the additional information 1 sets the operation of the contents of the read additional information 1 (S1613) in the host device 332 (S1614). When the content of the example shown in FIG. 14 is stored in the reserved word storage area 410-2, and when "son" is recognized as a reserved word in S1605, the trigger recognition unit 405 has an input interval in S1614. The expiration time T0 of the confirmation timer T is set to be longer than the normal value.

In FIG. 17A, when a word uttered by the user 331 is recognized as a reserved word registered in the host device 332, the host device responds to the word uttered by the user 331 following the recognized reserved word. 332 is a list of examples of operation contents that perform a specific operation.

When the host device 332 recognizes that the word uttered by the user 331 is a registered reserved word, the content of the word uttered by the user 331 (hereinafter referred to as an additional word) following the recognized reserved word. The operation content (hereinafter referred to as additional information 2) can be set according to the above.

For example, as shown in FIG. 17 (A), it is assumed that "Iroha" is registered as a reserved word. In this case, when the host device 332 recognizes the reserved word "Iroha" and does not recognize the word uttered by the user 331 following the reserved word "Iroha", the host device 332 does not change the already set operation content. When the host device 332 recognizes "chan" as a word uttered by the user 331 following the reserved word "Iroha", it determines that the user 331 is in a good mood, and when responding through the speaker 423, when responding. Change the operation content to raise the tone. Further, when the host device 332 recognizes "ya" as a word uttered by the user 331 following the reserved word "Iroha", the host device 332 presumes that the user 331 is a senior user, and the user 331 tends to speak slowly. Therefore, the expiration time T0 of the input interval confirmation timer shown in FIGS. 11A and 11B is changed to be longer than the normal set time. When the host device 332 recognizes "hey" as a word uttered by the user 331 following the reserved word "Iroha", it determines that the user 331 is angry and immediately says "sorry" through the speaker 423. Make an announcement.

In the example of FIG. 17A, the host device 332 sets the additional information 2 by setting a plurality of additional words for one reserved word and setting the additional information 2 for each combination of the plurality of additional words for the reserved word. Although an example of changing the operation content based on the content of is shown, it is also possible to set additional information 2 for each combination of a plurality of reserved words and a plurality of additional words. As shown in FIG. 17B, for example, it is assumed that the host device 332 has registered three reserved words, "Iroha", "Ookini", and "Ashindo". In this case, additional words may be defined for each reserved word, and additional information 2 may be set for each combination of the reserved word + the additional word.

Also, depending on the user, there are times when you want a specific action to be performed simply by issuing a reserved word. For example, if an individual has a habit, the habit is registered in the host device 332 as a reserved word, and the operation corresponding to this reserved word is registered in the host device 332, so that the device is suitable for the individual's characteristics. And the operation of the sensor can be easily controlled. In the example of the reserved word "Ashindo" in FIG. 17B, when the host device 332 recognizes the reserved word "Ashindo", the host device 332 recognizes the reserved word from the words uttered by the user 331. It is also possible to announce the information of beer stored in the refrigerator connected to the network 333 through the speaker 423.

In order to change the operation according to the content of the additional word for the reserved word, the host device 332 combines the additional word corresponding to the reserved word and the additional information 2 which is the operation content for the combination of the reserved word and the additional word. It is necessary to register in the host device 332 in advance. Therefore, the host device 332 has a mode for additionally registering the corresponding additional words and additional information with respect to the registered reserved words. The mode for adding additional information 1 to the reserved words already registered in the host device 332 is called the additional information 1 additional registration mode, and the mode for adding the additional words and the additional information 2 is called the additional information 2 additional registration mode. And.

As for the setting method of the additional information 2, the host device 332 may capture the voice emitted by the user 331 through the microphone 421 and analyze the captured voice data so that the additional information 2 can be registered. Alternatively, the display device 425 may display a menu for setting the additional information 2, and the user 331 may be able to register by operating according to the displayed menu. Alternatively, using an external device connected via the network I / F427 shown in FIG. 4, for example, a smartphone or tablet, additional information 2 corresponding to the reserved word and the additional word is set on the display screen of the smartphone or tablet. The user 331 may be able to register by displaying the menu to be displayed and operating according to the displayed menu screen.

18A, 18B and 18C show cases where additional words are registered and operation contents (additional information 2) for the additional words are registered for the registered reserved words shown in FIGS. 17A and 17B. This is an example of the processing sequence of.

In order to additionally register additional words for the registered reserved words, the user 331 changes the host device 332 to the "additional information 2 additional registration mode". When the host device is changed to the "additional information 2 additional registration mode", the user 331 emits a reserved word registered in the host device 332 and an additional word to be registered for the reserved word. The host device 332 first recognizes the reserved word from the words uttered by the user 331 (S1805).

The host device 332 captures the words uttered by the user 331 into the input management unit 420 through the microphone 421 (S1801). When the input management unit 420 captures the voice data, the internal state managed internally is changed to being recognized (reserved word) (S1802), and the input voice data is transferred to the trigger recognition unit 405 (S1803).

When the trigger recognition unit 405 receives the voice data transferred from the input management unit 420, the trigger recognition unit 405 reads the recognition data from the reserved word storage area 410-2 of the memory 410 (S1804) and transfers the data from the input management unit 420. Comparison with audio data is performed (S1805). When the input voice data can be recognized as a reserved word, the trigger recognition unit 405 notifies the input management unit 420 of the recognition completion notification (S1806). Upon receiving the recognition completion notification, the input management unit 420 shifts (S1807) the internal state managed internally from being recognized (reserved word) to waiting for input (additional word).

The host device 332 captures the words uttered by the user 331 following the reserved words into the input management unit 420 through the microphone 421 (S1808). Since the internal state managed internally is waiting for input (additional word) (S1807), the input management unit 420 transfers the input voice data to the trigger setting unit 403 (S1809). After that, similarly to the registration of the reserved words described with reference to FIGS. 5A and 5B, the trigger setting unit 403 saves the received voice data in the voice storage area 410-3 of the memory 410 (S1810), and adds a specified number of additional words. Is taken in (S1811).

When the trigger setting unit 403 determines that the specified number of times has not been reached as a result of checking whether the specified number of times has been reached, the trigger setting unit 403 displays to prompt the user 331 to input the voice of the additional word to be registered (S1812), and also performs input management. An input continuation notification is transmitted to unit 420 (S1814). In the display (S1813) prompting the user 331 to input the voice to be registered as an additional word, the trigger setting unit 403 sends a registration incomplete notification to the display device 425 (S1812), and the registration incomplete notification is sent. It is desirable that the display device 425 that has received the message blinks the LED in red, for example, by a display method that can be recognized by the user 331. Further, the user 331 may be prompted to input the voice to be registered by using the voice method instead of the display method. In this case, the trigger setting unit 403 sends a registration incomplete notification to the speaker 423, and the speaker 423 receiving the registration incomplete notification announces, for example, "Please enter again" to the user 331. But it may be. Alternatively, the trigger setting unit 403 may use both the display method and the voice method for prompting the user 331 to input the voice to be registered.

When the trigger setting unit 403 determines that the specified number of times has been reached as a result of checking whether the specified number of times has been reached, the trigger setting unit 403 reads out the voice data stored in the voice storage area 410-3 by that time (S1815) and uses the Internet. It is sent to the recognition data conversion unit 101-1 in the voice recognition cloud 101 in the cloud server 1 through 2 (S1816).

The recognition data conversion unit 101-1 converts the voice data sent from the trigger setting unit 403 into recognition data for recognizing additional words (S1817). When the conversion to the recognition data is completed, the recognition data conversion unit 101-1 sends the recognition data to the trigger setting unit 403 (S1818) via the Internet 2. The trigger setting unit 403 that has received the recognition data for recognizing the additional word (hereinafter referred to as the recognition data (additional word)) saves the received data in the reserved word storage area 410-2 of the memory 410 (S1819). ). When saving the recognition data (additional word), the trigger setting unit 403 saves it in association with the reserved word recognized in S1806. As a result, the recognition data (additional word) can be saved in association with the reserved word recognized in S1806.

In addition, the trigger setting unit 403 displays (S1822) notifying the user 331 that the registration of the additional word is completed. The display (S1822) notifying the user 331 that the registration of the reserved word is completed is a display in which the trigger setting unit 403 sends a registration completion notification to the display device 425 (S1821) and receives the registration completion notification. It is desirable that the device 425 blinks the LED in green (S1822) in a display method that can be recognized by the user 331. Alternatively, the trigger setting unit 403 may use a voice method instead of the display method to notify the user 331 that the registration of the reserved word is completed. In this case, the trigger setting unit 403 transmits a registration completion notification to the speaker 423 (S1821), and the speaker 423 receiving the registration completion notification announces, for example, "registration is completed" to the user 331. But it may be. Alternatively, the trigger setting unit 403 may use both a display method and a voice method for notifying the user 331 that the registration of the reserved word is completed. As a result, the user 331 can know the timing at which the content of the additional information 2 which is the operation content corresponding to the additional word is uttered in words.

When the LED blinks green to prompt the user 331 to input the additional information 2 (S1822), a menu for registering the additional information 2 is displayed on the display unit 425. The user 331 creates the additional information 2 by operating according to the displayed menu screen. The creation is completed and the additional information 2 is taken into the input management unit 420 (S1824). The input management unit 420 transfers the captured additional information 2 to the trigger setting unit 403 (S1825). The trigger setting unit 403 stores the transferred additional information 2 in the reserved word storage area 410-2 of the memory 410 (S1826).

When the additional information 2 is saved in the reserved word storage area 410-2 of the memory 410, the trigger setting unit 403 saves the additional information 2 in association with the reserved word recognized in S1806. As a result, it is possible to save the operation content (additional information 2) associated with the reserved word recognized in S1806 and associated with the additional word saved in S1819.

It is also possible to add only additional information to the registered reserved words later.

18D and 18E are examples of processing sequences in which only additional information is added to the registered reserved words, unlike FIGS. 18A, 18B and 18C.

The processes of S1850 to S1856 shown in FIG. 18D are the same as the processes of S1800 to S1806 shown in FIG. 18A, respectively. Further, the processes of S1871 to S1880 shown in FIGS. 18D and 18E are the same as the processes of S1821 to S1830 shown in FIG. 18C, respectively. The difference between the sequence example of FIGS. 18A, 18B and 18C and the sequence example of FIGS. 18D and 18E is that the process corresponding to the additional word registration process of S1807 to S1820 of FIGS. 18A, 18B and 18C is shown in FIG. This is a point not shown in 18D and FIG. 18E.

When the LED blinks green (S1871) to prompt the user 331 to input the additional information 1, a menu for registering the additional information 1 is displayed on the display unit 425. The user 331 creates the additional information 1 by operating according to the displayed menu screen. The creation is completed and the additional information 1 is taken into the input management unit 420 (S1874). The input management unit 420 transfers the captured additional information 1 to the trigger setting unit 403 (S1875). The trigger setting unit 403 stores the transferred additional information 1 in the reserved word storage area 410-2 of the memory 410 (S1876).

When the additional information 1 is saved in the reserved word storage area 410-2 of the memory 410, the trigger setting unit 403 saves the additional information 1 in association with the reserved word recognized in S1856. This makes it possible to save the operation content associated with the reserved word recognized in S1856.

19A and 19B show the words uttered by the user 331 when the additional words and the additional information 2 are stored in the reserved word storage area 410-2 of the memory 410 by the processes shown in FIGS. 18A, 18B and 18C. When the reserved word and the additional word are recognized from the inside, the additional information 2 corresponding to the combination of the recognized reserved word and the additional word is read from the reserved word storage area 410-2, and the operation is set for the host device 332. This is an example of the sequence of.

The processes of S1900 to S1908 shown in FIG. 19A are the same as the processes of S1600 to S1608 shown in FIG. 16A, respectively. The difference between the processing in FIGS. 19A and 19B and the processing in FIGS. 16A and 16B is that the processing for recognizing the additional words in S1909 to S1911 is added and the processing for reading the additional information 2 in S1912 to S1913. Is the point to do.

When the reserved word is successfully recognized in S1905 of FIG. 19A for the data in which the word uttered by the user 311 is captured, the trigger recognition unit 420 succeeds in recognizing the data in which the word uttered by the user 311 is captured. Comparison with the recognition data (additional word) read from the reserved word storage area 410-2 of the memory 410 in order to determine whether the voice data continuously input to the reserved word is an additional word. (S1911). When the trigger recognition unit 405 recognizes that the voice data following the reserved word is an additional word, the trigger recognition unit 405 reads the reserved word and the additional information 2 corresponding to the additional word from the reserved word storage area 410-2 of the memory 410 ( S1912). The trigger recognition unit 405 that has read the additional information 2 sets the operation of the contents of the read additional information 2 in the host device 332 (S1913).

By registering the reserved words, additional words, and additional information in the host device 332 as described above, the host device 332 operates the host device 332 and operates with respect to the devices and sensors connected to the host device 332 via the network. Can be freely controlled, and devices and sensors that suit each individual's lifestyle can be controlled.

FIG. 20 shows a voice recognition cloud according to the recognized reserved words when a plurality of reserved words are registered in the host device 332 and the user recognizes that the reserved words are one of the words uttered by the user 331. This is a list of examples of changing the voice recognition dictionary used in the voice text conversion unit 101-2 of 101.

The host device 332 can register a plurality of reserved words. When the host device 332 recognizes that the word uttered by the user 331 is one of a plurality of registered reserved words, the voice text conversion unit 101-2 of the voice recognition cloud 101 according to the recognized reserved word. The voice recognition dictionary for converting the voice used in is converted to text can be changed. For example, as shown in FIGS. 21A and 21B, the host apparatus 332 is assumed to have registered three "Hello", "Hello", "Ookini" as reserved words. In this case the host device 332, if you recognize the "Hello" reserved word, it is possible to issue an instruction to change the voice recognition dictionary used in speech-to-text conversion unit 101-2 of the speech recognition cloud 101 to Japanese dictionary .. When the reserved word "Hello" is recognized, the host device 332 issues a command to the voice text conversion unit 101-2 of the voice recognition cloud 101 to change the type of the voice recognition dictionary to the English dictionary. be able to. Furthermore, when the reserved word "Ookini" is recognized, the host device 332 changes the type of the voice recognition dictionary used in the voice text conversion unit 101-2 of the voice recognition cloud 101 to the dialect dictionary (Kansai dialect). You can issue orders.

In order to change the type of the voice recognition dictionary used in the voice text conversion unit 101-2 of the voice recognition cloud 101 according to the reserved words recognized by the host device 332, the user 331 registers the reserved words in the host device 332. At that time, it is necessary to register the type of the voice recognition dictionary (hereinafter referred to as additional information 3) used by the voice text conversion unit 101-2 according to the reserved word.

The processing sequence for registering the type of voice recognition dictionary (additional information 3) corresponding to the reserved word together with the registration of the reserved word is the processing sequence for registering the additional information 1 for the reserved word shown in FIGS. 15A and 15B. Instead of inputting the additional information 1 on the menu screen displayed on the display unit 425 (S1516), the input screen of the additional information 3 may be selected and input. Hereinafter, the flow of the process of registering the additional information 3 will be described using the processes after S1514 of FIG. 15B. The additional information 1 described after S1514 in FIG. 15B will be described by being read as the additional information 3.

When the LED blinks green (S1514) to prompt the user 331 to input the additional information 3, a menu for registering the additional information 3 is displayed on the display unit 425. The user 331 can select the type of the dictionary as the additional information 3 by inputting the additional information 3 according to the displayed menu screen. The creation is completed and the additional information 3 is taken into the input management unit 420 (S1516). The input management unit 420 transfers the captured additional information 3 to the trigger setting unit 403. The trigger setting unit 403 stores the transferred additional information 3 in the reserved word storage area 410-2 of the memory 410.

When the additional information 3 is saved in the reserved word storage area 410-2 of the memory 410, the trigger setting unit 403 saves the additional information 3 in association with the reserved word registered in S1513.

21A and 21B show the voice text conversion unit 101-2 when a plurality of reserved words are registered in the host device 332 as shown in FIG. 20 and each reserved word is recognized by the host device 332. An example of a sequence for changing the type of speech recognition dictionary to be used is shown. The processes of S2100 to S2113 shown in FIGS. 21A and 21B are the same as the processes of S1600 to S1613 shown in FIGS. 16A and 16B, respectively. The difference between the processing in FIGS. 21A and 21B and the processing in FIGS. 16A and 16B is that in the case of the processing of FIGS. 16A and 16B, after the trigger recognition unit 403 reads the additional information 1, the additional information 1 While the operation of the host device 332 is set based on the content (S1614), in the case of FIGS. 21A and 21B, after the trigger recognition unit 403 reads the additional information 3, the content of the additional information 3 is changed. Based on this, in order to change the type of the voice recognition dictionary used by the voice text conversion unit 101-2, communication with the voice text conversion unit 101-2 is performed (S2114-1 to S2114-3).

As for the display notifying the user that the recognition of the reserved word and the change of the voice recognition dictionary are completed, the trigger setting unit 403 sends a registration completion notification to the display device 425 (S2109), and the registration completion notification is sent. It is desirable that the display device 425 that has received the above is a display method that can be recognized by the user 331, for example, the LED is turned on in green. Alternatively, the trigger recognition unit 405 sends a recognition completion notification to the speaker 423, and the speaker 423 receiving the recognition completion notification is, for example, "Yes, yes? By the way, the voice recognition dictionary is a dialect dictionary (Kansai dialect). I changed it to a valve). ”You can also use a voice announcement method to the user 331. Alternatively, the trigger recognition unit 405 notifies the user 331 that the recognition of the reserved word and the change of the voice recognition dictionary corresponding to the recognized reserved word are completed, by displaying the method using the display device 425 and the speaker. Both voice methods using 423 may be used.

The operation content (additional information 1) corresponding to the reserved word shown in FIG. 14, the operation content for each additional word for the reserved word shown in FIGS. 17 (A) and 17 (B) (additional information 2), and the reservation shown in FIG. 20. The types of speech recognition dictionaries for words (additional information 3) can be registered in combination.

FIG. 22 shows registration of operation contents corresponding to the reserved words shown in FIG. 14, registration of additional words for the reserved words shown in FIG. 17A, registration of operation contents for the additional words, and voice recognition for the reserved words shown in FIG. This is a list of combinations when registering dictionary types in combination. The host device 332 is, for example, a reserved word for the "Hello" is configured to use the Japanese dictionary as the type of voice recognition dictionary. The host device 332, also "Chan", "Ya" as an additional language to the reserved word "Hello" to register the "Hey", the host device so as to raise the tone at the time of response in the case of the additional word "chan" Change the operation content of 332, change the setting content to lengthen the expiration time T0 of the input interval confirmation timer T when the additional word is "ya", and when the additional word is "hey", "sorry" The operation content is set so that it will be announced immediately.

FIG. 23 is a list of examples in which the type of the speech recognition dictionary used by the text conversion unit 101-2 is changed according to the contents other than the reserved words (hereinafter referred to as change conditions). For example, FIG. 23A is an example in which the time is set as a change condition. The host device 332 instructs the text conversion unit 101-2 of the voice recognition cloud 101 to change the type of the voice recognition dictionary used when converting the voice data into text according to the time when the voice recognition dictionary is used. An example is shown.

The host device 332 uses, for example, a family general dictionary from 05:00 to 08:00, a wife's dictionary from 08:00 to 16:00, and 16:00 to 20:00. Instruct the text converter 101-2 via the Internet 2 to use the family dictionary until 20:00 and to use the adult dictionary from 20:00 to 05:00.

Further, FIG. 23B is an example in the case where the change condition = the operation status of the host device 332. The host device 332 can instruct to change the type of the voice recognition dictionary used by the text conversion unit 101-2 according to the type of the operation status of the host device 332 when using the voice recognition dictionary.

The host device 332 uses, for example, the time / route search dictionary when the operation status = going to work, the general dictionary when the operation status = going out, and the refresh dictionary when the operation status = night mode. The text conversion unit 101-2 is instructed to do so through the Internet 2.

The host device 332 has a mode (hereinafter referred to as a change condition registration mode) for registering the change condition type information, which is information on the type of the voice recognition dictionary used according to the conditions.

The user 331 needs to register the change condition type information in the host device 332 in advance in order to properly use the type of the voice recognition dictionary according to the change condition.

The registration method for properly using the type of voice recognition dictionary according to the change condition is to allow the host device 332 to capture the voice emitted by the user 331 through the microphone 421 and analyze the captured voice data so that the voice can be registered. May be good. Alternatively, the display device 425 may display a menu for setting the additional information 1, and the user 331 may be able to register by operating according to the menu. Alternatively, using an external device connected via the network I / F427 shown in FIG. 4, for example, a smartphone or tablet, a menu for setting additional information 1 as a reserved word is displayed on the display screen of the smartphone or tablet. , The user 331 may be able to register by operating according to the displayed menu screen.

FIG. 24 is a process in which a menu for setting the change condition type information displayed on the display unit 425 is displayed, and the user 331 registers the type of the voice recognition dictionary to be used properly according to the change condition by operating according to the menu. This is an example of a sequence. The processing of S2417 to S2423 shown in FIG. 24 is the same as the processing of S1517 to S1523 of FIG. 15B, which is the registration sequence of the additional information 1.

The user 331 inputs the type of voice recognition dictionary to be used properly according to the change condition by operating according to the displayed menu screen. The change condition type information for which the input has been completed is taken into the input management unit 420 (S2417). The input management unit 420 transfers the captured change condition type information to the trigger setting unit 403 (S2418). The trigger setting unit 403 stores the transferred change condition type information in the reserved word storage area 410-2 of the memory 410 (S2419).

FIG. 25 shows that when the change condition type information for changing the type of the speech recognition dictionary according to the change condition is saved in the reserved word storage area 410-2 of the memory 410 as shown in FIG. 23, the change condition type information is saved. This is an example of a processing sequence in which the host device 332 notifies the voice text conversion unit 101-2 of the change of the voice recognition dictionary according to the content of the change condition type information.

It is desirable that the process of FIG. 25 is continuously performed, for example, after the process of recognizing the reserved word shown in FIG. 9B is completed (S911). Alternatively, after the reserved word is recognized, as shown in FIGS. 10A and 10B, when the user 331 issues the word to the host device 332 in order to control the device or sensor, the host device 332 captures the word. However, it is desirable to perform it at the timing (S1001).

FIG. 25 shows the timing (S1001) when the user 331 speaks a word to the host device 332 in order to control the device or the sensor as shown in FIGS. 10A and 10B, and the host device 332 captures the word. This is an example of determining a change in the voice recognition dictionary and notifying the result.

When the recognition of the reserved word is completed, the host device 332 continuously captures the voice emitted by the user into the input management unit 420 through the microphone 421 (S2501). The input management unit 420 transmits a read request (change condition type information) to the voice processing unit 407 in order to read the change condition type information at the timing when the voice data is fetched (S2502), and the processing for the fetched voice data is performed. Pause. Upon receiving the read request (change condition type information), the voice processing unit 407 reads the change condition type information including the combination of the change condition and the type of the voice recognition dictionary from the reserved word storage area 410-2 of the memory 410. (S2503). The voice processing unit 407 analyzes the "change condition" of the read change condition type information, and determines whether or not the content conforms to the state of the host device 332 (S2504). If it is determined that the result of the determination is met, the voice processing unit 407 reads out the "type of voice recognition dictionary" corresponding to the "change condition", and the type of the changed voice recognition dictionary is notified by the voice recognition dictionary type notification. Is notified to the voice-to-speech converter 101-2 via the Internet 2 (2505). The voice recognition dictionary type 101-2 that received the voice recognition dictionary type notification refers to the type of the notified voice recognition dictionary, and changes the type of the voice recognition dictionary currently in use to the type of the notified voice recognition dictionary. (S2506)
When the change of the type of the voice recognition dictionary is completed, the voice text conversion unit 101-2 notifies the voice processing unit 407 of the voice recognition dictionary change completion notification as a notification of the change completion (S2507).

When the voice processing unit 407 receives the voice recognition dictionary change completion notification (S2507), the voice processing unit 407 transmits a reading completion notification to the input management unit 420 as a notification that the reading of the change condition type information is completed (S2508). .. When the input management unit 420 receives the read completion notification (S2508), the input management unit 420 restarts the processing for the voice data captured in S2501.

The user 331 may forget the reserved word registered in the host device 332. In preparation for such a case, it is desirable that the user 331 can confirm the registered reserved words by a simple method.

FIG. 26 shows a part or all of the registered reserved words when the user 331 who registered the reserved words in the example of the processing sequence shown in FIGS. 5A and 5B forgets the registered reserved words. A list of examples of reserved words (hereinafter referred to as relief reserved words) and display contents (display range) for notifying the user is shown. For example, for the reserved word "I don't know", when all the reserved words registered in the host device 332 are displayed on the display unit 425 or displayed in the display area of the external device connected to the host device 332. Is shown. For the reserved word "tell me a little", a predetermined part of the reserved words registered in the host device 332 is displayed on the display unit 425, or an external device connected to the host device 332. Indicates the case of displaying in the display area of. For the reserved word "unused guy", among the reserved words registered in the host device 332, the reserved words that have not been used for the past one year are displayed on the display unit 425, or are connected to the host device 332. Shows the case of displaying in the display area of an external device. As the external device connected to the host device 332, for example, it is desirable that the display screen of a smartphone, tablet, LCD TV, or the like is relatively large so that the user can refer to many reserved words at one time.

In this way, the registration of reserved words for displaying the registered reserved words is changed to the mode of the host device = setting mode (reserved words (for display)), and the reserved words shown in FIGS. 5A and 5B are registered. Registration may be performed according to the registration processing sequence.

In the above example, when the user issues the "relief reserved word" shown in FIG. 26, the corresponding reserved word is immediately displayed. However, the host device 332 may ask the user 331 for the secret word before displaying the corresponding reserved word. After the user issues a "relief reserved word", the host device 332 emits a voice such as "mountain" through the speaker 423, and when the user 331 responds to this, for example, "river", the corresponding reserved word is emitted. It may be displayed.

Further, the host device 332 can also capture the words uttered by the user 331 and record or record a scene in which reserved words, additional words, or additional information are registered. Alternatively, when a reserved word or an additional word is recognized, the recognized scene can be recorded or recorded.

FIG. 27 shows a case where the host device 332 captures a word uttered by the user 331 and records or records a scene of registration of a reserved word, an additional word, or additional information, and recognition of the reserved word or the additional word. The functional block diagram of 332 is shown. The difference from FIG. 4 is that the host device 2700 has a camera 2702 for recording a scene in which a reserved word, an additional word, or additional information is registered, or a scene in which a reserved word or an additional word is recognized. The control management unit 2701 has EVT-Mg2701-3 in addition to APP-Mg2701-1 and CONF-Mg2701-2, and the system controller 402 has a playback control function for playing back recorded or recorded scene data. That is the point. EVT-Mg2701-3 is a function for recording or recording, which will be described later, due to the occurrence of a scene for registering reserved words, additional words, or additional information, and the occurrence of scenes for recognizing reserved words, additional words. have. Hereinafter, a process flow in which the host device 332 captures a word uttered by the user 331 and records or records a scene for registering a reserved word, an additional word, or additional information, and a scene for recognizing the reserved word or the additional word. The recording or the flow of the recording process will be described.

FIG. 28 shows the host device as a registration scene or a recognition scene when a scene for registering a reserved word, an additional word, or additional information occurs, or when a scene for recognizing a reserved word or an additional word occurs. 332 shows the passage of time when recording or recording.

At time t1, it is assumed that the host device 332 starts registration using a word uttered by the user as a reserved word. The start of the reserved word registration may be, for example, the timing at which the input management unit 420 performs the processing of S502 in the reserved word registration sequence of FIGS. 5A and 5B. When the input management unit 420 recognizes the start of registration of the reserved word, it notifies EVT-Mg2701-3 to that effect. Upon receiving the notification that the reserved word registration has started, EVT-Mg2701-3 records the reserved word registration scene as Rec1 through the microphone 421, or records the reserved word registration scene as Rec1 through the camera 2702. The end of the reserved word registration may be, for example, the timing when the input management unit 420 receives the registration completion notification of S512 in the reserved word registration sequence of FIGS. 5A and 5B. The input management unit 420, which has grasped the completion of the registration of the reserved word, notifies EVT-Mg2701-3 to that effect. Upon receiving the notification that the reserved word registration is completed, EVT-Mg2701-3 ends the recording of the reserved word registration scene performed through the microphone 421, or ends the recording of the reserved word registration scene performed through the camera 2702. Let me.

Similarly, at time t2, it is assumed that the host device 332 starts recognizing a word uttered by the user as a reserved word. The start of the reserved word recognition may be, for example, the timing at which the input management unit 420 performs the processing of S802 in the reserved word recognition sequence of FIGS. 8A and 8B. When the input management unit 420 recognizes the start of recognition of the reserved word, it notifies EVT-Mg2701-3 to that effect. Upon receiving the notification that the reserved word recognition has started, EVT-Mg2701-3 records the reserved word recognition scene as Rec2 through the microphone 421, or records the reserved word recognition scene as Rec2 through the camera 2702. The end of the recognition of the reserved word may be, for example, the timing when the input management unit 420 receives the recognition completion notification of S811 in the reserved word registration sequence of FIGS. 8A and 8B. The input management unit 420, which has grasped the completion of the registration of the reserved word, notifies EVT-Mg2701-3 to that effect. Upon receiving the notification that the reserved word registration is completed, EVT-Mg2701-3 ends the recording of the reserved word recognition scene performed through the microphone 421, or ends the recording of the reserved word recognition scene performed through the camera 2702. Let me.

Similarly, record or record registration or recognition events that occur at t3 and t4.

The host device 332 can reproduce the recorded or recorded registered scene or recognition scene.

FIG. 29 shows an example of how the data to be reproduced is displayed when the data of the recorded scene or the recorded scene is reproduced. In the example of FIG. 29, four icons of data to be reproduced are displayed in a form corresponding to the occurrence of an event with respect to the time axis of FIG. 28. The icon display of the data to be reproduced may be displayed on the display unit 425, for example. Alternatively, it may be displayed on an external device connected to the host device 332, for example, a smartphone, a tablet, an LCD TV, or the like.

The displayed icon indicates the date and time when the recording or recording was performed and the content of the data to be recorded or recorded. For example, when the display content of the icon is the reserved word registration "Ookini", it indicates that the content of the recorded or recorded data is a scene in which "Ookini" is registered as a reserved word. Similarly, when the display content of the icon is the reserved word recognition "Ookini", it indicates that the content of the recorded or recorded data is a scene in which "Ookini" is recognized as the reserved word.

The user 331 can confirm the recording or the recorded content of the target data by selecting the icon of the data to be reproduced.

Furthermore, the host device 332 issues an instruction to the cameras and microphones connected by the network 333, and when a scene in which reserved words, additional words, or additional information is registered occurs by these cameras or microphones, or When a scene that recognizes a reserved word or an additional word occurs, the registered scene or the recognition scene may be recorded or recorded.

As described above, the host device 332 recognizes the reserved word from the words uttered by the user 331, and based on the content of the additional information corresponding to the reserved word, the device or sensor connected by the network. Can be controlled. The control content of the target device or sensor may require high security. For example, it is assumed that a reserved word in which the opening / closing operation of the safe door is set is registered as additional information in the host device 332 so that the opening / closing of the safe door can be controlled by using the host device. In this case, when the host device 332 recognizes the corresponding reserved word, it opens and closes the door of the safe and uses a microphone or a camera around the safe to record or record the area around the safe, which is the device to be controlled. By recording, it is possible to maintain the security of the opening and closing operation of the safe door. The user 331 confirms the contents of the data recorded or recorded by using the microphone or camera connected by the network as well as the data recorded or recorded by using the microphone or camera built in the host device 332. Can be done. When the control content of the device or sensor to be controlled by the host device 332 requires high security, the host device 332 also has a microphone or a microphone around the device or sensor to be controlled before executing the control content. The validity of the person who emitted the recorded sound or the person in the recorded video may be confirmed by using the sound recorded by the camera or the recorded video. The host device 332 uses the feature points such as the voice and face of a specific person registered in advance and the microphones and cameras around the device or sensor to be controlled before executing the control content in the specific additional information. The corresponding control content may be executed only when the legitimacy of the relevant person is confirmed by comparing the sound collected by the sound and the captured video.

In the above description of the embodiment, the recognition data conversion unit 101-1, the voice text conversion unit 101-2, the text analysis unit 102-1 and the response / action generation unit 102-2 are all in the cloud server 1. Although it has been described as existing, some or all of them may be present in the host device 332. In that case as well, the example of the operation sequence of each process already described is the same as that described above.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

1 ... Cloud server, 2 ... Internet, 3 ... Home, 101 ... Voice recognition cloud, 102 ... Corresponding action generation cloud, 310 ... Various sensors, 320 ... Various equipment , 330 ... HGW (HomeGate Way), 331 ... user, 332 ... host device, 340 ... various home appliances.

Claims (3)

Based on the content of the first voice input from the outside, the execution of control of one or a plurality of devices is determined based on the content of the second voice input after the first voice is input. In electronic devices
Judgment voice data for determining that the first voice is a desired voice is created and managed by voice input from the outside a plurality of times, and the judgment voice data created and managed is used to describe the above. A management means for determining that the first voice is a desired voice, and
A control means for executing control of the one or a plurality of devices based on the content of the second voice is provided, and the management means uses the determination voice data to make the first voice a desired voice. An electronic device that controls one or more devices based on the content of the second voice by the control means when it is determined to be.
If the management means has an output unit that outputs the determination result by voice, and the management means determines that the first voice is a desired voice by using the judgment voice data, that effect. Is output from the output unit,
The management means has a determination criterion 2 having a plurality of criteria when determining that the first voice is desired data using the determination voice data, and the determination criterion 2 satisfies the determination result. An electronic device that changes the content output from the output unit according to any of the plurality of criteria. The electronic device according to claim 1, wherein the management means can create and manage a plurality of the determination voice data. Based on the content of the first voice input from the outside, the execution of control of one or a plurality of devices is determined based on the content of the second voice input after the first voice is input. In the control method
Judgment voice data for determining that the first voice is a desired voice is created and managed by voice input from the outside a plurality of times, and the judgment voice data created and managed is used to describe the above. A management means for determining that the first voice is a desired voice, and
A control means for executing control of the one or a plurality of devices based on the content of the second voice is provided, and the management means uses the determination voice data to make the first voice a desired voice. This is a control method in which the control means executes control of the one or a plurality of devices based on the content of the second voice when it is determined to be.
If the management means has an output unit that outputs the determination result by voice, and the management means determines that the first voice is a desired voice by using the judgment voice data, that effect. Is output from the output unit,
The management means has a determination criterion 2 having a plurality of criteria when determining that the first voice is desired data using the determination voice data, and the determination criterion 2 satisfies the determination result. A control method for changing the content output from the output unit according to any one of the plurality of criteria.

Images