WO2019198186A1

WO2019198186A1 - Electronic device and control method for same

Info

Publication number: WO2019198186A1
Application number: PCT/JP2018/015306
Authority: WO
Inventors: 秀人井澤; 玲子嘉和知; 邦朗本沢; 弘之野本
Original assignee: 東芝映像ソリューション株式会社
Priority date: 2018-04-11
Filing date: 2018-04-11
Publication date: 2019-10-17
Also published as: CN111656314A

Abstract

An objective of the present embodiment is to provide an electronic device and a control method for the same, the electronic device controlling various devices, which are connected by a network, so as to match the lifestyle of individual users.　An electronic device according to the present embodiment determines, from the content of an externally input first utterance, to control one or a plurality of devices on the basis of the content of a second utterance input after the first utterance. The electronic device comprises: a management means which uses a plurality of externally input utterances to create and manage determining data for determining that the first utterance is a desired utterance, and which uses the created and managed determining utterance data to determine that the first utterance is the desired utterance; and a control means that controls the one or plurality of devices on the basis of the content of the second utterance. When the managements means has determined that the first utterance is the desired utterance by using the determining utterance data, the control means controls the one or plurality of devices on the basis of the content of the second utterance.

Description

Electronic device and control method thereof

Embodiments of the present invention relate to an electronic device that controls a plurality of devices by voice and a control method thereof in the field of home automation in homes, offices, and small-scale offices.

Conventionally, in the field of home automation, there are voice recognition devices and methods for operating and controlling various devices in homes, offices and small-scale offices by voice input.

This voice recognition apparatus and method analyze voice inputted from a user to determine whether the inputted voice is voice that turns on the function of the apparatus, When it is determined that the sound is to turn on, the content of the continued sound is analyzed and processing based on the analysis result is performed. In addition, there is a method of identifying a user who has emitted a voice by recognizing characteristics of the voice input by the user and performing a process suitable for the user.

International Publication No.2015-029379 International Publication No. 2015-033523

As a form of a home automation system, there is a type in which each device is connected to each other via a home network, and a host device that totally controls the plurality of connected devices is connected to the network. In this case, the host device controls the operation of each device connected via the network, or manages information so that the user can centrally browse and collect information about each device.

For example, the user can control each device connected to the host device via a network by instructing the host device by voice, for example, and centrally browse information on each connected device. it can.

In such a form of home automation system, devices to be controlled can be easily connected via a network, so the number and types of connected devices tend to be large. In addition, there is a tendency that new participation in the network, setting change, and withdrawal from the network frequently occur due to addition, change, version upgrade, movement and disposal of the installation target device. In addition, since there are many types of operation contents and specifications of connected devices, home automation systems tend to be used in homes and offices regardless of gender. In particular, with the recent miniaturization of devices and sensors having a wide variety of functions, this trend has become more prominent.

However, conventional home automation systems are not sufficient for controlling a wide variety of devices and supporting a wide range of users. For example, when a home automation system is used in a home, it cannot be said that a device that matches the lifestyle of each family is sufficiently controlled.

The present embodiment has been made in view of the above problems, and proposes an electronic device that controls various devices connected by a network so as to match each lifestyle of the user and a control method thereof. Objective.

The electronic device according to the embodiment includes one or a plurality of devices based on the content of the first sound input from the outside, based on the content of the second sound input after the input of the first sound. In the electronic device that determines the execution of the control, the sound data for determination for determining that the first sound is a desired sound is created and managed by the sound input from the outside a plurality of times, and is created and managed. Management means for determining that the first sound is a desired sound using the determination sound data, and controlling the one or a plurality of devices based on the content of the second sound And when the management means determines that the first sound is a desired sound, the control means changes the content of the second sound to the content of the second sound. Based on said one or To perform the control of the few cars of the equipment.

FIG. 1 is a diagram illustrating an example of an overall image of a home automation system according to an embodiment. FIG. 2 is a list showing another example of the sensor according to the embodiment. FIG. 3 is a diagram illustrating an example of a host device according to an embodiment. FIG. 4 is a functional block diagram of the host device according to the embodiment. FIG. 5A is a diagram illustrating an example of a processing sequence in registering a reserved word according to an embodiment. FIG. 5B is a diagram illustrating an example of a processing sequence in registering a reserved word according to an embodiment. FIG. 6A is a diagram illustrating an example of a processing sequence in registering a reserved word according to an embodiment. FIG. 6B is a diagram illustrating an example of a processing sequence in registering a reserved word according to an embodiment. FIG. 7A is a diagram illustrating an example of a processing sequence in registering a reserved word according to an embodiment. FIG. 7B is a diagram illustrating an example of a processing sequence in registering a reserved word according to an embodiment. FIG. 8A is a diagram showing an example of a processing sequence in reserved word recognition according to an embodiment. FIG. 8B is a diagram illustrating an example of a processing sequence in reserved word recognition according to an embodiment. FIG. 9A is a diagram showing an example of a processing sequence in reserved word recognition according to an embodiment. FIG. 9B is a diagram showing an example of a processing sequence in reserved word recognition according to an embodiment. FIG. 10A is a diagram illustrating an example of a processing sequence for controlling a corresponding device or sensor based on words for controlling the device or sensor continuously issued by a user after recognizing a reserved word according to an embodiment. is there. FIG. 10B is a diagram illustrating an example of a processing sequence for controlling a corresponding device or sensor based on words for controlling the device or sensor continuously issued by the user after recognizing a reserved word according to an embodiment. is there. FIG. 11A is a diagram illustrating an example of a processing sequence in a case where words for controlling devices and sensors continuously issued by a user after a reserved word according to an embodiment is continued within a certain period of time. FIG. 11B is a diagram illustrating an example of a processing sequence in a case where words for controlling devices and sensors continuously issued by a user after a reserved word according to an embodiment is continued within a certain period of time. FIG. 12A is a diagram illustrating an example of a processing sequence in a case where words for controlling devices and sensors continuously issued by a user after a reserved word according to an embodiment is continued for a certain period of time. . FIG. 12B is a diagram illustrating an example of a processing sequence in a case where words for controlling devices and sensors continuously issued by a user after a reserved word according to an embodiment is continued for a certain period of time. . FIG. 13 is a list specifically showing the contents of control information used when controlling devices and sensors after recognizing a reserved word according to an embodiment. FIG. 14 is a list showing examples of operation contents to be changed according to a plurality of reserved words according to an embodiment. FIG. 15A is a diagram illustrating an example of a processing sequence for registering a plurality of reserved words according to an embodiment, in which operation contents to be changed according to each reserved word are also registered. FIG. 15B is a diagram illustrating an example of a processing sequence for registering a plurality of reserved words according to an embodiment, in which operation contents to be changed according to each reserved word are also registered. FIG. 16A is a diagram illustrating an example of a processing sequence for setting the operation content according to each reserved word in the recognition of the reserved word according to the embodiment. FIG. 16B is a diagram illustrating an example of a processing sequence for setting operation contents according to each reserved word in the recognition of the reserved word according to the embodiment. FIG. 17 is a list showing examples of operation contents set in accordance with words continuing to the reserved words in the reserved words according to the embodiment. FIG. 18A is a diagram illustrating an example of a processing sequence for setting the operation content according to a word continuing to a reserved word in recognition of a registered reserved word according to an embodiment. FIG. 18B is a diagram showing an example of a processing sequence for setting the operation content according to words that continue to the reserved word in the recognition of the registered reserved word according to the embodiment. FIG. 18C is a diagram showing an example of a processing sequence for setting the operation content according to a word continuing to the reserved word in the recognition of the registered reserved word according to the embodiment. FIG. 18D is a diagram illustrating another example of a processing sequence for setting operation contents according to words that continue to the reserved word in recognition of a registered reserved word according to an embodiment. FIG. 18E is a diagram showing another example of a processing sequence for setting operation content according to words that continue to the reserved word in recognition of a registered reserved word according to an embodiment. FIG. 19A is a diagram illustrating an example of a processing sequence for setting operation contents according to words that continue to the recognized reserved word in the recognition of the reserved word according to the embodiment. FIG. 19B is a diagram showing an example of a processing sequence for setting the operation content according to a word continuing to the recognized reserved word in the recognition of the reserved word according to the embodiment. FIG. 20 is a list showing examples of types of speech recognition dictionaries used in accordance with reserved words in the recognition of a plurality of reserved words according to an embodiment. FIG. 21A is a diagram showing an example of a processing sequence for changing the type of speech recognition dictionary used in accordance with a reserved word in the recognition of a plurality of reserved words according to an embodiment. FIG. 21B is a diagram showing an example of a processing sequence for changing the type of speech recognition dictionary used in accordance with a reserved word in the recognition of a plurality of reserved words according to an embodiment. FIG. 22 is a list showing an example in which, in the recognition of a plurality of reserved words according to one embodiment, the words that continue to the reserved words, the operation contents set according to the reserved words, and the type of the speech recognition dictionary to be used are changed. FIG. 23 is a list showing an example of changing the type of the speech recognition dictionary according to the embodiment according to contents other than reserved words. FIG. 24 is a diagram showing a sequence of processing for registering the type of the speech recognition dictionary to be changed according to the content other than the reserved word in the change of the type of the speech recognition dictionary according to the embodiment. FIG. 25 is a diagram showing a processing sequence when changing the type of the speech recognition dictionary to be registered according to the contents other than the reserved word in the change of the type of the speech recognition dictionary according to the embodiment. FIG. 26 shows a reserved word (for relief) for displaying a reserved word and a corresponding reserved word when the user forgets a registered reserved word in the processing according to the embodiment. It is a list which shows the example of a range. FIG. 27 is a functional block diagram of a host device according to an embodiment. FIG. 28 shows a case where the host device 332 detects that a scene for registering a reserved word, additional word, or additional information or a scene for recognizing a reserved word, additional word occurs in the processing according to an embodiment. It is a figure which shows an example of time passage at the time of recording or recording a registration scene or a recognition scene. FIG. 29 is a diagram illustrating an example of a state in which data to be reproduced is displayed when reproducing each data of a recorded or recorded scene according to an embodiment.

FIG. 1 is a diagram showing an example of the overall configuration of the home automation system according to the present embodiment. The home automation system includes a cloud server 1 including a group of servers placed in the cloud and various sensors 310 and various equipment devices 320 connected to each other via a network 333 via a host device 332 having a HGW (HomeGateway) function. It consists of a home 3 in which various home appliances 340 are arranged, and the Internet 2 that connects the cloud server 1 and the host device 332.

The home 1 is a home, office, or small scale where various sensors 310, various equipment devices 320, and various home appliances 340 are connected to each other via a home network 333 via a host device 332 having an HGW function. It is a business establishment and its size is not limited.

The host device 332 controls devices and sensors connected via the network 333 based on information set in advance and information notified from sensors connected via the network 333, and also relates to each device and sensor. It has a function to centrally manage information.

Furthermore, the host device 332 has a microphone and can capture words uttered by the user 331. When the host device 332 recognizes a predetermined keyword (hereinafter referred to as a reserved word) from words uttered by the user 331, the host device 332 takes in the words uttered by the user 331 following the reserved words, and By analyzing the contents, a response corresponding to the analysis result is returned to the user 331, or devices and sensors connected via the network 333 are controlled according to the analysis result.

Conversely, unless the host device 332 recognizes a reserved word from the words uttered by the user 331, the host device 332 does not continuously capture the words uttered by the user 331. This prevents the host device 332 from picking up unnecessary surrounding sounds and operating.

The recognition of reserved words is performed in the host device 332, the words uttered by the user 331 following the reserved words are continuously captured, and the contents of the captured words are analyzed in the cloud server 1. Details of the function of the host device 332 will be described later.

The various equipment 320 and the various household appliances 340 mean equipment for which the equipment 320 is not easily moved for convenience of explanation, and mean that the various household appliances 340 are relatively easy to move. The names of the exemplified equipment and home appliances do not limit the capabilities and functions of the individual devices.

Specific examples of the various sensors 310 include a security camera 311, a fire alarm 312, a human sensor 313, and a temperature sensor 314. Specific examples of the various equipment 320320 include an interphone 325, an illumination 326, an air conditioner 327, and a water heater 328. Specific examples of the various home appliances 340 include a washing machine 341, a refrigerator 342, a microwave oven 343, a fan 344, a rice cooker 345, and a television 346.

FIG. 2 shows another example of the various sensors 310 shown in FIG.

FIG. 3 shows various examples of the host device 332 shown in FIG.

The host device 332-1 is the host device 332 shown in FIG. 1, and is an example of a stationary type having a built-in HGW function. The host device 332-1 is connected to other devices and sensors arranged in the home 1 through the network 333, and is connected to the cloud server 1 through the Internet 2. Since the host device 332-1 is a stationary type, for example, an autonomous moving means such as a motor is not mounted.

The host device 332-2 is an example of a stationary type that does not have a built-in HGW function. Therefore, the host device 332-2 is connected to the HGW 330 through the network 333. The host device 332-2 is connected to other devices and sensors disposed in the home 1 via the network 333 via the HGW 330, and is connected to the cloud server 1 via the Internet 2 via the HGW 330. Since the host device 332-2 is a stationary type, for example, an autonomously moving means such as a motor is not mounted.

The host device 332-3 is a movable example having a built-in HGW function. The host device 332-3 is connected to other devices and sensors through the network 333, and is connected to the cloud server 1 through the Internet 2. Since the host device 332-3 is a movable type, it is an example in which means for autonomously moving, such as a motor, is mounted.

The host device 332-4 is an example of a movable type that does not have a built-in HGW function. Therefore, the host device 332-4 is connected to the HGW 330 through the network 333. The host device 332-4 is connected to other devices and sensors via the network 333 via the HGW 330, and is connected to the cloud server 1 via the Internet 2 via the HGW 330. Since the host device 332-4 is movable, it is an example in which means for autonomously moving, such as a motor, is mounted.

FIG. 4 shows functional blocks of the host device 332 shown in FIG. The host device 332 is connected to the system controller 402 that controls the entire internal processing, and the control management unit 401, trigger setting unit 403, trigger recognition unit 405, input management unit 420, and network 333 that controls each function thereby. Network I / F 427. The control management unit 401 internally manages a plurality of applications for controlling various operations of the host device 332, initial settings of various functional blocks of the host device 332, various state settings, operation settings, and the like. It consists of CONF-Mg 401-2 for managing the setting contents.

In addition, the host device 332 serves as an interface (I / F) with the user 331, such as a microphone 421 for capturing words uttered by the user 331, a speaker 423 for outputting a response to the user 331 by voice, and the user 331. And a display unit 425 for notifying the status of the host device 332.

The microphone 421 is connected to the input management unit 420. The input management unit 420 controls whether the voice data input from the microphone 421 is sent to the trigger setting unit 403, the trigger recognition unit 405, or the voice processing unit 407 according to the state managed internally. The display unit 425 notifies the state of the host device 332 to the user 331, and is, for example, an LED (Light Emitting Diode) or an LCD (Liquid Crystal Display).

The memory 410 is divided into three areas: an operation mode storage area 410-1, a reserved word storage area 410-2, and a voice storage area 410-3. The contents of the information stored in each area will be described later.

As described above, when the function of the host device 332 recognizes a reserved word among the words uttered by the user 331, the function utters the word uttered by the user 331 to the reserved word and displays the contents of the imported word. By performing the analysis, it has a function of returning a response according to the analysis result to the user 331 and controlling the operation of devices and sensors connected through the network 333.

In order to realize these functions, the host device 332 performs four processes. The first process is registration of reserved words. The second process is recognition of reserved words. The third process is registration of control contents of devices and sensors that control operations. The fourth process is control of devices and sensors for which control contents are registered.

First, registration of reserved words, which is the first process, will be described.
The host device 332 has a function of registering a reserved word in the host device 332. In order to register a reserved word, the host device 332 has a mode for registering a reserved word (hereinafter referred to as a reserved word registration mode).

FIG. 5A and FIG. 5B show examples of processing sequences of the host device 332 from the start of reserved word registration to the completion of registration in a state where the host device 332 is in the “reserved word registration mode” in order to register a reserved word. Is shown.

Note that the host device 332 may be able to change the mode by recognizing words uttered by the user 331 in a predetermined order in order to change the mode. Alternatively, a menu screen may be displayed on the display unit 425, and the mode may be changed by the user 331 operating the menu screen. Alternatively, the mode may be changed by the user 331 operating a menu screen for changing the mode of the host device 332 displayed on the smartphone or tablet connected via the network I / F 427.

When the user 331 utters a word to be registered as a reserved word, the host device 332 takes the voice data input from the microphone 421 into the input management unit 420 (S501). The input management unit 420 has a function of determining a transfer destination of input audio data according to a state managed internally. When the mode of the host device 332 is the setting mode, the input management unit 420 transfers the received audio data to the trigger setting unit 403 (S502). The trigger setting unit 403 stores the received audio data in the audio storage area 410-3 of the memory 410 (S503), and confirms whether the number of times the user 331 has acquired the audio has reached the specified number (S504). .

The trigger setting unit 403 displays a display prompting the user 331 to utter a word to be registered when it is determined that the specified number has not been reached as a result of checking whether the number of times the user 331 has captured the voice has reached the specified number. (S507) and an input continuation notification is sent to the input management unit 420 (S506). Upon receiving the input continuation notification, the input management unit 420 changes the internal state to a state waiting for voice input from the microphone (S500).

The display prompting the user 331 to input a word to be registered is a display in which the trigger setting unit 403 transmits a registration incomplete notification to the display device 425 (S505) and receives the registration incomplete notification. It is desirable that the display method be recognized by the user 331, for example, the device 425 blinks the light emitting diode (LED) in red (S507). Further, instead of the display method, a voice method may be used to prompt the user 331 to input a registered word. In this case, the trigger setting unit 403 transmits a registration incomplete notification to the speaker 423, and the speaker 423 that has received the registration incomplete notification announces to the user 331, for example, “Please input again”. But you can. Alternatively, the trigger setting unit 403 may use both a display method and a voice method to prompt the user 331 to input words to be registered. Alternatively, when the host device 332 is a movable type, the trigger setting unit 403 may issue an instruction to a moving unit that is not described so that the host device 332 repeatedly rotates and moves with a certain angular width, for example.

If the trigger setting unit 403 determines that the prescribed number of times has been reached as a result of checking whether the number of times the user 331 has captured the voice has reached the prescribed number, the trigger setting unit 403 stores it in the voice accumulation area 410-3 until then. Certain voice data is read (S508) and sent to the recognition data conversion unit 101-1 in the voice recognition cloud 101 in the cloud server 1 through the Internet 2 (S509).

The recognition data conversion unit 101-1 converts the voice data sent from the trigger setting unit 403 into recognition data for recognition as a reserved word (S510). When the conversion to the recognition data is completed, the recognition data conversion unit (101-1) sends the recognition data to the trigger setting unit 403 through the Internet 2 (S511). Upon receiving the recognition data, the trigger setting unit 403 stores the received data in the reserved word storage area 410-2 of the memory 410 (S512).

The trigger setting unit 403 displays (S514) informing the user 331 that registration of the reserved word is completed. In the display informing the user 331 that the registration of the reserved word is completed, the trigger setting unit 403 transmits a registration completion notification to the display device 425 (S514), and the display device 425 that has received the registration completion notification receives the registration completion notification. For example, it is desirable to use a display method that the user 331 can recognize, such as lighting the LED in green. Alternatively, the trigger setting unit 403 may use a voice method instead of the display method to notify the user 331 that registration of the reserved word has been completed. In this case, the trigger setting unit 403 may transmit a registration completion notification to the speaker 423, and the speaker 423 that has received the registration completion notification may announce, for example, “registration is completed” to the user 331. . Alternatively, the trigger setting unit 403 may use both a display method and a voice method to notify the user 331 that registration of the reserved word has been completed. Alternatively, when the host device 332 is a movable type, the trigger setting unit 403 may issue an instruction to a moving unit that is not described so that the host device 332 repeatedly moves linearly with a certain moving width, for example.

As described above, the trigger setting unit 403 has a role of managing the flow of data in registering reserved words.

6A and 6B show another sequence example from the start of registration of reserved words to the completion of registration. In some cases, it is insufficient to register the voice data captured by the host device 332 as a reserved word. An example of processing when the captured data is insufficient will be described.

The processing from S600 to S615 shown in FIGS. 6A and 6B is the same as the processing from S500 to S515 shown in FIGS. 5A and 5B, respectively. The difference between the process in FIGS. 5A and 5B and the process in FIGS. 6A and 6B is that the process of S616 is added to the process of FIG. 6B.

If the trigger setting unit 403 confirms whether the number of times the words uttered by the user 331 have been taken reaches the specified number (S604), and determines that the number has reached the specified number, the trigger setting unit 403 The voice data stored in 410-3 is read (S608), and sent to the recognition data conversion unit 101-1 in the voice recognition cloud 101 in the cloud server 1 through the Internet 2 (S609).

When the trigger setting unit 403 determines that the number of times the words uttered by the user 331 are not reached the specified number, the trigger setting unit 403 displays a message prompting the user 331 to utter the words to be registered (S607) and the input management unit An input continuation notice is sent to 420 (S606). The input management unit 420 that has received the input continuation notification transitions the internal state to a state waiting for input of sound from the microphone (S600).

The display prompting the user 331 to input a word to be registered is a display in which the trigger setting unit 403 transmits a registration incomplete notification to the display device 425 (S605) and receives the registration incomplete notification. It is desirable that the display method be recognized by the user 331, for example, the device 425 blinks the LED in red (S607). Further, instead of the display method, a voice method may be used to prompt the user 331 to input a registered word. In this case, the trigger setting unit 403 transmits a registration incomplete notification to the speaker 423, and the speaker 423 that has received the registration incomplete notification announces to the user 331, for example, “Please input again”. But you can. Alternatively, the trigger setting unit 403 may use both a display method and a voice method to prompt the user 331 to input words to be registered. Alternatively, when the host device 332 is a movable type, the trigger setting unit 403 may issue an instruction to a moving unit that is not described so that the host device 332 repeatedly rotates and moves with a certain angular width, for example.

The recognition data conversion unit 101-1 determines whether or not the received voice data can be converted into recognition data when converting all the voice data sent from the trigger setting unit 420 into the recognition data. (S616). If it is determined that some of the transmitted voice data cannot be converted into recognition data, the recognition data conversion unit 101-1 transmits a voice data addition request to the trigger setting unit 403 through the Internet 2 (S617). To do. Upon receiving the voice data addition request, the trigger setting unit 403 sets the number of times that the user 331 additionally inputs a word to be registered as a reserved word (S618), and notifies the input management unit 420 of continued input (S619). To be notified.

At the time when the trigger setting unit 403 sets the additional number of times that the user 331 additionally inputs (S618), for example, the LED of the display unit 425 remains lit in red. In accordance with this display, the user 331 issues words to be registered as reserved words for the number of times additionally set in S618.

When receiving the input continuation notification (S619), the input management unit 420 changes the internal state to input waiting (S600), and enters an input waiting state for words uttered by the user 331.

The processing shown in FIGS. 5A and 5B and the processing shown in FIGS. 6A and 6B are performed by collecting the audio data acquired after the number of times that the input management unit 402 has acquired the audio uttered by the user 331 reaches the specified number. This is an example of transmitting to the recognition data conversion unit 101-1 in the cloud server 1, but each time the input management unit 420 captures the voice uttered by the user 331, the captured voice data is converted into the recognition data conversion unit 101-. 1 may be transmitted. 7A and 7B, every time the input management unit 420 captures the voice uttered by the user 331, the captured voice data is sequentially sent to the recognition data conversion unit 101-1 in the cloud server 1 for recognition. It is an example of a sequence in the case of converting into data.

The processing from S700 to S702 shown in FIG. 7A is the same as the processing shown from S500 to S502 in FIG. 5A, respectively. Further, the processes of S703 and S704 shown in FIG. 7A are the same as the processes of S505 and S507 shown in FIG. 5A, respectively.

When the user 331 utters a word to be registered as a reserved word, the host device 332 takes the voice data input from the microphone 421 into the input management unit 420 (S701). Since the mode of the host device 332 is the reserved word registration mode, the input management unit 420 transfers the received voice data to the trigger setting unit 403 (S702). The trigger setting unit 403 sequentially transmits the received audio data to the recognition data conversion unit 101-1 in the cloud server 1 every time it is received (S706). The recognition data conversion unit 101-1 determines whether the received voice data can be converted into recognition data when converting the voice data sent from the trigger setting unit 403 into recognition data ( S707).

If it is determined that the received voice data cannot be converted into recognition data, the recognition data conversion unit 101-1 transmits a voice data addition request to the trigger setting unit 403 via the Internet 2 (S708). Upon receiving the voice data addition request, the trigger setting unit 403 (S708) checks whether the number of times the user 331 has taken in the voice has reached the specified number (S714). The trigger setting unit 403 displays a display prompting the user 331 to utter a word to be registered when it is determined that the specified number has not been reached as a result of checking whether the number of times the user 331 has captured the voice has reached the specified number. At the same time, an input continuation notice is sent to the input management unit 420 (S715), thereby causing the input management unit 420 to transition to a state waiting for voice input from the microphone (S700). When receiving the input continuation notification (S715), the input management unit 420 changes the internal state to input waiting (S700), and enters a state of waiting for input of words uttered by the user 331.

When it is determined that the received voice data can be converted into recognition data (S707), the recognition data conversion unit 101-1 converts the voice data into recognition data (S709). The recognition data conversion unit 101-1 converts the voice data input from the microphone 421 using all the recognition data including those already converted into the recognition data as a result of the conversion to the recognition data (S709). It is determined whether or not accuracy that can be recognized as a reserved word is secured (S710).

If it is determined that the voice data input from the microphone 421 is sufficiently accurate to be recognized as a reserved word by all the recognition data, the user 331 issues a word to be registered as a reserved word. In order to stop, the trigger setting unit 403 is notified to the trigger setting unit 403 via the Internet 2 of recognition data (with recognition data full notification) to which information indicating that the recognition data is sufficient is added (S711). The trigger setting unit 403 that has received the recognition data (recognition data sufficiency) is the recognition data received up to this point and sufficient recognition data to recognize the voice data input from the microphone 421 as a reserved word. Even if the number of times the user 331 has captured the voice has not reached the specified number, the user 331 is no longer prompted to input words to be registered (S712). The trigger setting unit 403 stores all the recognition data received up to this point in the reserved word storage area 410-2 (S716), and also stores it in the input management unit 420, the display unit 425, and the recognition data conversion unit 101-1. A registration completion notification is sent (S717) (S718) (S719). As a result, the accuracy of the converted recognition data makes it possible to stop the user 331 from issuing a word to be registered as a reserved word because the number of times the user's 331 has been captured does not reach the specified number. This makes it possible to register reserved words with a higher degree of freedom. The specified number of times can be changed by the user 331 as a setting value of the host device 332, and can be changed as one of additional information described later.

If the recognition data conversion unit 101-1 determines that the recognition data created up to this point does not ensure sufficient accuracy to recognize the voice data input from the microphone 421 as a reserved word Only the converted recognition data is sent to the trigger setting unit 403 (S713). The trigger setting unit 403 that has received the recognition data checks whether or not the number of times the user's 331 voice has been captured has reached the specified number (S714). If the trigger setting unit 403 determines that the specified number has not been reached as a result of checking whether the specified number has been reached, the trigger setting unit 403 continues to display the user 331 to utter the words to be registered, and to the input management unit 420. By sending an input continuation notification (S715), the input management unit 420 is shifted to a state waiting for voice input from the microphone (S700).

The display prompting the user 331 to input a word to be registered is a display in which the trigger setting unit 403 transmits a registration incomplete notification to the display device 425 (S703) and receives the registration incomplete notification. It is desirable that the display method be recognized by the user 331, for example, the device 425 blinks the LED in red (S704). Further, instead of the display method, a voice method may be used to prompt the user 331 to input a registered word. In this case, the trigger setting unit 403 transmits a registration incomplete notification to the speaker 423, and the speaker 423 that has received the registration incomplete notification announces to the user 331, for example, “Please input again”. But you can. Alternatively, the trigger setting unit 403 may use both a display method and a voice method to prompt the user 331 to input words to be registered. Alternatively, when the host device 332 is a movable type, the trigger setting unit 403 may issue an instruction to a moving unit that is not described so that the host device 332 repeatedly rotates and moves with a certain angular width, for example.

When the trigger setting unit 403 that has received the recognition data determines that the specified number has been reached as a result of checking whether the specified number has been reached (S714), a registration completion notification is received from the input management unit 420, the display unit 425, and the recognition. A registration completion notice is sent to the data conversion unit 101-1 (S717) (S718) (S719). Receiving the registration completion notification (S718), the recognition data conversion unit 101-1 clears the converted recognition data temporarily stored for performing the processing of S710.

Next, recognition of reserved words, which is the second process of the host device 332, will be described.

When the host device 332 recognizes a reserved word among the words uttered by the user 331, the host device 332 analyzes the contents of the words uttered by the user 331, and controls devices and sensors based on the analysis result. It has a function. In order to recognize the reserved word and control the device and sensor after recognizing the reserved word, the host device 332 has a mode for recognizing the reserved word and controlling the device and sensor (hereinafter referred to as an operation mode). ing.

8A and 8B show an example of a processing sequence of the host device 332 until it is recognized that the word uttered by the user 331 is one of registered reserved words in the operation mode.

When the user 331 utters a word, the host device 332 takes the voice data input from the microphone 421 into the input management unit 420 (S801). When the mode of the host device 332 is the operation mode, the input management unit 420 transfers the received voice data to the trigger recognition unit 405 (S802). When the trigger recognizing unit 405 receives the voice data transferred from the input management unit 420, the trigger recognizing unit 405 determines whether the transferred voice data is a reserved word or not in the reserved word storage area 410-2 of the memory 410. The data is compared with the recognition data read out (S803) (S804).

If the trigger recognition unit 405 determines that the input voice data cannot be recognized as a reserved word (S805), the trigger recognition unit 405 displays a message prompting the user 331 to emit the reserved word (S808) and inputs the input data to the input management unit 420. A continuation notice is sent (S807). Note that the display that prompts the user 331 to issue a reserved word is such that the trigger recognition unit 405 transmits a recognition incomplete notification to the display unit 425 (S806), and the display unit 425 that has received the recognition incomplete notification receives, for example, It is desirable to use a display method that the user 331 can recognize, such as blinking the LED in red (S808). The trigger recognizing unit 405 may prompt the user 331 to input a voice by using a voice method instead of the display method. In this case, the trigger recognizing unit 405 transmits a recognition incomplete notification to the speaker 423, and the speaker 423 that has received the recognition incomplete notification announces to the user 331, for example, "I did not hear". Good. Alternatively, the trigger recognition unit 405 may use both a display method and a voice method to prompt the user 331 to input a voice. Alternatively, when the host device 332 is a movable type, the trigger setting unit 403 may issue an instruction to a moving unit that is not described so that the host device 332 repeatedly rotates and moves with a certain angular width, for example.

When the input voice data can be recognized as a reserved word (S805), the trigger recognizing unit 405 displays that the voice uttered by the user 331 is recognized as a reserved word (S810). The display indicating that the voice uttered by the user 331 is recognized as a reserved word is displayed by the trigger recognition unit 405 transmitting a recognition completion notification to the display device 425 (S809) and receiving the recognition completion notification. It is desirable to use a display method that can be recognized by the user 331, for example, 425 turns on the LED in green (S810). The trigger recognizing unit 405 may notify that the voice uttered by the user 331 is recognized as a reserved word by using a voice method instead of the display method. In this case, the trigger recognition unit 405 transmits a recognition completion notification to the speaker 423, and the speaker 423 that has received the recognition completion notification also announces to the user 331, for example, “Yes” or “I heard it”. Good. Alternatively, the trigger recognition unit 405 may use both a display method and a voice method to indicate that the voice uttered by the user 331 is recognized as a reserved word. Alternatively, when the host device 332 is a movable type, the trigger setting unit 403 may issue an instruction to a moving unit that is not described so that the host device 332 repeatedly moves linearly with a certain moving width, for example.

FIG. 9A and FIG. 9B show another example of the processing sequence of the host device 332 until the word uttered by the user 331 is recognized as one of registered reserved words in the operation mode.

The difference between the sequence example of FIGS. 9A and 9B and the sequence example of FIGS. 8A and 8B is that the recognition probability is taken into consideration in the process of recognizing the reserved word. The recognition probability means that the recognition data is compared with the feature points such as the frequency component and strength of the voice data transferred from the input management unit 420, and the two match each other. The processes from S900 to S912 shown in FIGS. 9A and 9B are the same as the processes shown from S800 to S812, respectively, and the process in FIGS. 9A and 9B is different from the processes in FIGS. 8A and 8B in the processes from S913 to S916. This is the point where processing is added.

Upon receiving the voice data transferred from the input management unit 420, the trigger recognition unit 405 reads the recognition data from the reserved word storage area 410-2 of the memory 410 (S903), and is transferred from the input management unit 420. Comparison with audio data is performed (S904).

When the trigger recognition unit 405 determines that the input voice data is recognized as a reserved word (S905), the trigger recognition unit 405 proceeds to a recognition probability determination process (S913).

Here, the speech recognition processing performed by the trigger recognition unit 405 includes characteristics such as the frequency component and strength of the recognition data read from the reserved word storage area 410-2 of the memory 410 and the voice data transferred from the input management unit 420. A point is compared, and when both coincide with each other at a certain level or more, it is determined that the voice data transferred from the input management unit 420 is recognition data.

When the host device 332 compares the recognition data with the feature points such as the frequency component and strength of the audio data transferred from the input management unit 420, the host device 332 sets a threshold value for determining the level at which the two match. It is also possible to provide a plurality. By doing in this way, the host device 332 is not a two-way determination that the reserved word can be recognized / the reserved word cannot be recognized when recognizing the reserved word among the words uttered by the user, For example, it is possible to add a determination that a reserved word is not a correct reserved word, such that a reserved word can be recognized / a reserved word cannot be recognized / a reserved word cannot be recognized. By providing a plurality of recognition probability thresholds in this way, when the user 331 does not remember a reserved word accurately, for example, the user 331 repeatedly utters a word close to the reserved word, thereby capturing the word uttered by the user 331. The host device 332 responds according to the determination result that “the reserved word cannot be recognized”, and there is an advantage that the user 331 who sees the response content can approach the correct reserved word.

9A and 9B are examples in the case where two recognition probability thresholds are provided. Assuming that the threshold value for recognizing a reserved word is threshold value 1 and the threshold value for not recognizing a reserved word is threshold value 0, as a result of comparison in S904, if the recognition probability is greater than or equal to threshold value 1, it is determined that the reserved word has been recognized. It becomes. If the recognition probability is greater than or equal to the threshold 0 and less than the threshold 1, it is determined that the reserved word cannot be recognized. When the recognition probability is less than the threshold 0, it is determined that the reserved word cannot be recognized. Therefore, the process of S905 is a process of comparing the recognition probability with the threshold 0. The process of S913 is a process of comparing the recognition probability with the threshold 1 in size.

When the host device 332 determines that the recognition probability is greater than or equal to the threshold value 0 and less than the threshold value 1 (S913), the host device 332 displays a message prompting the user 331 to emit a reserved word (S915) and continues input to the input management unit 420. A notification is sent (S916). The display that prompts the user 331 to issue a reserved word is that the trigger recognition unit 405 sends an insufficient recognition notification to the display unit 425 (S914), and the display unit 425 that has received the insufficient recognition notification, for example, It is desirable to use a display method that the user 331 can recognize, such as blinking the LED in green (S915).

In this way, when the recognition probability is low, the display that prompts the user 331 to issue a reserved word can be changed from the display when the recognition is unsuccessful (S908) or the display when the recognition is successful (S910), The user 331 can recognize that his / her words are close to the reserved words, but do not correctly issue the reserved words.

Also, the trigger setting unit 403 may prompt the user 331 to input a voice by using a voice method instead of the display method. In this case, the trigger recognizing unit 405 transmits an insufficient recognition notification to the speaker 423 (S914), and the speaker 423 that has received this insufficient recognition notification, for example, “has something called?” To the user 331. The method of announcement may be used. Alternatively, the trigger recognition unit 405 may use both a display method and a voice method to prompt the user 331 to input a voice. Alternatively, when the host device 332 is a movable type, the trigger setting unit 403 may issue an instruction to a moving unit that is not described so that the host device 332 repeatedly rotates and moves with a certain angular width, for example.

Next, the third processing of the host device 332, registration of the control contents of the device and sensor that controls the operation, and the fourth processing, control of the device and sensor in which the control content is registered. explain.

First, an overall image of device and sensor control using the host device 332 will be described.

When the host device 332 recognizes a reserved word among the words uttered by the user 331, the host device 332 continuously captures the words uttered by the user after recognizing the reserved words, and analyzes the contents of the captured words. It has a function to control equipment and sensors.

10A and 10B show a case where the host device controls the device or sensor based on the content of the audio data including the control content of the device or sensor captured from the microphone 421 after the reserved word is recognized. An example of the processing sequence is shown. The internal state of the input management unit 420 transitions to recognized (S1000) since reserved word recognition has been completed.

When the user 331 utters a word including content for controlling the device and the sensor, the host device 332 takes the voice data (control content) into the input management unit 420 through the microphone 421 (S1002). Since the internal state has already been recognized, the input management unit 420 transfers the input voice data (control content) to the voice processing unit 407 (S1002). The voice processing unit 407 sends the transferred voice data (control contents) to the voice text conversion unit 101-2 in the voice recognition cloud 101 in the cloud server 1 via the Internet 2.

The voice text conversion unit 101-2 performs a process of converting voice data sent through the Internet 2 into text data (S1004). By this process, the voice uttered by the user 331 originally captured through the microphone 421 is converted into text data.

When the conversion to the text data is completed, the speech text conversion unit 101-2 stores the converted text data therein and sends a conversion completion notification to the speech processing unit 407 (S1005).

Upon receiving the conversion completion notification, the voice processing unit 407 transmits a text analysis request to the voice text conversion unit 101-2 (S1006). When receiving the text analysis request, the voice text conversion unit 101-2 sends the text analysis request to the text analysis unit 102-1 together with the data converted into the text stored therein (S1007). When the text analysis unit 102-1 receives the text analysis request (S1007), it analyzes the content of the accompanying text data (S1008). When the analysis of the content of the sent text data is completed, the text analysis unit 102-1 sends the analysis result to the response / action generation unit 102-2 as a text analysis result notification (S1009). Upon receiving the text analysis result (S1009), the response / action generation unit 102-2 generates a target device and a command for controlling the device based on the content (S1010), and sends the generated command as a response / action. It is sent to the voice processing unit 407 as a generation result notification (S1011).

When the voice processing unit 407 receives the response / action generation result notification (S1011), the voice processing unit 407 specifies the device or sensor to be controlled and the control content from the response / action generation result notification (S1012). The voice processing unit 407 converts the specified control target device or sensor and its control contents into a format that can be recognized by the control target device or sensor, and sends an action notification to the target device or target sensor through the network 333 at a necessary timing. (S1013).

Upon receiving the action notification (S1013), the control target device or sensor that is the notification destination of the action notification performs an operation based on the control content included therein (S1014).

When the user 331 continuously utters a voice, the host device 332 determines that the continuous voice is a series of voices and requests the user 331 to issue a reserved word on the way without requesting the continuous voice. Can be captured. On the contrary, the host device 332 requests the input of the reserved word again when the user 331 utters a sound after a certain period of time. Each case will be described with reference to FIGS. 11A and 11B and FIGS. 12A and 12B.

FIG. 11A and FIG. 11B are examples of processing sequences in the case where the user 331 continuously utters words within the time T0 after the recognition of the reserved word is completed. When the host device 332 takes in the audio data (control content) input from the microphone 421 to the input management unit 420 (S1101), the input management unit 420 starts the input interval confirmation timer T. When the next voice data (control content) issued by the user 331 through the microphone 421 is taken into the input management unit 420 at the time T1 before the input interval confirmation timer T expires (= T0) (S1121) The unit 420 transfers the captured audio data (control content) to the audio processing unit 407 (S1122). At the same time, the activated input interval confirmation timer T is activated again. The voice processing unit 407 sends the transferred voice data (control content) to the voice text conversion unit 101-2 in the voice recognition cloud 101 in the cloud server 1 via the Internet 2 (S1123). Thereafter, similarly to the processes from S1104 to S1110, the process of the voice data (S1123) transmitted in the voice recognition cloud 101 is continued.

The input interval confirmation timer T is activated at the timing when the input management unit 420 takes in the voice data input from the microphone 421. However, the input management unit 420 is not limited to this, and for example, the input management unit 420 is sent from the microphone 421. You may start at the timing which transfers data to the trigger setting part 403 or the audio | voice processing part 407. Moreover, you may start at the timing which the internal state of the input management part 420 changes to recognized (S1100).

FIG. 12A and FIG. 12B are examples in the case where the user 331 does not continuously emit voice within the time T0. When the host device 332 takes in the audio data (control content) input from the microphone 421 to the input management unit 420 (S1201), the input management unit 420 starts the input interval confirmation timer T. When the input interval confirmation timer T expires (= T0), the input management unit 420 changes the internal state to wait for input (S1220).

When the host device 332 captures the next audio data input from the microphone 421 after the time (= T0) when the input interval confirmation timer T expires (S1224), the host device 332 performs the device based on the acquired audio data. And a process for controlling the sensor are performed, and a display prompting the user 331 to issue a reserved word is performed.

When the input interval confirmation timer T expires, the input management unit changes the internal state to waiting for input (S1220), and notifies the voice processing unit 407 of a timeout notification (S1221). The voice processing unit 407 that has received the timeout notification transmits a recognition incomplete notification to the display unit 425 (S1222), and the display unit 425 that has received the recognition incomplete notification notifies the user 331 to issue a reserved word. A prompting display, for example, the LED blinks in red (S1223).

When the next voice data input from the microphone 421 is captured after the input interval confirmation timer T expires (S1224), the input management unit 420 changes the internal state during recognition (S1225) and the captured voice. The data is transferred to the trigger recognition unit 405 (S1226). Thereafter, the host device 332 performs the processing from S803 to S812 in FIGS. 8A and 8B or the processing from S903 to S916 in FIGS. 9A and 9B, and recognizes the reserved word again.

Next, registration of control contents for controlling a device or sensor using the host device 332 and control of the device or sensor performed based on the registered control content will be described.

FIG. 13 shows control information used when the host device 332 recognizes a reserved word and controls various sensors 310, various equipment devices 320, and various home appliances 340 as shown in the sequence diagrams of FIGS. 10A and 10B. A specific example of the contents is shown.

Item 1 includes information for controlling various sensors 310, various equipment 320, and various home appliances 340 included in the response / action generation result notification transmitted from the response / action generation unit 102-2 (hereinafter, response / action This is a specific example). This response / action generation information includes a “target” such as a device or a sensor controlled by the device 332 and a “command” indicating the content for controlling the control target. When the host device 332 receives the response / action generation result notification, the host device 332 extracts the action information contained therein, and controls the target device based on the content of the action information.

Examples of “commands” include “start command” for starting (operating) a target device to be controlled, “stop command” for ending (stopping), and “operation change command for changing contents (mode) during operation” And “setting change command” for changing contents (modes) set in advance in the target device.

In order for the response / action generation unit 102-2 to generate the response / action information included in the response / action generation result notification, the user 331 previously controls the device to be controlled, its control contents, and the host device to control the device. It is necessary to register a combination of words issued to 332 in the response / action generation unit 102-2 as an initial setting of the host device 332. Hereinafter, the registration of response / action information in the initial setting of the host device 332 will be described with reference to the example of FIG.

Item 2 is “target” which is a device controlled through the host device 332. The “target” is an identification name of devices and sensors included in the various sensors 310, the various equipment devices 320, and the various home appliances 340, and the air conditioner 1 is described as a specific example.

Item 3 is a “command” which is the control content of the device shown in “Item 2”. This “command” describes the command of the air conditioner 1 listed in item 2 as a specific example. The “start command” for moving the air conditioner, the “stop command” for stopping the air conditioner, and the “change operation” for changing the operation content of the air conditioner. “Command” and “Setting change command” for changing the setting contents of the air conditioner are described as examples.

The product specifications of each device and sensor of item 2 and item 3 are stored in advance in a product specification cloud server in which information on product specifications not described is stored. The user 331 obtains the product specification information of the item 2 and the item 3 of the target device and target sensor to be controlled through the host device 332 from the product specification cloud server.

Next, when the user 331 executes the control contents of the items 2 and 3 through the host device 332, the user 331 determines item 4 = “phrase”, which is a word uttered to the host device 332. This “phrase” is preferably the content corresponding to the command of the air conditioner 1 listed in item 3, for example, “turn on command” for “start command” for operating the air conditioner, “stop command for stopping the air conditioner” ”For air conditioning”, change the air conditioner operation “cooling” to “dry” “operation change command” for “change to dry”, air conditioner setting content for operation start time “Setting change command” for changing “to start operation at 10 o'clock” describes “turn on air conditioner at 10 o'clock” as an example.

The user 331 creates a combination (target, command, phrase) determined as described above as an initial setting of the host device 332. The user 331 performs the same creation for all the devices to be controlled through the host device 332, and finally generates a response / action information list in which all the devices to be controlled (objects, commands, phrases) are combined into one. To do. The created response / action information list is registered in the response / action generation unit 102-2 through the host device 332.

When the response / action information list is registered in the response / action generation unit 102-2, as shown in FIGS. 10A and 10B, the host device 332 continues the words that the user 331 issues after the recognition of the reserved words is completed. By taking in and analyzing, it is possible to control devices and sensors.

For example, in the case where the word uttered by the user 331 = turn on the air conditioner, the speech text conversion unit 101-2 converts the input speech data into the text “Eat Kontetsu”, and the text analysis unit 102-1 , Analyzes that the text data “Eakontsute” is the content of “Turn on the air conditioner”. Based on the analysis result, the response / action generating unit 102-2 refers to the list of registered response / action information, and the response / action information corresponding to the analysis result of “phrase” “turn on air conditioner” Search for. As a result, the response / action information (target = air conditioner 1, command = operation start) is extracted, and the response / action information of (target = air conditioner 1, command = operation start) is set in the response / action generation result notification. To the voice processing unit 407.

The audio processing unit 407 refers to the response / action information set in the received response / action generation result notification, and selects the corresponding device or sensor from among the various sensors 310, the various equipment 320, and the various home appliances 340. Control.

Next, in the case of controlling a device or sensor using the host device 332, a description will be given of a case where the control content for controlling the device or sensor under various conditions or the operation content of the host device 332 is changed.

FIG. 14 shows that when a plurality of reserved words are registered in the host device 332, the host device 332 recognizes the word uttered by the user 331 as one of the reserved words, and performs in accordance with the recognized reserved word. It is a list of examples of contents.

The host device 332 can register a plurality of reserved words, and when each of the plurality of reserved words is recognized, the operation content corresponding to the recognized reserved word (hereinafter referred to as additional information 1). Can be set.

As shown in FIG. 14, it is assumed that the host device 332 has registered three reserved words, for example, “Iroha”, “I like you”, and “Sonya”. When the host device 332 recognizes the word uttered by the user 331 as the reserved word “Iroha”, the operation content already set is not changed, but the word uttered by the user 331 is the reserved word “I am like”. In the case of recognition, the operation is changed so as to be surely announced through the speaker 423 that “the master is pleased” when the words uttered by the user 331 are recognized thereafter. In addition, when the user 331 recognizes the reserved word “son” as the reserved word, the host device 332 determines that the user 331 is a senior user. The setting is changed so that the expiration time T0 of the input interval confirmation timer shown in FIGS. 11A and 11B is longer than the normal setting time.

The example of FIG. 14 shows an example in which the host device 332 changes the operation content of the host device itself. However, the present invention is not limited to this, and controls the operation of devices and sensors connected to the host device 332 via the network 333. May be.

The host device 332 needs to register the additional information 1 for each reserved word in the host device 332 in advance in order to change the operation of the host device 332 according to a plurality of reserved words.

When registering a reserved word in the host device 332, the host device 332 has a mode for registering additional information 1 corresponding to the reserved word to be registered (hereinafter referred to as a reserved word registration (additional information 1) mode). ing.

FIG. 15A and FIG. 15B show the case where the reserved word and the additional information 1 corresponding to the reserved word are registered in the state where the host device 332 is in the “reserved word registration (additional information 1) mode”. An example of a processing sequence of the host device 332 from registration start to registration completion of additional information 1 is shown. The processing from S1500 to S1514 shown in FIGS. 15A and 15B is the same as the processing from S500 to S514 shown in FIGS. 5A and 5B, respectively. 15A and 15B are different from FIG. 5A and FIG. 5B in that S1515 is different from S515 and that S1516 to S1523 are added.

The trigger setting unit 403 displays (S1514) informing the user 331 that registration of the reserved word has been completed. The display informing the user that registration of the reserved word has been completed (S1515) is performed by the trigger setting unit 403 transmitting a registration completion notification to the display device 425 (S1514) and receiving the registration completion notification. It is desirable to use a display method that can be recognized by the user 331, for example, 425 causes the LED to blink green. Accordingly, the trigger setting unit 403 can prompt the user 331 to register the additional information 1.

The user 331 who recognizes that the LED is blinking in green (S1515) can set the additional information 1 corresponding to the reserved word registered in S1511.

The setting method of the additional information 1 may be such that the host device 332 captures the voice uttered by the user 331 through the microphone 421, and can be registered by analyzing the captured voice data. Alternatively, a menu for setting the additional information 1 may be displayed on the display device 425 so that the user 331 can perform registration in accordance with the menu. Alternatively, using an external device connected via the network I / F 427 shown in FIG. 4, such as a smartphone or tablet, a menu for setting additional information 1 corresponding to the reserved word on the display screen of the smartphone or tablet is displayed. The user 331 may be registered by operating in accordance with the displayed menu screen. FIGS. 15A and 15B show an example of a processing sequence in the case where a menu for setting additional information 1 displayed on the display unit 425 is displayed and the user 331 operates according to the menu to register the additional information 1.

When the LED blinks green to prompt the user 331 to input the additional information 1 (S1515), a menu for registering the additional information 1 is displayed on the display unit 425. The user 331 creates additional information 1 by operating according to the displayed menu screen. The additional information 1 that has been created is taken into the input management unit 420 (S1517). The input management unit 420 transfers the acquired additional information 1 to the trigger setting unit 403. The trigger setting unit 403 stores the transferred additional information 1 in the reserved word storage area 410-2 of the memory 410 (S1519).

The trigger setting unit 403 stores the additional information 1 in association with the reserved word registered in S1513 when storing the additional information 1 in the reserved word storage area 410-2 of the memory 410.

Also, the voice processing unit 407 performs a display (S1522) informing the user 331 that the registration of the additional information 1 has been completed. The display informing the user 331 that the registration of the additional information 1 has been completed (S1522), the voice processing unit 407 transmits a registration completion notification to the display device 425 (S1520), and the registration completion notification has been received. It is desirable that the display device 425 perform the display method that the user 331 can recognize, for example, the LED is lit in green.

16A and 16B show the recognition of reserved words among the words uttered by the user 331 when the additional information 1 is stored in the reserved word storage area 410-2 of the memory 410 by the processing shown in FIGS. 15A and 15B. In this example, the recognized additional information 1 of the reserved word is read from the reserved word storage area 410-2 and an operation is set for the host device 332.

The processing from S1600 to S1612 shown in FIGS. 16A and 16B is the same as the processing from S800 to S812 shown in FIGS. 8A and 8B, respectively. The difference between the processing of FIGS. 16A and 16B and the processing of FIGS. 8A and 8B is that the processing of S1613 and S1614 is added.

When the word uttered by the user 331 is recognized as a reserved word (S1605), the trigger recognizing unit 405 reads the additional information 1 corresponding to the corresponding reserved word from the reserved word storage area 410-2 of the memory 410. The trigger recognition unit 405 that has read the additional information 1 sets the operation of the content of the read additional information 1 (S1613) in the host device 332 (S1614). When the contents of the example shown in FIG. 14 are stored in the reserved word storage area 410-2, when “Son” is recognized as a reserved word in S1605, the trigger recognizing unit 405 determines the input interval in S1614. The expiration time T0 of the confirmation timer T is set to make the normal value longer.

FIG. 17A shows that when a word uttered by the user 331 is recognized as a reserved word registered in the host device 332, the host device 331 responds to the word uttered by the user 331 following the recognized reserved word. 332 is a list of examples of operation contents for performing a specific operation.

When the host device 332 recognizes that the words uttered by the user 331 are registered reserved words, the contents of the words uttered by the user 331 (hereinafter referred to as additional words) following the recognized reserved words. The operation content (hereinafter referred to as additional information 2) can be set in accordance with.

For example, as shown in FIG. 17A, it is assumed that “Iroha” is registered as a reserved word. In this case, when the host device 332 recognizes the reserved word “Iroha” and does not recognize the word issued by the user 331 following the reserved word “Iroha”, the operation content that has already been set is not changed. When the host device 332 recognizes “chan” as a word uttered by the user 331 following the reserved word “Iroha”, the host device 332 determines that the user 331 is in good mood, and when responding through the speaker 423, Change the operation to raise the tone. Further, when the host device 332 recognizes “ya” as a word uttered by the user 331 following the reserved word “Iroha”, the host device 332 estimates that the user 331 is a senior user, and the user 331 tends to speak slowly. Therefore, the expiration time T0 of the input interval confirmation timer shown in FIGS. 11A and 11B is changed to be longer than the normal set time. When the host device 332 recognizes “oi” as a word uttered by the user 331 following the reserved word “Iroha”, the host device 332 determines that the user 331 is angry and immediately transmits “sorry” through the speaker 423. Try to announce.

In the example of FIG. 17A, the host device 332 sets the additional information 2 by setting a plurality of additional words for one reserved word and setting the additional information 2 for each combination of the plurality of additional words for the reserved word. Although the example in which the operation content is changed based on the content of the additional information 2 is shown, it is also possible to set the additional information 2 for each combination of a plurality of reserved words and a plurality of additional words. As shown in FIG. 17B, for example, it is assumed that the host device 332 has registered “Iroha”, “Ookini”, and “Ashindo” as reserved words. In this case, an additional word may be defined for each reserved word, and additional information 2 may be set for each combination of the reserved word and the additional word.

Also, some users may want a specific action just by issuing a reserved word. For example, if there is a phrase of a certain individual, the phrase is registered in the host device 332 as a reserved word, and an operation corresponding to the reserved word is registered in the host device 332 as well. And control of the operation of the sensor can be executed easily. In the example of the reserved word “Ashindo” in FIG. 17B, when the host device 332 recognizes the reserved word “Ashindo”, the host device 332 recognizes the reserved word from the words uttered by the user 331. It is also possible to announce the beer information stored in the refrigerator connected to the network 333 through the speaker 423.

In order to change the operation according to the content of the additional word for the reserved word, the host device 332 uses a combination of the additional word corresponding to the reserved word and the additional information 2 that is the operation content for the combination of the reserved word and the additional word. It is necessary to register with the host device 332 in advance. Therefore, the host device 332 has a mode for additionally registering corresponding additional words and additional information with respect to the registered reserved words. A mode for adding additional information 1 to a reserved word already registered in the host device 332 is called an additional information 1 additional registration mode, and a mode for adding additional words and additional information 2 is called an additional information 2 additional registration mode. And

The setting method of the additional information 2 may be configured such that, similar to the setting of the additional information 1, the voice uttered by the user 331 is captured by the host device 332 through the microphone 421, and the captured voice data is analyzed to be registered. Alternatively, a menu for setting the additional information 2 may be displayed on the display device 425 so that the user 331 can perform registration in accordance with the displayed menu. Alternatively, by using an external device connected via the network I / F 427 shown in FIG. 4, for example, a smartphone or tablet, additional information 2 corresponding to the reserved word and the additional word is set on the display screen of the smartphone or tablet. A menu to be displayed may be displayed, and the user 331 may perform registration according to an operation according to the displayed menu screen.

18A, 18B, and 18C show a case where an additional word is registered and an operation content (additional information 2) is registered for the registered reserved word shown in FIGS. 17A and 17B. It is an example of a processing sequence.

In order to additionally register additional words for registered reserved words, the user 331 changes the host device 332 to “additional information 2 additional registration mode”. When the host device is changed to the “additional information 2 additional registration mode”, the user 331 issues a reserved word registered in the host device 332 and an additional word to be registered for the reserved word. The host device 332 first recognizes a reserved word from the words uttered by the user 331 (S1805).

The host device 332 captures the words uttered by the user 331 into the input management unit 420 through the microphone 421 (S1801). When the input management unit 420 captures the voice data, the input management unit 420 changes the internal state managed internally to being recognized (reserved word) (S1802), and transfers the input voice data to the trigger recognition unit 405 (S1803).

Upon receiving the voice data transferred from the input management unit 420, the trigger recognition unit 405 reads the recognition data from the reserved word storage area 410-2 of the memory 410 (S1804), and has been transferred from the input management unit 420. Comparison with audio data is performed (S1805). If the input voice data can be recognized as a reserved word, the trigger recognition unit 405 notifies the input management unit 420 of a recognition completion notification (S1806). Receiving the recognition completion notification, the input management unit 420 changes the internal state managed internally from being recognized (reserved word) to waiting for input (additional word) (S1807).

The host device 332 takes in a word issued by the user 331 following the reserved word into the input management unit 420 through the microphone 421 (S1808). Since the internal state managed internally is waiting for input (additional word) (S1807), the input management unit 420 transfers the input voice data to the trigger setting unit 403 (S1809). Thereafter, like the reserved word registration described with reference to FIGS. 5A and 5B, the trigger setting unit 403 saves the received voice data in the voice storage area 410-3 of the memory 410 (S1810), while adding the specified number of additional words. Is taken in (S1811).

If the trigger setting unit 403 determines that the specified number has not been reached as a result of checking whether the specified number has been reached, the trigger setting unit 403 displays a message prompting the user 331 to input the additional word to be registered (S1812) and input management. The input continuation notification is transmitted to the unit 420 (S1814). In addition, the display that prompts the user 331 to input a voice to be registered as an additional word (S1813), the trigger setting unit 403 transmits a registration incomplete notification to the display device 425 (S1812), and the registration incomplete notification. It is desirable that the display device 425 that has received the message blinks the LED in red, for example, so that the user 331 can recognize the display method. Further, the user 331 may be prompted to input a voice to be registered by using a voice method instead of the display method. In this case, the trigger setting unit 403 transmits a registration incomplete notification to the speaker 423, and the speaker 423 that has received the registration incomplete notification announces to the user 331, for example, “Please input again”. But you can. Alternatively, the trigger setting unit 403 may use both a display method and a voice method to prompt the user 331 to input a voice to be registered.

If the trigger setting unit 403 determines that the specified number has been reached as a result of checking whether the specified number has been reached, the trigger setting unit 403 reads the voice data stored in the voice storage area 410-3 until then (S1815), 2 is sent to the recognition data conversion unit 101-1 in the voice recognition cloud 101 in the cloud server 1 (S1816).

The recognition data conversion unit 101-1 converts the voice data sent from the trigger setting unit 403 into recognition data for recognizing additional words (S1817). When the conversion to the recognition data is completed, the recognition data conversion unit 101-1 sends the recognition data to the trigger setting unit 403 through the Internet 2 (S1818). Upon receiving the recognition data for recognizing the additional word (hereinafter referred to as recognition data (additional word)), the trigger setting unit 403 stores the received data in the reserved word storage area 410-2 of the memory 410 (S1819). ). When saving the recognition data (additional word), the trigger setting unit 403 saves it in association with the reserved word recognized in S1806. As a result, the recognition data (additional word) can be stored in association with the reserved word recognized in S1806.

Also, the trigger setting unit 403 performs a display (S1822) informing the user 331 that the registration of additional words has been completed. A display that informs the user 331 that the registration of the reserved word has been completed (S1822) is a display in which the trigger setting unit 403 transmits a registration completion notification to the display device 425 (S1821) and receives the registration completion notification. It is desirable to use a display method that the user 331 can recognize, such as the device 425 blinking the LED in green (S1822). Alternatively, the trigger setting unit 403 may use a voice method instead of the display method to notify the user 331 that registration of the reserved word has been completed. In this case, the trigger setting unit 403 transmits a registration completion notification to the speaker 423 (S1821), and the speaker 423 that has received the registration completion notification announces to the user 331, for example, “registration is complete”. But you can. Alternatively, the trigger setting unit 403 may use both a display method and a voice method to notify the user 331 that registration of the reserved word is completed. Thereby, the user 331 can know the timing at which the content of the additional information 2 which is the operation content corresponding to the additional word is uttered.

When the LED blinks green to prompt the user 331 to input the additional information 2 (S1822), a menu for registering the additional information 2 is displayed on the display unit 425. The user 331 creates additional information 2 by operating according to the displayed menu screen. The creation is completed and the additional information 2 is taken into the input management unit 420 (S1824). The input management unit 420 transfers the acquired additional information 2 to the trigger setting unit 403 (S1825). The trigger setting unit 403 stores the transferred additional information 2 in the reserved word storage area 410-2 of the memory 410 (S1826).

The trigger setting unit 403 stores the additional information 2 in association with the reserved word recognized in S1806 when storing the additional information 2 in the reserved word storage area 410-2 of the memory 410. As a result, the operation content (additional information 2) associated with the reserved word recognized in S1806 and associated with the additional word saved in S1819 can be saved.

It is also possible to add only additional information to registered reserved words later.

18D and 18E are examples of processing sequences in the case where only additional information is added to a registered reserved word, unlike FIGS. 18A, 18B, and 18C.

The processing from S1850 to S1856 shown in FIG. 18D is the same as the processing from S1800 to S1806 shown in FIG. 18A, respectively. Also, the processing from S1871 to S1880 shown in FIGS. 18D and 18E is the same as the processing from S1821 to S1830 shown in FIG. 18C, respectively. The difference between the sequence example of FIGS. 18A, 18B, and 18C and the sequence example of FIGS. 18D and 18E is that the processing corresponding to the additional word registration processing from S1807 to S1820 in FIGS. 18A, 18B, and 18C There is no point in 18D and FIG. 18E.

When the LED blinks green to prompt the user 331 to input the additional information 1 (S1871), a menu for registering the additional information 1 is displayed on the display unit 425. The user 331 creates additional information 1 by operating according to the displayed menu screen. The creation is completed and the additional information 1 is taken into the input management unit 420 (S1874). The input management unit 420 transfers the acquired additional information 1 to the trigger setting unit 403 (S1875). The trigger setting unit 403 stores the transferred additional information 1 in the reserved word storage area 410-2 of the memory 410 (S1876).

The trigger setting unit 403 stores the additional information 1 in association with the reserved word recognized in S1856 when storing the additional information 1 in the reserved word storage area 410-2 of the memory 410. As a result, the operation content associated with the reserved word recognized in S1856 can be saved.

19A and 19B show the words uttered by the user 331 when the additional word and the additional information 2 are stored in the reserved word storage area 410-2 of the memory 410 by the processing shown in FIGS. 18A, 18B, and 18C. When a reserved word and an additional word are recognized from among them, additional information 2 corresponding to the recognized reserved word and additional word combination is read from the reserved word storage area 410-2, and an operation is set for the host device 332 This is a sequence example.

The processing from S1900 to S1908 shown in FIG. 19A is the same as the processing from S1600 to S1608 shown in FIG. 16A, respectively. 19A and 19B differs from the processing in FIGS. 16A and 16B in that additional word recognition processing from S1909 to S1911 is added, and additional information 2 reading processing from S1912 to S1913. It is a point to do.

If the reserved word is successfully recognized in S1905 of FIG. 19A for the data captured by the user 311, the trigger recognition unit 405 successfully recognizes the data captured by the user 311. Comparison with recognition data (additional word) read from the reserved word storage area 410-2 of the memory 410 in order to determine whether the voice data continuously input to the reserved word is an additional word (S1911). When the voice data continuing to the reserved word is recognized as an additional word, the trigger recognition unit 405 reads the corresponding reserved word and additional information 2 corresponding to the additional word from the reserved word storage area 410-2 of the memory 410 ( S1912). The trigger recognition unit 405 that has read the additional information 2 sets the operation of the content of the read additional information 2 in the host device 332 (S1913).

As described above, by registering reserved words, additional words, and additional information in the host device 332, the host device 332 operates the host device 332, and operates on devices and sensors connected to the host device 332 through the network. Can be controlled freely, and it is possible to control devices and sensors suitable for the individual lifestyle.

FIG. 20 shows that when a plurality of reserved words are registered in the host device 332, when the user 331 recognizes any one of the reserved words, the voice recognition cloud according to the recognized reserved word. 10 is a list of examples of changing a speech recognition dictionary used in the speech text conversion unit 101-2 of 101.

The host device 332 can register a plurality of reserved words. When the host device 332 recognizes that the word uttered by the user 331 is one of a plurality of registered reserved words, the voice text conversion unit 101-2 of the speech recognition cloud 101 according to the recognized reserved word. It is possible to change a speech recognition dictionary for converting voice to text used in the process. For example, as shown in FIGS. 21A and 21B, the host apparatus 332 is assumed to have registered three "Hello", "Hello", "Ookini" as reserved words. In this case the host device 332, if you recognize the "Hello" reserved word, it is possible to issue an instruction to change the voice recognition dictionary used in speech-to-text conversion unit 101-2 of the speech recognition cloud 101 to Japanese dictionary . If the reserved word “Hello” is recognized, the host device 332 instructs the speech text conversion unit 101-2 of the speech recognition cloud 101 to change the type of the speech recognition dictionary to the English dictionary. be able to. Furthermore, when the reserved word “OOKINI” is recognized, the host device 332 changes the type of the speech recognition dictionary used in the speech text conversion unit 101-2 of the speech recognition cloud 101 to a dialect dictionary (Kansai dialect). An order can be issued.

In order to change the type of the speech recognition dictionary used in the speech text conversion unit 101-2 of the speech recognition cloud 101 according to the reserved word recognized by the host device 332, the user 331 registers the reserved word in the host device 332 In this case, it is necessary to register the type of speech recognition dictionary (hereinafter referred to as additional information 3) used in the speech text conversion unit 101-2 corresponding to the reserved word.

The processing sequence for registering the type (additional information 3) of the speech recognition dictionary corresponding to the reserved word together with the registration of the reserved word is a processing sequence for registering additional information 1 for the reserved word shown in FIGS. 15A and 15B. The additional information 1 may be selected and input instead of inputting the additional information 1 on the menu screen displayed on the display unit 425 (S1516). Hereinafter, the flow of the process of registering the additional information 3 will be described using the processes after S1514 in FIG. 15B. Additional information 1 described after S1514 in FIG. 15B will be described as additional information 3.

When the LED blinks green to prompt the user 331 to input the additional information 3 (S1514), a menu for registering the additional information 3 is displayed on the display unit 425. The user 331 can select a type of dictionary as the additional information 3 by performing an input operation of the additional information 3 in accordance with the displayed menu screen. The creation is completed and the additional information 3 is taken into the input management unit 420 (S1516). The input management unit 420 transfers the acquired additional information 3 to the trigger setting unit 403. The trigger setting unit 403 stores the transferred additional information 3 in the reserved word storage area 410-2 of the memory 410.

The trigger setting unit 403 stores the additional information 3 in association with the reserved word registered in S1513 when storing the additional information 3 in the reserved word storage area 410-2 of the memory 410.

21A and 21B show the case where each reserved word is recognized by the host device 332 when a plurality of reserved words are registered in the host device 332 as shown in FIG. The sequence example which changes the kind of speech recognition dictionary to be used is shown. The processing from S2100 to S2113 shown in FIGS. 21A and 21B is the same as the processing from S1600 to S1613 shown in FIGS. 16A and 16B, respectively. The difference between the processing in FIGS. 21A and 21B and the processing in FIGS. 16A and 16B is that, in the case of the processing in FIGS. 16A and 16B, after the trigger recognition unit 405 reads out the additional information 1, While the operation of the host device 332 is set based on the contents (S1614), in the case of FIGS. 21A and 21B, after the trigger recognition unit 405 reads the additional information 3, the contents of the additional information 3 are changed. Based on this, in order to change the type of the speech recognition dictionary used in the speech text conversion unit 101-2, exchange with the speech text conversion unit 101-2 is performed (S2114-1 to S2114-3).

The trigger setting unit 403 transmits a registration completion notification to the display device 425 (S2109) to notify the user that the reserved word recognition and the change of the voice recognition dictionary have been completed. It is desirable that the display device 425 that has received the signal illuminates the LED in green, for example, so that the user 331 can recognize the display method. Alternatively, the trigger recognition unit 405 sends a recognition completion notification to the speaker 423, and the speaker 423 that has received the recognition completion notification is, for example, “Yes? What is the voice recognition dictionary? It may be a method of announcing to the user 331 by voice. Alternatively, the trigger recognizing unit 405 notifies the user 331 that the reserved word has been recognized and the speech recognition dictionary corresponding to the recognized reserved word has been changed. Both voice methods using 423 may be used.

The operation content corresponding to the reserved word shown in FIG. 14 (additional information 1), the operation content for each additional word (additional information 2) shown in FIGS. 17A and 17B, and the reservation shown in FIG. The types of voice recognition dictionaries (additional information 3) for words can be registered in combination.

FIG. 22 shows the registration of the operation content corresponding to the reserved word shown in FIG. 14, the registration of the additional word for the reserved word shown in FIG. 17A, the registration of the operation content for the additional word, and the voice recognition for the reserved word shown in FIG. It is a list of combinations when registering dictionary types in combination. The host device 332 is, for example, a reserved word for the "Hello" is configured to use the Japanese dictionary as the type of voice recognition dictionary. The host device 332, also "Chan", "Ya" as an additional language to the reserved word "Hello" to register the "Hey", the host device so as to raise the tone at the time of response in the case of the additional word "chan" When the additional word is “Ya”, the setting content is changed to increase the expiration time T0 of the input interval confirmation timer T. When the additional word is “O”, “Sorry, There is no action "so that it will be announced immediately.

FIG. 23 is a list of examples in which the type of the speech recognition dictionary used in the text conversion unit 101-2 is changed according to contents other than reserved words (hereinafter referred to as change conditions). For example, FIG. 23A shows an example in which time is set as the change condition. The host device 332 instructs the text conversion unit 101-2 of the voice recognition cloud 101 to change the type of the voice recognition dictionary used when the voice data is converted into text depending on the time for which the voice recognition dictionary is used. An example is shown.

For example, the host device 332 uses the family general dictionary from time 05:00 to 08:00, uses the wife dictionary from time 08:00 to 16:00, and time 16:00 to 20:00. The text conversion unit 101-2 is instructed through the Internet 2 to use the family general dictionary until the time 20:00 to 05:00 and use the adult dictionary.

FIG. 23B shows an example in which the change condition = the operation status of the host device 332 is set. The host device 332 can instruct to change the type of the speech recognition dictionary used by the text conversion unit 101-2 depending on the type of operation status of the host device 332 when using the speech recognition dictionary.

The host device 332 uses, for example, a time / route search dictionary when the operation status = from now on, uses a general dictionary when the operation status = goes out, and uses a refresh dictionary when the operation status = night mode. Thus, the text conversion unit 101-2 is instructed through the Internet 2.

The host device 332 has a mode for registering change condition type information (hereinafter referred to as a change condition registration mode) or later, which is information on the type of speech recognition dictionary used in accordance with conditions.

The user 331 needs to register the change condition type information in the host device 332 in advance in order to use the type of the voice recognition dictionary according to the change condition.

The registration method for properly using different types of voice recognition dictionaries according to the change condition is to allow the host device 332 to capture the voice uttered by the user 331 through the microphone 421 and analyze the captured voice data so that registration can be performed. Also good. Alternatively, a menu for setting the additional information 1 may be displayed on the display device 425 so that the user 331 can perform registration in accordance with the menu. Alternatively, using an external device connected via the network I / F 427 shown in FIG. 4, for example, a smartphone or tablet, a menu for setting additional information 1 as a reserved word is displayed on the display screen of the smartphone or tablet. The user 331 may perform registration according to an operation according to the displayed menu screen.

FIG. 24 displays a menu for setting change condition type information displayed on the display unit 425, and processing when the user 331 operates according to the menu to register the type of the voice recognition dictionary to be used depending on the change condition. It is an example of a sequence. The processing from S2417 to S2423 shown in FIG. 24 is the same as the processing from S1517 to S1523 in FIG.

The user 331 inputs the type of voice recognition dictionary to be used according to the change condition by operating according to the displayed menu screen. The change condition type information for which input has been completed is taken into the input management unit 420 (S2417). The input management unit 420 transfers the captured change condition type information to the trigger setting unit 403 (S2418). The trigger setting unit 403 stores the transferred change condition type information in the reserved word storage area 410-2 of the memory 410 (S2419).

FIG. 25 shows the case where the change condition type information for changing the type of the speech recognition dictionary according to the change condition is stored in the reserved word storage area 410-2 of the memory 410 as shown in FIG. This is an example of a processing sequence in the case where the host device 332 notifies the voice text conversion unit 101-2 of the change of the voice recognition dictionary according to the content of the change condition type information that has been changed.

25 is preferably performed continuously after the reserved word recognition process shown in FIG. 9B is completed (S911). Alternatively, after the reserved word is recognized, when the user 331 utters the host device 332 to control the device or sensor as shown in FIGS. 10A and 10B, the host device 332 captures the word. It is desirable to carry out at the timing (S1001).

FIG. 25 shows a timing (S1001) when the host device 332 fetches the words when the user 331 utters words to the host device 332 to control the devices and sensors as shown in FIGS. 10A and 10B. It is an example in the case of performing determination of change of the speech recognition dictionary and notification of the result.

When the recognition of the reserved word is completed, the host device 332 continuously captures the voice uttered by the user into the input management unit 420 through the microphone 421 (S2501). The input management unit 420 transmits a read request (change condition type information) to the audio processing unit 407 in order to read the change condition type information at the timing when the audio data is acquired (S2502). Pause. Receiving the read request (change condition type information), the voice processing unit 407 reads the change condition type information including the combination of the change condition and the type of the voice recognition dictionary from the reserved word storage area 410-2 of the memory 410. (S2503). The voice processing unit 407 analyzes the “change condition” of the read change condition type information, and determines whether or not the content is compatible with the state of the host device 332 (S2504). If it is determined as a result of the determination, the speech processing unit 407 reads “the type of the speech recognition dictionary” corresponding to the “change condition”, and the type of the speech recognition dictionary after the change by the speech recognition dictionary type notification. Is sent to the voice text conversion unit 101-2 through the Internet 2 (2505). Upon receiving the voice recognition dictionary type notification, the voice text conversion unit 101-2 refers to the notified voice recognition dictionary type and changes the type of the voice recognition dictionary currently in use to the notified voice recognition dictionary type. (S2506)
When the change of the type of the voice recognition dictionary is completed, the voice text conversion unit 101-2 notifies the voice processing unit 407 of the voice recognition dictionary change completion notification as the change completion notification (S2507).

Upon receiving the voice recognition dictionary change completion notification (S2507), the voice processing unit 407 transmits a read completion notification to the input management unit 420 as a notification that the change condition type information has been read (S2508). . When the input management unit 420 receives the read completion notification (S2508), the input management unit 420 resumes the processing for the audio data captured in S2501.

The user 331 may forget the reserved word registered in the host device 332. In preparation for such a case, it is desirable that the user 331 can confirm a registered reserved word by a simple method.

FIG. 26 shows a case where the user 331 who registered a reserved word in the example of the processing sequence shown in FIGS. 5A and 5B forgets a registered reserved word, and a part or all of the registered reserved word is changed to the user 331. A list of examples of reserved words (hereinafter referred to as “reserved reserved words”) and display contents (display range) for notifying the user is shown. For example, for a reserved word “I don't know”, all of the reserved words registered in the host device 332 are displayed on the display unit 425 or displayed in the display area of an external device connected to the host device 332. Is shown. In addition, for a reserved word of “tell me”, a predetermined part of reserved words registered in the host device 332 is displayed on the display unit 425 or an external device connected to the host device 332. It shows the case of displaying in the display area. For a reserved word “unused”, a reserved word that has not been used in the past year among the reserved words registered in the host device 332 is displayed on the display unit 425 or connected to the host device 332. This shows the case where the data is displayed in the display area of the external device. The external device connected to the host device 332 is preferably a device that has a relatively large display screen, such as a smartphone, tablet, or liquid crystal television, and allows the user to refer to many reserved words at once.

As described above, the registration of reserved words for displaying registered reserved words is performed by changing the mode of the host device to the setting mode (reserved words (for display)) and changing the reserved words shown in FIGS. 5A and 5B. Registration may be performed according to a registration processing sequence.

The above example is an example in which the corresponding reserved word is displayed immediately when the user issues the “reserved reserved word” shown in FIG. However, the host device 332 may ask the user 331 for a secret before displaying the corresponding reserved word. After the user issues a “reserve reserved word”, the host device 332 emits a voice such as “mountain” through the speaker 423, and when the user 331 responds with “river”, for example, the corresponding reserved word is displayed. It may be displayed.

Further, the host device 332 can capture or record a scene in which a reserved word, an additional word, or additional information is registered by taking in a word uttered by the user 331. Alternatively, when a reserved word or additional word is recognized, the recognized scene can be recorded or recorded.

FIG. 27 shows a case where the host device 332 captures a word uttered by the user 331 and records or records a reserved word, additional word, or additional information registration, reserved word or additional word recognition scene. A functional block diagram of 332 is shown. The difference from FIG. 4 is that the host device 2700 has a camera 2702 for recording a scene for registering a reserved word, additional word, or additional information, or for recording a scene for recognizing a reserved word or additional word. The control management unit 2701 has EVT-Mg2701-3 in addition to APP-Mg2701-1 and CONF-Mg2701-2, and has a playback control function for playing back the recorded or recorded scene data. It is a point. EVT-Mg2701-3 is a function for recording or recording, which will be described later, due to the occurrence of a scene that registers reserved words, additional words, or additional information, and the occurrence of scenes that recognize reserved words, additional words have. Hereinafter, the host device 332 captures a word uttered by the user 331, and records or records a scene for registering a reserved word, additional word, or additional information, and a scene for recognizing the reserved word or additional word. A recording process or a recording process flow will be described.

FIG. 28 shows a registered scene or a recognition scene when a scene for registering a reserved word, additional word, or additional information occurs, or when a scene for recognizing a reserved word, additional word is generated. Reference numeral 332 indicates the passage of time when recording or recording.

At time t1, it is assumed that the host device 332 starts registration using the words uttered by the user as reserved words. The start of reserved word registration may be, for example, the timing at which the input management unit 420 performs the processing of S502 in the reserved word registration sequence of FIGS. 5A and 5B. When recognizing the start of reserved word registration, the input management unit 420 notifies the EVT-Mg 2701-3 to that effect. The EVT-Mg2701-3 that has received the notification that the reserved word registration is started records the reserved word registration scene as Rec1 through the microphone 421, or records the reserved word registration scene as Rec1 through the camera 2702. The end of the reserved word registration may be, for example, the timing when the input management unit 420 receives the registration completion notification of S512 in the reserved word registration sequence of FIGS. 5A and 5B. The input management unit 420 that grasps the end of the registration of the reserved word notifies the EVT-Mg 2701-3 to that effect. The EVT-Mg2701- 3 that has received the reserved word registration completion finishes recording the reserved word registration scene that was performed through the microphone 421, or ends recording of the reserved word registration scene that was performed through the camera 2702 Let

Similarly, it is assumed that at time t2, the host device 332 starts recognizing a word uttered by the user as a reserved word. The start of reserved word recognition may be, for example, the timing at which the input management unit 420 performs the processing of S802 in the reserved word recognition sequence of FIGS. 8A and 8B. When the input management unit 420 recognizes the start of recognition of the reserved word, it notifies the EVT-Mg 2701-3 to that effect. The EVT-Mg2701-3 that has received the notification of the start of reserved word recognition records the reserved word recognition scene as Rec2 through the microphone 421, or records the reserved word recognition scene as Rec2 through the camera 2702. The end of the recognition of the reserved word may be the timing when the input management unit 420 receives the recognition completion notification in S811 in the reserved word registration sequence of FIGS. 8A and 8B, for example. The input management unit 420 that grasps the end of the registration of the reserved word notifies the EVT-Mg 2701-3 to that effect. The EVT-Mg2701- 3 that has received the reserved word registration completion terminates the recording of the reserved word recognition scene performed through the microphone 421 or terminates the recording of the reserved word recognition scene performed through the camera 2702. Let

Similarly, record or record registration or recognition events that occurred at t3 and t4.

The host device 332 can play back a recorded scene or a recorded scene that has been recorded.

FIG. 29 shows an example of a state in which data to be reproduced is displayed when each data of a recorded or recorded scene is reproduced. In the example of FIG. 29, four icons of data to be reproduced are displayed in a form corresponding to the state of occurrence of events with respect to the time axis of FIG. The icon display of the data to be reproduced may be displayed on the display unit 425, for example. Or you may display on the external device connected to the host apparatus 332, for example, a smart phone, a tablet, a liquid crystal television.

The displayed icon represents the date and time of recording or recording and the content of the data to be recorded or recorded. For example, if the displayed content of the icon is reserved word registration “OOKINI”, it indicates that the content of the recorded or recorded data is a scene in which “OOKINI” is registered as a reserved word. Similarly, when the display content of the icon is reserved word recognition “OOKINI”, it indicates that the content of the recorded or recorded data is a scene in which “OOKINI” is recognized as a reserved word.

The user 331 can confirm the recording of the target data or the recorded contents by selecting the icon of the data to be reproduced.

Furthermore, the host device 332 issues instructions to the cameras and microphones connected via the network 333, and when a scene for registering reserved words, additional words, or additional information is generated by these cameras or microphones, or When a scene for recognizing reserved words and additional words occurs, a registered scene or a recognition scene may be recorded or recorded.

As described above, the host device 332 recognizes a reserved word from words uttered by the user 331, and based on the content of the additional information corresponding to the reserved word, a device or sensor connected via the network. Can be controlled. The control contents of the target devices and sensors may require high security. For example, it is assumed that a reserved word in which the opening / closing operation of the safe door is set as additional information is registered in the host device 332 so that the opening / closing control of the safe door can be performed using the host device. In this case, when the host device 332 recognizes the corresponding reserved word, the host device 332 opens and closes the door of the safe and records or records the periphery of the safe as a control target device using a microphone and a camera around the safe. By recording, it is possible to keep the security of the opening and closing operation of the safe door. The user 331 confirms the contents of data recorded or recorded using a microphone or camera connected to the network, as well as data recorded or recorded using a microphone or camera built in the host device 332. I can do it. When the control content of the device or sensor to be controlled by the host device 332 requires high security, the host device 332 further includes a microphone or a device around the control target device or sensor before the control content is executed. The validity of the person who produced the recorded voice or the person of the recorded video may be confirmed using the voice recorded using the camera or the recorded video. The host device 332 uses a pre-registered feature point such as a voice or face of a specific person and a microphone or camera around the device or sensor to be controlled before executing the control content in the specific additional information. The control content may be executed only when the sound of the collected sound or the captured video is compared and the validity of the corresponding person is confirmed.

In the description of the above embodiment, the recognition data conversion unit 101-1, the speech text conversion unit 101-2, the text analysis unit 102-1 and the response / action generation unit 102-2 are all included in the cloud server 1. Although described as existing, some or all of these may exist in the host device 332. Also in this case, the example of the operation sequence of each process already described is the same as that already described.

Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 ... Cloud server, 2 ... Internet, 3 ... Home, 101 ... Voice recognition cloud, 102 ... Corresponding action generation cloud, 310 ... Various sensors, 320 ... Various equipment , 330 ... HGW (HomeGateWay), 331 ... user, 332 ... host device, 340 ... various home appliances.

Claims

Based on the content of the first audio input from the outside, the execution of control of one or a plurality of devices is determined based on the content of the second audio input after the input of the first audio. In electronic equipment,
The determination sound data for determining that the first sound is a desired sound is created and managed by sound input from the outside a plurality of times, and the determination sound data that is created and managed is used for the determination. Management means for determining that the first voice is a desired voice;
Control means for executing control of the one or a plurality of devices based on the content of the second voice, and the first voice is a desired voice using the judgment voice data by the management means. An electronic device that executes control of the one or more devices based on the content of the second sound by the control means.
The electronic device according to claim 1, wherein the electronic device is connected to the one or a plurality of devices controlled based on the content of the second sound via a network.
2. The electronic apparatus according to claim 1, wherein the management means can create and manage a plurality of the determination audio data.
A display unit that can display a determination result by the management unit, and when the first sound is determined by the management unit to be a desired sound using the determination sound data; The electronic device of Claim 1 displayed on a display part.
An output unit for outputting the determination result by the management means in a voice, and when the first voice is determined by the management means to be a desired voice using the determination voice data, to that effect The electronic device according to claim 1, wherein the electronic device is output from the output unit.
The management means has the determination criterion 1 having a plurality of criteria and satisfies the determination result when determining that the first sound is desired data using the determination sound data. The electronic device according to claim 4, wherein contents displayed on the display unit are changed according to any of the plurality of criteria.
The management means has the determination criterion 2 having a plurality of criteria and satisfies the determination result when determining that the first sound is desired data using the determination sound data. The electronic device according to claim 5, wherein contents output from the output unit are changed according to any of the plurality of criteria.
When the first sound is determined by the management means to be a desired sound using the determination sound data, based on part or all of the content of the second sound, The electronic device according to claim 1, wherein the content of control of one or a plurality of devices is changed.
When the first sound is determined by the management means to be a desired sound using the determination sound data, based on part or all of the content of the second sound, The electronic device according to claim 1, wherein the operation content of the electronic device is changed.
In the case where the management means has a plurality of the determination sound data, the one or a plurality of the sound is determined depending on the type of the determination sound data used to determine that the first sound is a desired sound. The electronic device according to claim 8, wherein the content of control of the device is changed.
When the management unit has a plurality of determination sound data, the operation content of the electronic device is determined according to the type of the determination sound data used to determine that the first sound is a desired sound. The electronic device according to claim 9, wherein the electronic device is changed.
The management means has a confirmation timer for confirming the input timing of the sound input from the outside, and when the determination timer determines that the timing of the sound input from the outside is more than a predetermined time, The electronic device according to claim 1, wherein input of the first sound is requested.
Selecting means for selecting a type of speech recognition dictionary used for analyzing the content of the second speech;
When the management means has a plurality of the determination sound data, the selection means determines whether the first sound is a desired sound according to the type of the determination sound data used for the determination. The electronic device according to claim 1, wherein the type of recognition dictionary is determined.
The selection means for selecting a type of the voice recognition dictionary used for analyzing the content of the second voice;
The electronic device according to claim 13, wherein the selection unit determines a type of the voice recognition dictionary according to a state of the electronic device.
Based on the content of the first audio input from the outside, the execution of control of one or a plurality of devices is determined based on the content of the second audio input after the input of the first audio. In the control method,
The determination sound data for determining that the first sound is a desired sound is created and managed by sound input from the outside a plurality of times, and the determination sound data that is created and managed is used for the determination. Management means for determining that the first voice is a desired voice;
Control means for executing control of the one or a plurality of devices based on the content of the second voice, and the first voice is a desired voice using the judgment voice data by the management means. A control method for executing control of the one or a plurality of devices based on the content of the second sound by the control means when it is determined that