JP2024072184A - Information terminal, intercom system, control method and program - Google Patents

Abstract

[Problem] To improve user convenience. [Solution] An information terminal 10 operates as an intercom device. The information terminal 10 includes a voice recognition unit 182, a control processing unit 183, and an alarm unit 9. The voice recognition unit 182 performs voice recognition on a voice input to a microphone 131. The control processing unit 183 performs control based on the voice recognition result of the voice recognition unit 182. The alarm unit 9 issues an alarm according to the success or failure of the voice recognition by the voice recognition unit 182. [Selected Figure] Figure 1

Description

This disclosure generally relates to an information terminal, an intercom system, a control method, and a program, and more specifically to an information terminal that operates as an intercom device, an intercom system that includes an information terminal, and a control method and a program for the information terminal.

Intercom systems used in apartment buildings and the like are known (see, for example, Patent Document 1). The intercom system of Patent Document 1 includes a lobby intercom installed at the common entrance of the apartment building, an intercom master unit installed in each dwelling unit, and a door phone slave unit installed outside each dwelling unit (at the entrance).

In this type of intercom system, a visitor calls the intercom master unit using the lobby intercom. In response to this call, the resident of the residence performs a specified operation on the intercom master unit, which starts a conversation between the lobby intercom and the intercom master unit. Similarly, when the intercom master unit is called using the doorphone slave, the resident of the residence performs a specified operation on the intercom master unit in response to the call, which starts a conversation between the doorphone slave and the intercom master unit.

JP 2004-64249 A

There are systems that perform voice recognition based on the voices uttered by residents of a residence as a specified operation. When using voice recognition in an information terminal (e.g., an intercom master unit) or an intercom system, it is desirable to improve the convenience of the user (resident, etc.).

The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide an information terminal, an intercom system, a control method, and a program that can improve user convenience.

An information terminal according to one aspect of the present disclosure operates as an intercom device. The information terminal includes a voice recognition unit, a control processing unit, and an alarm unit. The voice recognition unit performs voice recognition on a voice input to a microphone. The control processing unit performs control based on the voice recognition result of the voice recognition unit. The alarm unit issues an alarm according to the success or failure of the voice recognition by the voice recognition unit.

An intercom system according to one aspect of the present disclosure includes the information terminal and another terminal that communicates with the information terminal, including making calls.

A control method according to one aspect of the present disclosure is a control method for an information terminal that operates as an intercom device. The control method includes a voice recognition step, a control processing step, and a notification step. The voice recognition step includes performing voice recognition processing based on a voice input to a microphone. The control processing step includes performing control based on a voice recognition result in the voice recognition step. The notification step includes performing a notification according to the success or failure of the voice recognition processing in the voice recognition step.

A program according to one aspect of the present disclosure is a program for causing one or more processors to execute the control method.

This disclosure makes it possible to improve user convenience.

FIG. 1 is a block diagram of an information terminal according to an embodiment. FIG. 2 is a block diagram of an intercom system including the information terminal. FIG. 3 is a front view of the information terminal in a standby state. FIG. 4 is a front view of the information terminal in a calling state. FIG. 5 is a front view of the information terminal in a standby state where a trigger word has been voice-recognized. FIG. 6 is a front view of the information terminal of the above embodiment, showing a state in which a trigger word has been voice-recognized in a calling state. FIG. 7 is a conceptual diagram showing a state in which a command word is voice-recognized in the above information terminal in a standby state. FIG. 8 is a conceptual diagram showing a state in which a command word is voice-recognized in a calling state of the information terminal of the above embodiment. FIG. 9 is a conceptual diagram showing an example of a call screen displayed on the display unit of the information terminal. FIG. 10 is a conceptual diagram showing a state in which the information terminal fails to recognize a command word. FIG. 11 is a conceptual diagram showing an example of a notification screen displayed on the display unit of the information terminal when control of the device fails. FIG. 12 is a conceptual diagram showing an example of a notification screen displayed on the display unit of the information terminal when control of the device fails. FIG. 13 is a conceptual diagram showing an example of a notification screen displayed on the display unit of the information terminal when control of the device fails. FIG. 14 is a conceptual diagram showing an example of a notification screen displayed on the display unit of the information terminal when control of the device fails. FIG. 15 is a conceptual diagram showing an example of a guide screen displayed on the display unit of the information terminal. FIG. 16 is a conceptual diagram for explaining an operation example 1 of the information terminal according to the above embodiment. FIG. 17 is a conceptual diagram illustrating an operation example 2 of the information terminal according to the embodiment of the present invention. FIG. 18 is a flowchart of a control method for the information terminal.

The embodiment and modified examples described below are merely examples of the present disclosure, and the present disclosure is not limited to the embodiment and modified examples. Various modifications other than the embodiment and modified examples described below are possible according to the design, etc., as long as they do not deviate from the technical concept of the present disclosure.

(1) Overview An information terminal 10 (see FIG. 1) of this embodiment is applied to an intercom system 1 as shown in FIG. 2. The intercom system 1 is applied to, for example, an apartment building or other collective housing 5. The intercom system 1 of this embodiment includes an information terminal 10. In this embodiment, the intercom system 1 includes multiple (two in FIG. 2) information terminals 10 each operating as an intercom device. The intercom system 1 further includes a lobby intercom 20 (another terminal), a control device 30, and multiple (two in FIG. 2) entrance sub-units 40 (other terminals).

In the intercom system 1, each of the multiple information terminals 10 and the lobby intercom 20 are configured to communicate via the control device 30. In the intercom system 1, the multiple information terminals 10 and the multiple entrance units 40 correspond one-to-one. The intercom system 1 of this embodiment may be applied to facilities other than the apartment building 5, such as detached houses. Alternatively, the intercom system 1 may be applied to non-residential facilities such as offices, stores, schools, or nursing homes.

In other words, the intercom system 1 may be applied to a facility or an apartment complex. If the facility is residential, it may correspond to a detached house or a dwelling unit in an apartment complex 5, and if it is non-residential, it may correspond to an office, store, school, or nursing home. The apartment complex may be an apartment complex 5 with multiple dwelling units (private areas), a non-residential apartment complex with multiple offices or stores (private areas), or a complex where the lower floors are non-residential apartment complexes and the upper floors are apartment complexes.

Each of the multiple information terminals 10 is, for example, a dwelling unit terminal (intercom master unit) provided in each of the multiple dwelling units E2 included in the apartment complex 5. Each information terminal 10 is assumed to be, for example, a housing information panel that functions as a room master unit provided in a room or the like within each dwelling unit E2.

Each information terminal 10 is connected to the control device 30 via the second trunk line 62, the branch line 63, and the splitter 50. Each information terminal 10 is configured to communicate (e.g., make calls and send control signals, etc.) with the lobby intercom 20 via the control device 30. Furthermore, each information terminal 10 is connected to a corresponding entrance unit 40 via a connection line 64. Each information terminal 10 is configured to communicate (e.g., make calls and send control signals, etc.) with the corresponding entrance unit 40.

The lobby intercom 20 is provided, for example, in the shared entrance (lobby) E1 of the apartment building 5. The lobby intercom 20 is connected to the control device 30 via the first trunk line 61. The lobby intercom 20 is configured to communicate (for example, make calls and transmit video signals, etc.) with each information terminal 10 via the control device 30. The lobby intercom 20 is attached, for example, to the wall of the shared entrance E1. The lobby intercom 20 transmits a video signal to the information terminal 10, which enables the information terminal 10 to display a video (image).

The control device 30 is provided, for example, in the management room E3 of the apartment building 5. The control device 30 is connected to the lobby intercom 20 via a first trunk line 61, and is connected to each information terminal 10 via a second trunk line 62. In other words, the control device 30 is configured to relay communication between each information terminal 10 and the lobby intercom 20.

Each of the multiple entrance sub-units 40 is provided, for example, at the outer entrance of dwelling unit E2 of apartment building 5. Each entrance sub-unit 40 is connected to a corresponding information terminal 10 via a connection line 64. Each entrance sub-unit 40 is configured to communicate (e.g., make calls, transmit video signals, etc.) with the corresponding information terminal 10.

In this embodiment, the first trunk 61, the second trunk 62, the branch 63, and the connection 64 are all twisted pair wires. In other words, in reality, the first trunk 61, the second trunk 62, the branch 63, and the connection 64 are each composed of two electric wires, but are shown as a single line in the drawings. At least one of the first trunk 61, the second trunk 62, the branch 63, and the connection 64 may be an electric wire other than a twisted pair wire.

The information terminal 10 according to this embodiment acquires the voice of a resident or the like in the dwelling unit E2 (hereinafter, sometimes simply referred to as a "user") and performs voice recognition on the acquired voice. As shown in FIG. 1, the information terminal 10 according to this embodiment includes a sound acquisition unit 13 and a voice recognition unit 182. The sound acquisition unit 13 acquires a sound including the user's voice. The sound acquisition unit 13 acquires a voice input to the microphone 131. The sound acquisition unit 13 includes a microphone 131 here. It is assumed that the microphone 131 is provided in the information terminal 10 itself. However, the microphone 131 may be provided in a device other than the information terminal 10 (for example, a mobile terminal such as a smartphone, a tablet terminal, or a wearable terminal), and the information terminal 10 may acquire the user's voice by communicating with the other device. The voice recognition unit 182 performs voice recognition on the voice input to the microphone 131 (sound acquired by the sound acquisition unit 13). The voice recognition unit 182 performs voice recognition on the voice input to the microphone 131 for control commands related to the control performed by the control processing unit 183.

The information terminal 10 also controls the operation of the intercom system 1 based on the results of the voice recognition. That is, the information terminal 10 includes a control processing unit 183. The control processing unit 183 performs control based on the voice recognition results of the voice recognition unit 182.

In short, the information terminal 10 is configured to be capable of voice operation. For example, when the information terminal 10 determines (by voice recognition) that the voice of the user in the dwelling unit E2 contains a keyword (command word) of a control command for controlling the opening and closing of the front door 200 (door) provided at the shared entrance E1, it performs control according to the command word. Specifically, when the information terminal 10 detects "unlock" as a control command through voice recognition by the voice recognition unit 182, it performs unlocking control of the electric lock 201 to open the front door 200 (see FIG. 2). Here, the front door 200 is configured to be opened and closed by the electric lock 201 (see FIG. 2).

In this way, by configuring the information terminal 10 to be voice operable, it is possible to improve user convenience.

However, in an information terminal 10 configured to be voice operable, depending on the surrounding sound environment, the volume and quality of the user's voice, etc., even if the user speaks a keyword, the voice recognition unit 182 may not be able to correctly recognize the keyword. In such cases, the control intended by the user will not be performed, which may reduce user convenience.

Therefore, the information terminal 10 of this embodiment further includes a notification unit 9. The notification unit 9 issues a notification according to the success or failure of the voice recognition by the voice recognition unit 182. This allows the user to know whether the voice recognition by the voice recognition unit 182 was successful or not through the notification by the notification unit 9. For example, if the notification by the notification unit 9 indicates that a keyword has not been correctly voice recognized, the user can take measures such as speaking the keyword again. This can improve user convenience.

In this embodiment, as an example, it is assumed that the information terminal 10 is a residence terminal (intercom master unit). However, this is not limited to this, and the information terminal 10 may be, for example, a lobby intercom 20 or a front door sub-unit 40. Furthermore, the information terminal 10 may be, for example, a manager's room master unit included in the intercom system 1.

(2) Details (2-1) Information Terminal As shown in Fig. 1, the information terminal 10 includes a first communication unit 11, a second communication unit 12, a sound acquisition unit 13, an operation unit 14, a sound output unit 15, a display unit 16, a storage unit 17, a processing unit 18, and a third communication unit 19. The information terminal 10 further includes a housing 100 (see Fig. 3) that holds the first communication unit 11, the second communication unit 12, the sound acquisition unit 13, the operation unit 14, the sound output unit 15, the display unit 16, the storage unit 17, the processing unit 18, and the third communication unit 19.

The information terminal 10 has, for example, a computer system having a processor and a memory. The processor executes a program stored in the memory, causing the computer system to function as a processing unit 18. The program executed by the processor is pre-recorded in the memory of the computer system here, but may also be provided by being recorded on a non-transitory recording medium such as a memory card, or may be provided via a telecommunications line such as the Internet.

The first communication unit 11 is a communication interface for communicating with the lobby intercom 20 (its communication unit 21). The first communication unit 11 is connected to the control device 30 via the second main line 62, the branch line 63, and the splitter 50. The first communication unit 11 transmits audio signals, control signals, etc. to the lobby intercom 20 via the control device 30. Furthermore, the first communication unit 11 receives call signals, audio signals, video signals, etc. from the lobby intercom 20 via the control device 30.

The second communication unit 12 is a communication interface for communicating with the outside (here, the server 70). The second communication unit 12 is connected to the server 70 via the network NT1. The second communication unit 12 may receive various information from the server 70. In addition, the control device 30 may be connected to the network NT1 and the server 70 via a gateway or the like, and the information terminal 10 may be connected to the server 70 via the branch line 63 and the control device 30.

The third communication unit 19 is a communication interface for communicating with the front door unit 40. The second communication unit 12 is connected to the front door unit 40 via a connection line 64. The third communication unit 19 transmits audio signals, control signals, etc. to the front door unit 40, and receives call signals, audio signals, video signals, etc. from the front door unit 40.

The information terminal 10 can be in three operating states depending on the communication state with the lobby intercom 20 and the front door unit 40. The operating states that the information terminal 10 in this embodiment can be in include a call state, a call state, and a standby state. The call state is a state in which a call signal is received from the lobby intercom 20 or the front door unit 40 and a call is being received. The call state is a state in which communication is established with the lobby intercom 20 or the front door unit 40 so that a call can be made. The standby state is a state in which the information terminal 10 is neither in a call state nor in a call state.

For convenience, in the following description, terminals capable of communicating with the information terminal 10, including making calls (here, the lobby intercom 20 and the entrance handset 40) are also referred to as "other terminals." Note that the terminals capable of communicating with the information terminal 10, including making calls (other terminals), may be the manager's room master unit.

The sound acquisition unit 13 acquires sounds including at least the user's voice, and outputs sound information (sound data) related to the acquired sounds to the processing unit 18. The sound acquisition unit 13 includes a microphone 131. The sound acquisition unit 13 converts ambient sounds including the voice (sound) of a user located in front of the information terminal 10, which are input to the microphone 131, into analog sound signals (sound information), and outputs the analog sound signals to the processing unit 18.

The sound output unit 15 includes a speaker 151. When the information terminal 10 is in a state where it can communicate with the lobby intercom 20 (a call state), the speaker 151 outputs a sound (including the voice of a visitor, etc.) based on a voice signal (sound data) transmitted from the lobby intercom 20. When the information terminal 10 is in a state where it can communicate with the front door unit 40 (a call state), the speaker 151 outputs a sound (including the voice of a visitor, etc.) based on a voice signal (sound data) transmitted from the front door unit 40. In addition, the speaker 151 outputs various voice messages and electronic sounds stored in the memory unit 17, for example, depending on the status of the information terminal 10.

The sound acquisition unit 13 includes a microphone 131. Here, the sound acquisition unit 13 includes two microphones 131, a first microphone 1311 and a second microphone 1312.

As shown in FIG. 3, the first microphone 1311 and the second microphone 1312 are respectively arranged at the upper left and right sides when viewed from the front of the housing 100 of the information terminal 10. The first microphone 1311 and the second microphone 1312 are arranged apart. Here, the first microphone 1311 is located farther from the speaker 151 than the second microphone 1312. In other words, the first microphone 1311 is the far-end microphone.

In the present embodiment, the information terminal 10 uses the user's voice input to the first microphone 1311 as voice data for a call when making a call with another terminal (lobby intercom 20 or entrance unit 40), for example. The voice input to the first microphone 1311 is mainly subjected to voice processing for the call in the voice processing unit 181 of the processing unit 18, and is transmitted to the other terminal during the call. In other words, by using the voice of the far-end first microphone 1311 as the voice data for the call, rather than the second microphone 1312, the (visitor's) voice output from the speaker 151 is prevented from being heard by the microphone 131.

On the other hand, the information terminal 10 of this embodiment uses, for example, the user's voice input to the first microphone 1311 and the second microphone 1312 as voice data for voice recognition. In particular, the voice processing unit 181 of the processing unit 18 has a function of using the two microphones 131 to execute processing related to beamforming, which enhances the directivity of the sound receiving direction. From the user's voice input from the first microphone 1311 and the second microphone 1312, the processing unit 18 estimates from which direction the user is speaking to the information terminal 10, either left or right, and executes processing to increase the sensitivity to voice signals from a specific direction based on the estimation result.

The operation unit 14 is configured to accept operations by the user. As shown in FIG. 3, the operation unit 14 has a call button 141, an end button 142, an unlock button 143, and a home button 144.

The call button 141 is a button for starting a call state (a call with a visitor, etc.) with another terminal (lobby intercom 20 or front door unit 40) in response to a call from that terminal. For example, when the call button 141 is pressed while the first communication unit 11 is receiving a call signal to call a resident (calling state), a voice call becomes possible between the information terminal 10 and the other terminal (lobby intercom 20). Also, when the call button 141 is pressed while the third communication unit 19 is receiving a call signal to call a resident, a voice call becomes possible between the information terminal 10 and the other terminal (front door unit 40).

The end button 142 is a button for ending a call (call state) with another terminal (lobby intercom 20 or entrance handset 40). For example, when a voice call is being made with the lobby intercom 20 via the first communication unit 11 (call state), pressing the end button 142 ends the voice call with the lobby intercom 20. Note that in the call state, pressing the call button 141 also ends the voice call.

The unlock button 143 is a button for unlocking the electric lock 201 provided on a door (here, the front door 200 provided at the common front door E1; see FIG. 2) for entering and exiting the facility (here, the apartment building 5) where the information terminal 10 is installed. For example, when the unlock button 143 is pressed during a voice call with the lobby intercom 20 via the first communication unit 11 (call state), an unlock signal for unlocking the electric lock 201 is transmitted from the information terminal 10 via the control device 30. As a result, the electric lock 201 is unlocked, and the visitor can open the front door 200 and enter beyond the common front door E1. When the unlock button 143 is pressed, the information terminal 10 ends the voice call with the lobby intercom 20 in addition to unlocking the electric lock 201.

The home button 144 is a button for calling up a specific home screen that is displayed on the display unit 16. When the home button 144 is pressed, the information terminal 10 performs control to change the screen of the display unit 16 to the home screen.

In this embodiment, as an example, the call button 141, the end button 142, the unlock button 143, and the home button 144 are realized by a capacitance-type touch sensor arranged around the display unit 16 on the front side of the housing 100 of the information terminal 10, as shown in FIG. 3. That is, the user can input an operation by simply lightly touching the call button 141, the end button 142, the unlock button 143, or the home button 144 with the tip of a finger. However, the call button 141, the end button 142, the unlock button 143, and the home button 144 are not limited to being realized by a touch sensor, and at least some of these may be realized by a push button-type switch.

The display unit 16 is, for example, a liquid crystal display. Various screens can be displayed on the display unit 16. For example, the display unit 16 is configured to display video. When the information terminal 10 is in a state where it can make a call (communicate) with the lobby intercom 20, the display unit 16 displays video captured by the lobby intercom 20, which is the communication target. When the information terminal 10 is in a state where it can make a call (communicate) with the front door sub-unit 40, the display unit 16 displays video captured by the front door sub-unit 40, which is the communication target. Note that, when the information terminal 10 is equipped with a touch panel display, the touch panel display may serve as both the display unit 16 and the operation unit 14.

The storage unit 17 is composed of a readable and writable memory (storage device). The storage unit 17 is, for example, a flash memory.

The storage unit 17 stores, for example, a dictionary file used in voice recognition. The dictionary file may be downloaded from the server 70 via the second communication unit 12 and stored. In this embodiment, the dictionary file includes a trigger dictionary D0 and a command dictionary D1. In addition to the dictionary file, the storage unit 17 also stores information on the screen to be displayed on the display unit 16, information on voice messages, various setting information, and the like.

Trigger words are registered in the trigger dictionary D0. A trigger word is a predetermined keyword that triggers the detection of a command word of a control command. The trigger words registered in the trigger dictionary D0 are predetermined. For example, when the information terminal 10 is manufactured, a predetermined trigger word is registered in the trigger dictionary D0 of the storage unit 17. The trigger word is, for example, the product name of the information terminal 10 or an abbreviation thereof, and is preferably a relatively small number of characters. Note that the user of the information terminal 10 may be able to register any trigger word in the trigger dictionary D0 using the sound acquisition unit 13, the operation unit 14, the display unit 16, etc. Alternatively, one of a plurality of trigger word candidates that are predetermined and stored in the first storage unit 171 may be registered in the trigger dictionary D0 as a trigger word in response to an operation on the operation unit 14, etc.

In the command dictionary D1, command words of multiple types of control commands are registered. The multiple types of control commands are commands for controlling at least one of the information terminal 10 and one or more devices connected to the information terminal 10. Here, the information terminal 10 is connected to, for example, a controller of a HEMS (Home Energy Management System) installed in the dwelling unit E2 so as to be able to communicate with the controller, and the HEMS controller is connected to be able to control one or more electrical devices (air conditioner, lighting device, air purifier, range hood, etc.). The "one or more devices connected to the information terminal 10" may include, for example, one or more electrical devices that can be controlled by the HEMS controller. When the information terminal 10 receives a control command related to the control of the device connected to the information terminal 10 by voice input, it controls the corresponding electrical device, for example, via the HEMS controller.

Command words of the control commands for controlling the information terminal 10 include, for example, the command word "answer" for starting a call with this terminal (a call with a visitor, etc.) in response to a call from another terminal (lobby intercom 20 or front door unit 40), the command word "unlock" for unlocking the electric lock 201 of the front door 200, and the command word "end" for ending a call with another terminal. Command words of the control commands for controlling one or more devices connected to the information terminal 10 include, for example, "turn on the device name (e.g., turn on the air conditioner)" for turning on the power of the device to be controlled, and "turn off the device name (e.g., turn off the air conditioner)" for turning off the power of the device to be controlled.

As shown in FIG. 1, the processing unit 18 includes a voice processing unit 181, a voice recognition unit 182, a control processing unit 183, a display processing unit 184, a call processing unit 185, a notification processing unit 186, and a presentation processing unit 187.

The audio processing unit 181 acquires analog sound data input from the first microphone 1311 and output by the sound acquisition unit 13 for voice communication, and converts it into a digital sound signal. The audio processing unit 181 is also configured to perform filtering processing or the like on the sound data input from the first microphone 1311 and output by the sound acquisition unit 13. The audio processing unit 181 includes, for example, an echo canceller and/or an echo suppressor. The echo canceller and the echo suppressor suppress or remove echo components from the sound data output from the microphone 131. The audio processing unit 181 is also configured to acquire sound data input from the first microphone 1311 and the second microphone 1312 and output by the sound acquisition unit 13 for voice recognition, and perform the above-mentioned beamforming processing.

The voice recognition unit 182 performs voice recognition on the voice input to the microphone 131 (in this embodiment, the first microphone 1311 and the second microphone 1312). In other words, the voice recognition unit 182 performs voice recognition processing on the voice acquired by the sound acquisition unit 13. The voice recognition unit 182 performs voice recognition processing on the sound processed by the voice processing unit 181 using the dictionary files (trigger dictionary D0, command dictionary D1) stored in the storage unit 17.

As shown in FIG. 1, the voice recognition unit 182 includes a first voice recognition unit 1821 and a second voice recognition unit 1822.

The first voice recognition unit 1821 performs voice recognition (hereinafter also referred to as "trigger voice recognition") on the voice input to the microphone 131 to determine whether or not a trigger word is included. The first voice recognition unit 1821 performs trigger voice recognition using a trigger dictionary D0 stored in the storage unit 17. The first voice recognition unit 1821 monitors the sound processed by the voice processing unit 181 through trigger voice recognition, and determines whether or not a trigger word is included.

The second voice recognition unit 1822 performs voice recognition (hereinafter also referred to as "command voice recognition") for control commands for the voice input to the microphone 131. The second voice recognition unit 1822 performs command voice recognition using the command dictionary D1 stored in the storage unit 17. The second voice recognition unit 1822 monitors the sound processed by the voice processing unit 181 by command voice recognition and determines whether any command word is included. When the second voice recognition unit 1822 succeeds in voice recognition of a command word of a control command for the voice input to the microphone 131 (detection of a command word), it outputs the command word (control command) to the control processing unit 183.

For each of the command words of the multiple types of control commands registered in the command dictionary D1, an operating state of the information terminal 10 in which it is valid is set. For example, the command word "respond" is only valid in a call state in which a call is being received from another terminal (i.e., a control request by that command word can be accepted). Table 1 below shows a list of specific control commands (command words) and valid operating states. The control commands are associated with control events (operation events) and stored in the memory unit 17.

Here, "Trigger word required" in Table 1 indicates whether or not voice recognition of the trigger word is required in advance when performing voice recognition of each command word. Specifically, "required" in the "trigger word required" column indicates that the voice recognition of the command word is considered successful only when the voice recognition of the trigger word and the voice recognition of the command word are successively successful. "The voice recognition of the trigger word and the voice recognition of the command word are successively successful" means, for example, that the voice recognition of the command word is successful within a predetermined standby time after the voice recognition of the trigger word is successful. The length of the standby time is, for example, set in advance. The length of the standby time is, for example, 3 seconds, but is not limited to this. The information terminal 10 may be configured so that the user can set the length of the standby time. Also, "no" in the "trigger word required" column in Table 1 indicates that the voice recognition of the command word is considered successful both when the voice recognition of only the command word without the trigger word is successful and when the voice recognition of the trigger word and the voice recognition of the command word are successively successful.

In addition, the "security set" in the operation event column of Table 1 means, for example, an event in which the security mode is turned on by using the security function of the information terminal 10 for issuing an alarm when a suspicious person's intrusion is detected. In addition, the "door monitor check" in the operation event column of Table 1 means, for example, an event in which real-time video captured by the door monitor (imaging unit) of the entrance slave unit 40 is displayed on the screen of the display unit 16 of the information terminal 10. In addition, the "recording playback" in the operation event column of Table 1 means, for example, an event in which a video captured by the imaging unit 27 of a visitor who has called the information terminal 10 through the lobby intercom 20 and a recorded message from the visitor to convey the purpose of the visitor are played back. In addition, the "operation end" in the operation event column of Table 1 means, for example, an event in which the display of the door monitor or the playback of the recording is ended during door monitor check or recording playback.

In addition, "scene control" in Table 1 means that one or more devices associated with each operation event (when coming home, when going out, when waking up, etc.) are controlled together. For example, when the command word (control command) "I'm home" associated with the operation event "operation when coming home" is voice recognized, one or more devices associated with "operation when coming home", such as control to turn on the lighting device, control to turn on the air conditioner, and control to lock the electronic lock, are controlled. The devices to be controlled and the control contents corresponding to each operation event in scene control may be set by the user using the information terminal 10.

From Table 1, we can see that the number of command words that are valid in the talking and ringing states is less than the number of command words that are valid in the standby state. Also, from Table 1, we can see that the operation events related to command words that are valid in the talking and ringing states are limited to those related to intercom control, and even more so, to events that are highly related to "talk".

Also, from Table 1, it can be seen that in a call state, voice recognition of the command word is considered to be successful with only the command word, without the trigger word. This reduces the possibility that the trigger word will be leaked to a visitor if the user utters the trigger word during a call with the visitor. Note that in one variant of the information terminal 10, even in a call state, if voice recognition of only the command word is successful without the trigger word, voice recognition of the command word (e.g., "answer" in Table 1) may be considered to be successful. In this case, it is possible to reduce the possibility that the trigger word will be leaked to a visitor if the user utters the trigger word during a call from the visitor (e.g., when there is a visitor at the front door of dwelling unit E2).

The control processing unit 183 performs control based on the voice recognition result of the voice recognition unit 182. The control processing unit 183 performs control based on the control command voice-recognized by the second voice recognition unit 1822. The control processing unit 183 performs control according to the control command based on the control command of the command word voice-recognized by the second voice recognition unit 1822. For example, when the second voice recognition unit 1822 voice-recognizes the command word "unlock" in a call state in which the information terminal 10 is receiving a call from the lobby intercom 20 or in a call state in which the information terminal 10 is in a call with the lobby intercom 20, the control processing unit 183 performs control (hereinafter also referred to as "unlock control") to unlock the electric lock 201 of the entrance door 200 provided at the shared entrance (lobby) E1 of the apartment building 5. In addition, when the second voice recognition unit 1822 recognizes the command word "answer" during a call from the lobby intercom 20, the control processing unit 183 performs control (hereinafter also referred to as "call control") to start a call with the lobby intercom 20 that is the call source (a call with a visitor, etc.).

The control processing unit 183 also performs control in response to operations on the operation unit 14. For example, when the call button 141 is pressed in the call state, the control processing unit 183 performs call control. When the unlock button 143 is pressed in the call state or call state, the control processing unit 183 performs unlock control. Each button on the operation unit 14 is set to be enabled or disabled depending on the operating state of the information terminal 10. For example, the call button 141 is set to be enabled in the call state and call state, and is set to be disabled in other operating states (standby states). The unlock button 143 is set to be enabled in the call state and call state, and is set to be disabled in other operating states (standby states).

In the information terminal 10, at least one of the multiple buttons on the operation unit 14, here the call button 141 and the unlock button 143, emits light (either on or blinks) when it is valid (i.e., when the corresponding button is in a state where it can accept a control request by touching the button), thereby notifying the user that the button is valid. For example, in the calling state, as shown in FIG. 4, the call button 141 and the unlock button 143 emit light. Also, in the calling state, the unlock button 143 emits light. Note that in FIG. 4 to FIG. 9, for ease of understanding, the illuminated buttons among the call button 141, the unlock button 143, and the home button 144 are indicated with dot hatching.

The display processing unit 184 causes the display unit 16 to display various screens. For example, the display processing unit 184 performs processing to display a home screen, a setting screen, and a screen corresponding to each operation state. For example, when the home button 144 is pressed, the display processing unit 184 causes the display unit 16 to display a home screen displaying a menu for requesting various operation controls. Also, in a calling state in which a call signal is being received from another terminal, the display processing unit 184 causes the display unit 16 to display an incoming call screen indicating that the device is in an incoming call state (calling state) (see FIG. 4).

The display processing unit 184 also performs processing to display on the display unit 16 images captured by other terminals (lobby intercom 20, entrance handset 40) that are the communication targets (active on a call or the call originator).

The display processing unit 184 also displays the voice input to the microphone 131 as text on the display unit 16. The display processing unit 184 converts the sound data of the voice input to the microphone 131 into text data using an automatic voice recognition function and displays the text data on the display unit 16. The display processing unit 184 may display all voices input to the microphone 131 as text, or may display only specific voices as text. For example, the display processing unit 184 may display, as text, the voice that is voice-recognized after the first voice recognition unit 1821 recognizes the trigger word and during the time (standby time) during which the voice input of the command word is awaited. That is, the display processing unit 184 may display, as text, the voice that is input to the microphone 131 and after the trigger word is voice-recognized by the voice recognition unit 182. By displaying the voice as text, the user can confirm that the keyword that he or she spoke has been correctly voice-recognized, which may improve the user's convenience.

The call processing unit 185 handles calls with other terminals.

The call processing unit 185 acquires sound data (sound information) related to the sound acquired by the sound acquisition unit 13 (microphone 131). The call processing unit 185 performs call processing using only one of the voice input to the first microphone 1311 or the voice input to the second microphone 1312. The call processing unit 185 performs call processing using only the voice input to the first microphone 1311, which is relatively far from the speaker 151. This makes it possible to suppress howling that occurs when the microphone 131 picks up the voice (of a visitor) output from the speaker 151.

The call processing unit 185 outputs the sound data (sound information) acquired from the sound acquisition unit 13 to a device operated by the person at the other end of the call. The call processing unit 185 transmits the sound data to another terminal (lobby intercom 20 or front door unit 40) that is currently in a call via the first communication unit 11 or the third communication unit 19. For example, when the information terminal 10 is currently in a call with the lobby intercom 20, the call processing unit 185 transmits a signal of the sound data (audio signal) to the lobby intercom 20 via the first communication unit 11.

The call processing unit 185 also acquires a sound data signal (audio signal) from another terminal (lobby intercom 20 or front door unit 40) during a call via the first communication unit 11 or the third communication unit 19. The call processing unit 185 transmits the sound data (sound information) acquired from the other terminal during a call to the sound output unit 15, and outputs it as sound from the speaker 151.

The notification processing unit 186 performs control to issue notifications depending on the success or failure of voice recognition by the voice recognition unit 182.

When the voice recognition by the voice recognition unit 182 is successful, the notification processing unit 186 controls the elements of the information terminal 10 to issue a notification. Furthermore, when the voice recognition by the voice recognition unit 182 fails, the notification processing unit 186 controls the elements of the information terminal 10 to issue a notification. Furthermore, when the control based on the voice recognition result of the voice recognition unit 182 fails, the notification processing unit 186 controls the elements of the information terminal 10 to issue a notification. That is, the information terminal 10 includes a notification unit 9 (the notification processing unit 186 and the control target of the notification processing unit 186) that issues a notification according to the success or failure of the voice recognition by the voice recognition unit 182. In the information terminal 10 of this embodiment, the control target of the notification processing unit 186 includes the display unit 16 and the speaker 151.

The notification processing unit 186 issues a notification when the voice recognition unit 182 (first voice recognition unit 1821) has successfully recognized a trigger word in response to a voice input to the microphone 131. More specifically, when the voice recognition unit 182 (first voice recognition unit 1821) recognizes a trigger word in a standby state of the information terminal 10, the notification processing unit 186 causes the home button 144 to emit light as shown in FIG. 5.

For example, when the voice recognition unit 182 recognizes a trigger word, the notification processing unit 186 lights up the home button 144 for the entire standby time (e.g., 3 seconds) for waiting for a command word. However, this is not limited to the above, and when the voice recognition unit 182 recognizes a trigger word, the notification processing unit 186 may blink the home button 144 (e.g., light up three times for a short time during the standby time of 3 seconds), or light up the home button 144 once for a short time (less than 3 seconds). Furthermore, the notification processing unit 186 may turn off the home button 144 when the second voice recognition unit 1822 recognizes a command word, even during standby time.

The notification processing unit 186 may also illuminate the home button 144 when the voice recognition unit 182 (first voice recognition unit 1821) recognizes a trigger word when the information terminal 10 is in a call state and/or a call state. For example, when a trigger word is voice recognized in a call state, the home button 144 illuminates in addition to the call button 141 and the unlock button 143 (see FIG. 6). When a trigger word is voice recognized in a call state, the home button 144 illuminates in addition to the unlock button 143.

In this way, in the information terminal 10 of this embodiment, the notification unit 9 (notification processing unit 186 and home button 144) issues a notification when the voice recognition unit 182 has successfully recognized the voice of the trigger word for the voice input to the microphone 131. By the notification unit 9 issuing a notification when the voice recognition of the trigger word is successful, the user can know that the trigger word he or she spoke has been correctly voice-recognized by the information terminal 10, and can confirm that he or she should next speak a command word. Conversely, when the notification unit 9 does not issue a notification (the home button 144 does not emit light) even though the user has spoken the trigger word, the user can know that the trigger word he or she spoke was not correctly voice-recognized by the information terminal 10, and can confirm that he or she should take measures, such as speaking the trigger word again. This can improve user convenience.

In addition, in the information terminal 10 of this embodiment, the notification of successful voice recognition of the trigger word includes notification by display. When the trigger word is spoken, it is assumed that the user will then speak the command word. If the successful voice recognition of the trigger word were to be notified by sound (alert sound), the alert sound and the user's voice may interfere with each other, which may make accurate voice recognition of the command word difficult. In contrast, in the information terminal 10 of this embodiment, the successful voice recognition of the trigger word is notified by display, making it possible to prevent interference between the alert sound and the command word spoken by the user, and improving the accuracy of voice recognition of control commands.

In addition, in the information terminal 10 of this embodiment, the display notification includes lighting or blinking of the indicator light (home button 144). This makes it easier for the user to understand the success of voice recognition for the trigger word. In addition, in the information terminal 10 of this embodiment, the indicator light (home button 144) is used for purposes other than display notification. This makes it possible to simplify the configuration of the information terminal 10 (reducing the size of the housing 100).

The notification processing unit 186 further issues a notification when the voice recognition unit 182 (second voice recognition unit 1822) recognizes a control command (command word) from the voice input to the microphone 131.

More specifically, when the voice recognition unit 182 (second voice recognition unit 1822) recognizes a command word during the standby time after the voice recognition unit 182 (first voice recognition unit 1821) recognizes a trigger word while the information terminal 10 is in a standby state, the notification processing unit 186 outputs a voice indicating the control content based on the control command from the speaker 151 as shown in FIG. 7. Note that in FIG. 7, for example, in the phrase "XXX, turn on the light!" spoken by the resident, "XXX" corresponds to the trigger word, and "Turn on the light!" following "XXX" corresponds to the command word of the control command.

In addition, when the voice recognition unit 182 (second voice recognition unit 1822) recognizes the command word "Answer" during the standby time after the voice recognition unit 182 (first voice recognition unit 1821) recognizes the trigger word in the call state of the information terminal 10, the notification processing unit 186 outputs a voice indicating the control content based on the control command from the speaker 151 as shown in FIG. 8. That is, when the voice recognition unit 182 recognizes the trigger word (here, "XXX") and the command word "Answer" for starting a call state with another terminal in the call state, the notification processing unit 186 outputs a message indicating that the state will be switched to a call state from the speaker 151. Note that when switching from the call state to the call state, the display processing unit 184 changes the incoming call screen displayed on the display unit 16 to the call screen Im0 (see FIG. 9).

In this way, in the information terminal 10 of this embodiment, the notification unit 9 (notification processing unit 186 and speaker 151) issues a notification when the voice recognition unit 182 has successfully recognized a control command related to control performed by the control processing unit 183 for the voice input to the microphone 131. This allows the user to know that the command word uttered by the user has been correctly voice recognized by the information terminal 10, which can improve user convenience.

In addition, in the information terminal 10 of this embodiment, the notification of successful voice recognition of a control command (command word) includes a notification by sound. This allows the user to reliably confirm through sound that voice recognition of the command word has been successful.

In addition, in the information terminal 10 of this embodiment, the sound notification includes the output of a voice indicating the control content based on the control command. This allows the user to confirm that the command word he or she spoke was correctly recognized by voice.

In short, in the information terminal 10 of this embodiment, the notification unit 9 issues a notification by display when the voice recognition unit 182 succeeds in voice recognition of a trigger word for a voice input to the microphone 131. Also, the notification unit 9 issues a notification by sound when the voice recognition unit 182 succeeds in voice recognition of a control command related to control performed by the control processing unit 183 for a voice input to the microphone 131. Note that this is not limited to this, and a notification by sound may be issued instead of/in addition to a notification by display when voice recognition of a trigger word is successful. Also, a notification by display may be issued instead of/in addition to a notification by sound when voice recognition of a control command is successful.

The notification processing unit 186 issues a notification when the voice recognition by the voice recognition unit 182 (second voice recognition unit 1822) of the command word fails for the voice input to the microphone 131. More specifically, when the information terminal 10 is in a standby state and the standby time has elapsed without the command word being voice recognized after the voice recognition unit 182 (first voice recognition unit 1821) voice recognizes the trigger word, the notification processing unit 186 outputs a message indicating that the voice recognition of the command word has failed (that none of the command words have been voice recognized) from the speaker 151 as shown in FIG. 10. The notification processing unit 186 may display a text message indicating that the voice recognition of the command word has failed on the display unit 16, or may notify the failure of the voice recognition of the command word by another visual means (for example, by illuminating one of the buttons of the operation unit 14).

In this way, the notification unit 9 (notification processing unit 186 and speaker 151) issues a notification when voice recognition has failed. A voice recognition failure includes a predetermined standby time that has elapsed without a control command related to control performed by the control processing unit 183 being voice recognized after it is detected that the voice input to the microphone 131 contains a trigger word. This allows the user to know that the command word that he or she spoke was not correctly voice recognized by the information terminal 10.

Table 2 below shows a list of control commands (command words), messages output from speaker 151 when voice recognition of each control command is successful (the "Success" column in Table 2), and messages output from speaker 151 when voice recognition of each control command fails (the "Failure" column in Table 2). Note that "message" here also includes electronic sounds (such as the "beep" in Table 2).

When voice recognition of the command word "I'm back" is successful, if there are unread notifications, the message "Welcome back. You have unread information" will be output; if there are no unread notifications, the message "Welcome back. There is no unread information" will be output.

As can be seen from Table 2, when voice recognition fails (when the standby time has elapsed after the trigger word has been voice recognized without the command word being voice recognized), a common message stating "Command could not be recognized" is output from speaker 151.

As can be seen from Table 2, in the information terminal 10 of this embodiment, when voice recognition of a command word is successful, information on the control content based on the control command of that command word is output. This allows the user to confirm that the command word he or she spoke was correctly recognized by voice.

As can be seen from Table 2, in the information terminal 10 of this embodiment, a notification is issued if voice recognition of a command word fails. This allows the user to know that the command word he or she spoke was not correctly voice-recognized by the information terminal 10.

As described above, the notification processing unit 186 also issues a notification when control based on the voice recognition result fails. That is, the notification processing unit 186 issues a notification when the voice recognition unit 182 (second voice recognition unit 1822) recognizes a command word for the voice input to the microphone 131, but the control by the control processing unit 183 based on the control command of the command word fails. For example, if the requested control content is in an impossible state, the control by the control processing unit 183 may fail. Failure of control by the control processing unit 183 may include, for example, a failure caused by a conflict with another process being executed, and a failure caused by a communication error with another device.

As an example of a failure of control by the control processing unit 183, when the command word "Show me the door monitor" is voice-recognized in a situation where the image of the door monitor cannot be acquired, the notification processing unit 186 issues a notification. For example, when the command word "Show me the door monitor" is voice-recognized while the image of the door monitor is being checked by another device such as a monitor device (sub-parent unit) installed in the dwelling unit E2, or when the other device such as a monitor device (sub-parent unit) is in a call state with another terminal, the notification processing unit 186 outputs an electronic beep from the speaker 151 and displays a screen (notification screen Im1) as shown in FIG. 11 on the display unit 16. As shown in FIG. 11, the notification screen Im1 includes a display area A11 that displays a message "Operated by another device. Please operate after a while," which indicates that the image of the door monitor cannot be displayed because the door monitor is being used by another device (sub-parent unit, etc.), and a display area A12 that displays a confirmation button B11 for returning to the home screen when operated by the user.

As an example of a failure of control by the control processing unit 183, when a command word to operate a device that is currently operating or being set up is voice-recognized, the notification processing unit 186 issues a notification. For example, when the command word "Turn on the air conditioner" is voice-recognized even though the air conditioner is operating, the notification processing unit 186 outputs a "beep beep" electronic sound from the speaker 151 and displays a screen (notification screen Im2) as shown in FIG. 12 on the display unit 16. As shown in FIG. 12, the notification screen Im2 includes a display area A21 that displays a message "Device is being operated or set up, so operation is not possible" indicating that the device to be controlled (e.g., air conditioner) is operating or being set up, and a display area A22 that displays a button confirmation B21 for returning to the home screen when operated by the user.

As an example of a control failure by the control processing unit 183, when a command word to operate an appliance during an alarm is recognized by voice, the notification processing unit 186 issues a notification. For example, when the command word "Turn on the air conditioner" is recognized by voice even when an alarm such as a gas leak or incomplete combustion is occurring, the notification processing unit 186 outputs a "beep beep" electronic sound from the speaker 151 and displays a screen (notification screen Im3) as shown in FIG. 13 on the display unit 16. As shown in FIG. 13, the notification screen Im3 includes a display area A31 that displays a message saying "Operation is not possible because an alarm is occurring," indicating that an alarm is occurring, and a display area A32 that displays a confirmation button B31 for returning to the home screen when operated by the user.

As an example of a control failure by the control processing unit 183, when a command word for intercom control or device control is voice-recognized and the device to be controlled is not connected, the notification processing unit 186 issues a notification. For example, when the command word "Turn on the air conditioner" is voice-recognized even though the air conditioner is not connected to the HEMS controller, the notification processing unit 186 outputs a "beep beep" electronic sound from the speaker 151 and displays a screen (notification screen Im4) as shown in FIG. 14 on the display unit 16. As shown in FIG. 14, the notification screen Im4 includes a display area A41 that displays a message saying "Device operation may not have been successful. Please check the operating status of the device," indicating that control of the device to be controlled could not be executed, and a display area A42 that displays a confirmation button B41 for returning to the home screen when operated by the user.

In this way, in the information terminal 10 of this embodiment, the notification unit 9 (notification processing unit 186, display unit 16, and speaker 151) issues a notification when control based on the voice recognition result fails. Notification when control based on the voice recognition result fails includes at least one of (here, both) a notification by sound and a notification by display. This can improve convenience for the user.

If the control based on the voice recognition result fails, the notification unit 9 may notify the user of the failure of the control based on the voice recognition result without making the notification that would be made if the voice recognition was successful. For example, if the command word "Turn on the air conditioner" is voice recognized even though an air conditioner is not connected to the HEMS controller, the notification unit 9 may display the notification screen Im4 shown in FIG. 14 on the display unit 16 without outputting the voice "Beep, turning on the air conditioner."

The presentation processing unit 187 performs control for the presentation of information related to voice recognition. The presentation processing unit 187 controls elements of the information terminal 10 to present information related to voice recognition. The objects controlled by the presentation processing unit 187 include the display unit 16 and the speaker 151.

The presentation processing unit 187 controls the elements of the information terminal 10 to present guidance regarding voice recognition. The guidance regarding voice recognition includes, for example, a list of keywords that can be voice-recognized by the voice recognition unit 182. The guidance regarding voice recognition further includes, for example, a list of control contents to be executed when a keyword (command word) is voice-recognized by the voice recognition unit 182. For example, the presentation processing unit 187 displays a guidance screen Im5 shown in FIG. 15 on the display unit 16 in response to an appropriate operation of the operation unit 14 by the user. As shown in FIG. 15, the guidance screen Im5 includes display areas A51 to A54 that display a list of keywords that can be voice-recognized by the voice recognition unit 182, and display areas A55 to A58 that display a list of control contents corresponding to each keyword. In addition, the guidance screen Im5 includes a display area A59 that displays a back button B51 for returning to the previous screen by being operated by the user, and a display area A50 that displays arrow buttons B52 and B53 for displaying a list of other keywords and control contents. The presentation processing unit 187 may output the guidance regarding voice recognition by voice from the speaker 151. However, from the viewpoint of allowing the user to speak keywords while checking the guidance regarding voice recognition, it is more preferable that the guidance regarding voice recognition be presented by display.

In this way, the information terminal 10 of this embodiment is equipped with a guidance presentation unit 81 (presentation processing unit 187 and the control object of the presentation processing unit 187) that presents guidance regarding voice recognition. In the information terminal 10 of this embodiment, the control object of the presentation processing unit 187 includes the display unit 16 and the speaker 151. This can further improve convenience for the user.

The presentation processing unit 187 also controls elements of the information terminal 10 to present candidates for voice-recognizable control commands. Here, the candidates for voice-recognizable control commands refer to command words that are valid in the current operating state of the information terminal 10. For example, in a call state, the presentation processing unit 187 displays a call screen Im0 as shown in FIG. 9 on the display unit 16. As shown in FIG. 9, the call screen Im0 includes a display area A1 for displaying a message "Operate by voice" to inform the user that voice operation is possible, a display area A2 for displaying candidates for voice-recognizable control commands (here, "Unlock" and "Exit"), and a display area A3 for displaying the control content corresponding to the control command. The call screen Im0 further includes a display area A4 that displays a recording button B1 that can be operated by the user to start recording the contents of the call, a display area A5 that displays a volume button B2 that can be operated by the user to adjust the volume, a display area A6 that displays an icon indicating that a call is in progress, and a display area A7 that displays another terminal that is currently on a call (here, entrance 1, which is the lobby intercom 20).

The presentation processing unit 187 may output candidates for voice-recognizable control commands by voice from the speaker 151 instead of or in addition to displaying them on the display unit 16. For example, in a standby state of the information terminal 10, candidates for voice-recognizable control commands may be output by voice from the speaker 151 in response to an appropriate operation on the operation unit 14 or successful voice recognition of a predetermined keyword. In addition, in the information terminal 10, in a state in which candidates for voice-recognizable control commands are being output by voice from the speaker 151, at least one of the first voice recognition unit 1821 and the second voice recognition unit 1822 may be disabled. "The first voice recognition unit 1821 (the second voice recognition unit 1822) is disabled" may mean, for example, that the circuit elements constituting the first voice recognition unit 1821 (the second voice recognition unit 1822) are not being supplied with power and are not operating, or that the first voice recognition unit 1821 (the second voice recognition unit 1822) is operating but is not performing voice recognition processing or is discarding keywords that have been voice recognized.

In this way, the information terminal 10 of this embodiment is equipped with a command candidate presentation unit 82 (presentation processing unit 187 and the control object of the presentation processing unit 187) that presents candidates for control commands that can be voice recognized. In the information terminal 10 of this embodiment, the control object of the presentation processing unit 187 includes the display unit 16 and the speaker 151. This can further improve convenience for the user.

(2-2) Lobby Intercom The lobby intercom 20 includes, as shown in FIG. 2, a communication unit 21, a processing unit 22, a call unit 23, a display unit 24, an operation unit 25, a memory unit 26, and an imaging unit 27.

The lobby intercom 20 has, for example, a microcomputer having a processor and memory. The processor executes a program stored in the memory, causing the microcomputer to function as the processing unit 22. The program executed by the processor is pre-recorded in the memory of the microcomputer here, but it may also be provided by recording it on a non-transitory recording medium such as a memory card, or it may be provided via a telecommunications line such as the Internet.

The communication unit 21 is a communication interface for communicating with the information terminal 10 (the first communication unit 11 of the information terminal 10). The communication unit 21 is connected to the control device 30 (the communication unit 31 of the control device 30) via the first trunk line 61. The communication unit 21 transmits audio signals, video signals, etc. to the information terminal 10 via the control device 30. Furthermore, the communication unit 21 receives audio signals, control signals, etc. from the information terminal 10 via the control device 30. Here, the communication signal from the lobby intercom 20 contains information (e.g., address information, etc.) for identifying the information terminal 10. Therefore, only information terminals 10 that are assigned address information that matches the address information contained in this communication signal can receive the communication signal.

The processing unit 22 is configured to control the communication unit 21, the call unit 23, and the imaging unit 27, etc.

The communication unit 23 includes a speaker and a microphone, and is configured to enable communication with the information terminal 10.

The display unit 24 is, for example, a liquid crystal display. The display unit 24 is configured to display an image captured by the imaging unit 27. The display unit 24 is also configured to display a message to a visitor, etc. The message is, for example, a message to encourage the visitor, etc. to speak. The display unit 24 displays a message such as "Please speak." In this case, a similar voice message may be output (alert) from the speaker of the communication unit 23 or a speaker provided separately from the speaker of the communication unit 23. The display unit 24 and the speaker may also be used together. If the lobby intercom 20 is equipped with a touch panel display, the touch panel display may serve as both the display unit 24 and the operation unit 25.

The operation unit 25 is configured to accept operations by, for example, visitors to the apartment complex 5, residents, etc. The operation unit 25 is, for example, an input interface having a plurality of push button switches, a touch panel, etc.

The storage unit 26 is composed of a readable and writable memory. The storage unit 26 is, for example, a flash memory. The storage unit 26 stores, for example, video data of a video (image) captured by the imaging unit 27.

The imaging unit 27 has an imaging element and is a camera for capturing an image of a subject (such as a visitor). In this embodiment, the imaging area (field of view) of the imaging unit 27 is set in front of the lobby intercom 20. In this embodiment, the imaging unit 27 is a camera that captures moving images. Furthermore, in this embodiment, the imaging unit 27 is a camera that captures color images. Note that the imaging unit 27 may be a camera that captures still images (still camera) or a camera that captures monochrome images.

The imaging element is, for example, a two-dimensional image sensor such as a CCD (Charge Coupled Devices) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. The imaging unit 27 forms an image of light from the subject on the imaging surface (light-receiving surface) of the imaging element using an optical system such as a lens, and converts the light from the subject into an electrical signal in the imaging element. The imaging unit 27 then outputs the output signal of the imaging element to the processing unit 22 as a video signal.

(2-3) Control Device As shown in FIG. 2, the control device 30 includes a communication unit 31, a processing unit 32, and a storage unit 33.

The control device 30 has, for example, a microcomputer having a processor and a memory. The processor executes a program stored in the memory, causing the microcomputer to function as the processing unit 32. The program executed by the processor is pre-recorded in the memory of the microcomputer here, but may also be provided by being recorded on a non-transitory recording medium such as a memory card, or may be provided via a telecommunications line such as the Internet.

The communication unit 31 includes a communication interface for communicating with each information terminal 10 and the lobby intercom 20. The communication unit 31 is connected to the lobby intercom 20 via a first trunk line 61, and is connected to each information terminal 10 via a second trunk line 62. In other words, the communication unit 31 is configured to relay communication between each information terminal 10 and the lobby intercom 20. Furthermore, the communication unit 31 is connected to the electric lock 201 of the front door 200, and is configured to be able to communicate with the electric lock 201.

The processing unit 32 is configured to control the communication unit 31. The processing unit 32 controls the unlocking operation of the electric lock 201 of the front door 200 in response to an instruction from the information terminal 10. For example, the processing unit 32 outputs an unlocking signal to the electric lock 201 via the communication unit 31 to instruct the electric lock to unlock.

The storage unit 33 is composed of a readable and writable memory. The storage unit 33 is, for example, a flash memory. The storage unit 33 stores, for example, a correspondence table showing the correspondence between the room number assigned to each dwelling unit E2 and the address information assigned to each information terminal 10. That is, in the control device 30, the processing unit 32 refers to the correspondence table to create a signal in which the room number included in the signal from the lobby intercom 20 is replaced with the address information of the corresponding information terminal 10, and causes the communication unit 31 to transmit this signal to each information terminal 10. Then, in each information terminal 10, the processing unit 18 acquires the information included in the signal when the address information included in the signal received by the first communication unit 11 matches the address information stored in the storage unit 17. Also, in each information terminal 10, the processing unit 18 discards the information included in the signal when the address information included in the signal received by the first communication unit 11 does not match the address information stored in the storage unit 17.

In this embodiment, the communication unit 31 is connected to the electric lock 201, but the configuration is not limited to this. The control device 30 may have a communication unit separate from the communication unit 31, and the separate communication unit may be connected to the electric lock 201. The lobby intercom 20 may have a communication unit (electric lock communication unit) separate from the communication unit 21, and the electric lock communication unit may be connected to the electric lock 201. In this case, when the information terminal 10 performs unlocking control, an unlocking signal is output from the lobby intercom 20 to the electric lock 201, and the electric lock 201 is unlocked.

2, each of the front door slave units 40 is connected to a corresponding information terminal 10 via a connection line 64. The front door slave unit 40 transmits audio signals, video signals, etc. to the information terminal 10. Furthermore, the front door slave unit 40 receives audio signals, control signals, etc. from the information terminal 10.

(3) Operation Examples The operation of the information terminal 10 will be described with reference to operation examples 1 and 2 with reference to FIGS. 16 and 17. FIG.

(3-1) Operation example 1
Operation example 1 is an example of an operation in which the information terminal 10 receives a call from another terminal, and in response to a command word uttered by the user, the information terminal 10 enters a call state. Fig. 16 is a conceptual diagram in which the states of the information terminal 10 in operation example 1 are arranged in chronological order from the left.

When in standby mode, for example, when there is an incoming call from the lobby intercom 20 (incoming call; T1), the information terminal 10 makes the speaker 151 ring to notify the user of the incoming call. The information terminal 10 also displays an incoming call screen on the display unit 16 to indicate that there is an incoming call (calling state). Furthermore, the information terminal 10 makes the call button 141 and the unlock button 143 emit light (light up or blink).

When the information terminal 10 is in a call state, the user responds to the call by performing a voice operation on the microphone 131. As shown in FIG. 16, when the information terminal 10 is in a call state, the user utters a trigger word, "XXX, answer!", and a command word.

When the first voice recognition unit 1821 voice recognizes the trigger word "XXX" (trigger voice recognition; T2), the information terminal 10 causes the home button 144 to emit light (light up or blink) in addition to the illuminated call button 141 and unlock button 143. This allows the user to confirm that the trigger word has been correctly voice recognized by the information terminal 10. Furthermore, once the trigger word has been voice recognized, the information terminal 10 waits for the user to speak a command word.

Furthermore, when the second voice recognition unit 1822 voice-recognizes the subsequently spoken command word "Answer!" (command voice recognition; T3), the information terminal 10 stops the ringtone and switches from the ringing state to a talking state, enabling a conversation with the person operating the lobby intercom 20. The information terminal 10 also outputs a message "Call will be answered, beep" from the speaker 151 to inform the user that the input of the command word has been accepted (i.e., the voice input was successful). This allows the user to confirm that the information terminal 10 is now in a talking state.

If the information terminal 10 does not detect a valid command word within the standby time (e.g., 3 seconds) after detecting the trigger word, it may output a message saying "Command not recognized."

(3-2) Operation example 2
Operation example 2 is an example of the operation of the information terminal 10 regarding a lighting device, which is an operation event of device control. Note that the lighting device referred to here is one of a plurality of devices that are installed in the dwelling unit E2 and can be controlled via the HEMS controller. Fig. 17 is a conceptual diagram in which the operations of the information terminal 10 in operation example 2 are arranged in chronological order from the left.

In the standby state (standby; T11), the information terminal 10 turns off the call button 141, the unlock button 143, and the home button 144.

When the information terminal 10 is in standby mode, the user executes the lighting device control operation by performing a voice operation on the microphone 131. As shown in FIG. 17, when in standby mode, the user utters a trigger word such as "XXX, turn on the light!" and a command word (control command).

When the first voice recognition unit 1821 of the information terminal 10 voice recognizes the trigger word "XXX" (trigger voice recognition; T12), the information terminal 10 causes the home button 144 to emit light (light up or blink). This allows the user to confirm that the trigger word has been correctly voice recognized by the information terminal 10. In addition, once the trigger word has been voice recognized, the information terminal 10 waits for the user to speak a command word.

When the second voice recognition unit 1822 voice-recognizes the subsequently spoken command word "Turn on the lights!" (command voice recognition; T13), the information terminal 10 executes the operation of controlling the lighting device (turning on the lighting device). The information terminal 10 also outputs a message "Beep, turning on the lights" from the speaker 151 to inform the user that the input of the control command has been accepted (i.e., the voice input was successful). This allows the user to confirm that the command word has been voice-recognized and that the control according to the voice recognition result (controlling the lighting device) has been successful.

(4) Modifications The above embodiment is merely one of various embodiments of the present disclosure. The above embodiment can be modified in various ways depending on the design, etc., as long as the object of the present disclosure can be achieved. In addition, the same function as the information terminal 10 according to the above embodiment may be embodied in a control method, a computer program, or a non-transitory recording medium on which a computer program is recorded, etc.

Specifically, one aspect of the control method is a control method for an information terminal that operates as an intercom device. As shown in FIG. 18, the control method includes a voice recognition step ST1, a control processing step ST2, and a notification step ST3. The voice recognition step ST1 includes performing voice recognition processing based on a voice input to the microphone 131. The control processing step ST2 includes performing control based on the voice recognition result in the voice recognition step ST1. The notification step ST3 includes performing a notification according to the success or failure of the voice recognition processing in the voice recognition step ST1.

Below, we will list some variations of the above embodiment. The variations described below can be applied in appropriate combinations.

The information terminal 10, lobby intercom 20, and front door unit 40 in the present disclosure include a computer system. The computer system is mainly composed of a processor and memory as hardware. The processor executes a program recorded in the memory of the computer system to realize the functions of the information terminal 10, lobby intercom 20, and front door unit 40 in the present disclosure. The program may be pre-recorded in the memory of the computer system, provided through a telecommunication line, or provided by recording it in a non-transitory recording medium such as a memory card, optical disk, or hard disk drive that can be read by the computer system. The processor of the computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large-scale integrated circuit (LSI). The integrated circuits such as IC or LSI referred to here are called different names depending on the degree of integration, and include integrated circuits called system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). Furthermore, a field-programmable gate array (FPGA) that is programmed after the LSI is manufactured, or a logic device that allows the reconfiguration of the connection relationship within the LSI or the reconfiguration of the circuit partition within the LSI, can also be used as a processor. The multiple electronic circuits may be integrated into one chip, or may be distributed among multiple chips. The multiple chips may be integrated into one device, or may be distributed among multiple devices. The computer system referred to here includes a microcontroller having one or more processors and one or more memories. Therefore, the microcontroller is also composed of one or more electronic circuits including a semiconductor integrated circuit or a large-scale integrated circuit.

In addition, it is not essential that the multiple functions of each of the information terminal 10, the lobby intercom 20, and the front door unit 40 are concentrated in one housing. The components of each of the information terminal 10, the lobby intercom 20, and the front door unit 40 may be distributed across multiple housings. Conversely, the multiple functions of each of the information terminal 10, the lobby intercom 20, and the front door unit 40 may be concentrated in one housing. Furthermore, at least some of the functions of each of the information terminal 10, the lobby intercom 20, and the front door unit 40, for example, some of the functions of the information terminal 10, may be realized by the cloud (cloud computing) or the like.

In one modified example, voice recognition may be performed based on a voice input to a microphone provided on a device other than the information terminal 10 (an intercom handset or a mobile terminal in the same dwelling E2) or a microphone externally attached to the information terminal 10. For example, the information terminal 10 may perform voice recognition based on a voice input to a microphone provided on a smartphone carried by a user as an example of a mobile terminal. In this case, a dedicated application software for communicating with the information terminal 10 is installed on the smartphone (mobile terminal). The mobile terminal transfers the voice (data) input to the microphone to the information terminal 10 by wireless communication conforming to a standard such as Wi-Fi (registered trademark). In other words, when the user is in the dwelling E2 and the smartphone is within a certain range from the information terminal 10, the user may be able to indirectly respond to a call on the information terminal 10 or make a call by voice operation using the smartphone.

In one modified example, the first voice recognition unit 1821 that performs trigger voice recognition and the second voice recognition unit 1822 that performs command voice recognition may be implemented separately (for example, as separate circuits). Then, in the standby state of the information terminal 10, only the first voice recognition unit 1821 and the sound acquisition unit 13 may be in an active state (activated state, i.e., a state in which power is consumed), and a voice including a trigger word may be monitored for input to the microphone 131. That is, in the standby state of the information terminal 10, the second voice recognition unit 1822 that performs command voice recognition and the control processing unit 183, etc. may be in an inactive state (inactivated state, i.e., a state in which power is not consumed or low power consumption is consumed). Then, the second voice recognition unit 1822 and the control processing unit 183, etc. may be activated and become active, for example, when a call signal is received or when the first voice recognition unit 1821 voice recognizes a trigger word.

In one variant, an element (control object) that notifies when voice recognition of a trigger word is successful, an element (control object) that notifies when voice recognition of a control command is successful, an element (control object) that notifies when voice recognition fails, an element (control object) for notifying when control based on the voice recognition result fails, an element (control object) for presenting guidance regarding voice recognition, and an element (control object) for presenting candidates for control commands that can be voice recognized may be the same as or different from each other.

In one variation, when the unlock button 143 is pressed or the voice recognition of a command word (e.g., "unlock") is successful during a call from the front door unit 40 as another terminal or during a call with the front door unit 40, the information terminal 10 may send an unlock signal to unlock the electric lock on the front door of the facility (apartment E2). The information terminal 10 may unlock the electric lock on the front door via the front door unit 40 or another device.

In one variant, when the intercom system 1 is applied to a detached house, when the unlock button 143 is pressed or the voice recognition of a command word (e.g., "unlock") is successful during a call from or a conversation with the entrance handset of the detached house, the information terminal 10 may transmit an unlock signal to unlock the electric lock on the entrance door of the detached house. The information terminal 10 may unlock the electric lock on the entrance door via the entrance handset or another device.

(5) Aspects As described above, the information terminal (10) of the first aspect operates as an intercom device. The information terminal (10) includes a voice recognition unit (182), a control processing unit (183), and an alarm unit (9). The voice recognition unit (182) performs voice recognition on a voice input to the microphone (131). The control processing unit (183) performs control based on the voice recognition result of the voice recognition unit (182). The alarm unit (9) performs an alarm depending on the success or failure of the voice recognition by the voice recognition unit (182).

This aspect makes it possible to improve user convenience.

In the information terminal (10) of the second aspect, in the first aspect, the notification unit (9) issues a notification when the voice recognition unit (182) successfully recognizes a trigger word in response to a voice input to the microphone.

According to this aspect, the user can be informed that the trigger word he or she spoke has been correctly voice-recognized by the information terminal (10), thereby improving user convenience.

In the information terminal (10) of the third aspect, in the second aspect, the notification of successful speech recognition of the trigger word includes a display notification.

This aspect makes it possible to improve the accuracy of voice recognition of control commands.

In the fourth aspect of the information terminal (10), in the third aspect, the display notification includes turning on or blinking an indicator light.

This aspect makes it easier for the user to understand the success of voice recognition for the trigger word.

In the information terminal (10) of the fifth aspect, in the fourth aspect, the indicator light is used for purposes other than displaying information.

According to this aspect, it is possible to simplify the configuration of the information terminal (10).

In the sixth aspect of the information terminal (10), in any one of the first to fifth aspects, the notification unit (9) issues a notification when the voice recognition unit (182) successfully recognizes a control command related to control performed by the control processing unit (183) for a voice input to the microphone.

According to this embodiment, the user can be informed that the command word he or she spoke has been correctly voice-recognized by the information terminal (10), thereby improving user convenience.

In the information terminal (10) of the seventh aspect, in the sixth aspect, the notification of successful voice recognition of the control command includes a sound notification.

This aspect allows the user to be reliably informed by sound that voice recognition of the command word was successful.

In the information terminal (10) of the eighth aspect, in the seventh aspect, the sound notification includes output of a voice indicating the control content based on the control command.

This aspect allows the user to confirm that the command word he or she spoke was correctly recognized.

In the information terminal (10) of the ninth aspect, in the first aspect, the notification unit (9) notifies by display when the voice recognition unit (182) has successfully recognized a trigger word for a voice input to the microphone. The notification unit (9) notifies by sound when the voice recognition unit (182) has successfully recognized a control command for control performed by the control processing unit (183) for a voice input to the microphone.

According to this aspect, the user can be informed that the trigger word and command word that he or she spoke have been correctly voice-recognized by the information terminal (10), and it is possible to improve the accuracy of voice recognition of control commands.

The information terminal (10) of the tenth aspect is any one of the first to ninth aspects, and further includes a display processing unit (184) that displays the voice input to the microphone (131) as text on the display unit (16).

According to this aspect, the user can confirm that the keywords he or she spoke have been correctly recognized, which can improve user convenience.

In the information terminal (10) of the eleventh aspect, in the tenth aspect, the voice recognition unit (182) performs voice recognition as to whether or not the voice input to the microphone contains a trigger word. The display processing unit (184) displays, as text, the voice input to the microphone after the trigger word has been voice-recognized by the voice recognition unit (182).

According to this embodiment, the user can confirm that the keyword (command word) that he or she spoke after the trigger word has been correctly recognized by voice, thereby improving user convenience.

The information terminal (10) of the 12th aspect is any one of the first to 11th aspects and further includes a guidance presentation unit (81) that presents guidance regarding voice recognition.

This aspect makes it possible to improve user convenience.

In the information terminal (10) of the thirteenth aspect, in any one of the first to twelfth aspects, the voice recognition unit (182) performs voice recognition for a control command related to the control performed by the control processing unit (183) for a voice input to the microphone. The information terminal (10) further includes a command candidate presentation unit (82) that presents candidates for control commands that can be voice recognized.

This aspect makes it possible to improve user convenience.

In the information terminal (10) of the 14th aspect, in any one of the first to 13th aspects, the notification unit (9) issues a notification when voice recognition fails.

According to this aspect, the user can be informed that the keyword he or she spoke was not correctly recognized by the information terminal (10), thereby improving user convenience.

In the information terminal (10) of the 15th aspect, in the 14th aspect, a failure in voice recognition includes a predetermined standby time elapses without a control command related to control performed by the control processing unit (183) being voice recognized after detecting that a trigger word is included in the voice input to the microphone.

According to this aspect, the user can be informed that the command word he or she spoke was not correctly recognized by the information terminal (10), thereby improving user convenience.

In the information terminal (10) of the 16th aspect, in any one of the first to fifteenth aspects, the notification unit (9) further issues a notification when control based on the voice recognition result fails.

This aspect makes it possible to improve user convenience.

The intercom system (1) of the 17th aspect includes an information terminal (10) of any one of the first to 16th aspects, and another terminal (lobby intercom 20, entrance handset 40) that communicates with the information terminal (10), including making calls.

This aspect makes it possible to improve user convenience.

The control method of the 18th aspect is a control method for an information terminal (10) that operates as an intercom device. The control method includes a voice recognition step (ST1), a control processing step (ST2), and an alert step (ST3). The voice recognition step (ST1) includes performing voice recognition processing based on a voice input to the microphone (131). The control processing step (ST2) includes performing control based on the voice recognition result in the voice recognition step (ST1). The alert step (ST3) includes performing an alert in the voice recognition step depending on the success or failure of the voice recognition processing in (ST1).

This aspect makes it possible to improve user convenience.

The program of the 19th aspect causes one or more processors to execute the control method of the 18th aspect.

This aspect makes it possible to improve user convenience.

REFERENCE SIGNS LIST 1 Intercom system 10 Information terminal 131 Microphone 151 Speaker 16 Display unit 182 Voice recognition unit 183 Control processing unit 184 Display processing unit 20 Lobby intercom (another terminal)
40 Entrance door unit (other terminal)
81 Guidance presentation unit 82 Command candidate presentation unit 9 Notification unit ST1 Voice recognition step ST2 Control processing step ST3 Notification step

Claims (19)

An information terminal that operates as an intercom device,
a voice recognition unit that performs voice recognition on a voice input to a microphone;
a control processing unit that performs control based on a voice recognition result of the voice recognition unit;
and a notification unit that notifies the user according to whether the voice recognition by the voice recognition unit is successful or not.
Information terminal. The notification unit issues a notification when the voice recognition unit has successfully recognized a trigger word for the voice input to the microphone.
2. An information terminal according to claim 1. The notification of the success of the voice recognition of the trigger word includes a notification by display.
3. An information terminal according to claim 2. The notification by the display includes lighting or blinking of an indicator light.
4. An information terminal according to claim 3. The indicator light is used for purposes other than the notification by the display.
5. An information terminal according to claim 4. the notification unit issues a notification when the voice recognition unit has succeeded in recognizing a control command related to the control performed by the control processing unit with respect to the voice input to the microphone;
2. An information terminal according to claim 1. The notification of success of the voice recognition of the control command includes a sound notification.
7. An information terminal according to claim 6. The notification by the sound includes output of a voice indicating a control content based on the control command.
8. An information terminal according to claim 7. The notification unit is
When the voice recognition unit has successfully recognized a trigger word from the voice input to the microphone, a notification is given by displaying the trigger word.
a notification by sound is given when the voice recognition unit has succeeded in recognizing a control command related to the control performed by the control processing unit with respect to the voice input to the microphone;
2. An information terminal according to claim 1. a display processing unit that displays the voice input to the microphone as text on a display unit;
2. An information terminal according to claim 1. the voice recognition unit performs voice recognition as to whether or not the voice input to the microphone includes a trigger word;
the display processing unit causes the voice input to the microphone after the trigger word has been recognized by the voice recognition unit to be displayed as text.
An information terminal according to claim 10. The system further includes a guidance presentation unit that presents guidance regarding the voice recognition.
2. An information terminal according to claim 1. the voice recognition unit performs voice recognition on the voice input to the microphone for a control command related to a control to be performed by the control processing unit,
The information terminal further includes a command candidate presentation unit that presents candidates of the control command that can be voice-recognized.
2. An information terminal according to claim 1. The notification unit issues a notification when the voice recognition has failed.
2. An information terminal according to claim 1. The failure of the voice recognition includes a state in which, after detecting that the voice input to the microphone contains a trigger word, a predetermined standby time elapses without a control command related to a control performed by the control processing unit being voice-recognized.
An information terminal according to claim 14. The notification unit further issues a notification when the control based on the voice recognition result has failed.
2. An information terminal according to claim 1. An information terminal according to any one of claims 1 to 16,
and another terminal that performs communication, including telephone calls, with the information terminal.
Intercom system. A method for controlling an information terminal that operates as an intercom device, comprising:
a voice recognition step of performing voice recognition processing based on a voice input to a microphone;
a control processing step for performing control based on a result of the voice recognition in the voice recognition step;
a notification step of performing a notification according to success or failure of the voice recognition process in the voice recognition step.
Control methods. 20. A method for causing one or more processors to execute the control method of claim 18,
program.

Images