JP2011118822A - Electronic apparatus, speech detecting device, voice recognition operation system, and voice recognition operation method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To remotely operate an electronic apparatus body by voice recognition, etc., with high accuracy and low power consumption without requiring complicated user operation. <P>SOLUTION: A television set 100 and a cellular phone 200 collect ambient voice. The cellular phone 200 detects a start of user's speech from the collected voices, transmits a speech start signal and voice signal corresponding to the voice to the television set 100, detects an end of the user's speech from the collected voices, transmits a speech end signal to the television set 100 and stops transmission of the voice signal. The television set 100, on receiving the speech start signal, starts an instruction extracting part for extracting an operation instruction by voice recognition, and extracts an operation instruction on the basis of the voice signal based on the collected voices and the received voice signals. The television set 100, on receiving the speech end signal, stops the instruction extracting part. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an electronic device that can be remotely operated by utterance, an utterance detection device that detects an utterance, a voice recognition operation system that operates an electronic device main body by voice recognition using the utterance detection device, a voice recognition operation method, and an electronic device Alternatively, the present invention relates to a program that is executed by a computer that controls the speech detection apparatus.

  The remote control of an AV device such as a television is usually used for simple operations such as power on / off operation and channel switching operation. However, in recent years, the number of remote control operation buttons has increased in proportion to the increase in additional functions such as a recording / playback function, a content playback function via the Internet connection, and a home network function, and the menu operation has become complicated. . For this reason, it is becoming difficult for those who are not familiar with AV equipment to use the remote control.

  Therefore, research and development of operation techniques that recognize the viewer's facial expressions, gestures, and voices, operate home electric appliances based on the recognition results, and do not require a remote controller, have become active. The ultimate goal is to realize technology that allows home appliances to automatically determine what operation the viewer wants to perform according to the viewer's actions, and improve user convenience It is expected to contribute greatly to

  One form of operating home appliances without using a remote control is an operation based on voice recognition. If you are a sound vocal cord owner, utterance can be done by everyone without special learning and you can express your thoughts intuitively, so you can operate AV equipment by voice recognition If it becomes, it can raise a user's convenience remarkably. In addition, with a voice, it is possible to directly input a keyword (for example, a search keyword) without feeling troublesome input by keyboard operation or remote control operation.

  An apparatus for operating a device by voice has been conventionally proposed. For example, a device in which a microphone is installed in a television main body, a user's operation command input to the microphone is recognized by voice, and a television is operated is a typical example. However, since the sound collected from the microphone is mixed with the sound generated from the speaker, the voice recognition performance is deteriorated. Therefore, a system is disclosed in which the sound output performance is improved by using the known noise output from the speaker and performing adaptive noise removal such as an echo canceller (for example, Patent Documents). 1).

  However, in order to realize more accurate speech recognition, not only echo canceller processing that removes sound from the speaker from sound collected by the microphone, but also ambient noise to extract speech from a distant person. Spatial noise removal processing, speech utterance determination processing to determine whether the voice collected by the microphone is non-command or operation command such as chat, and finally convert the voice to feature quantity, Various processes such as a voice recognition process for performing pattern matching are required.

  In addition, in a device that detects a command by voice recognition, it is necessary to always start the voice recognition function. As a result, redundant noise removal processing and speech recognition processing are executed even during a time period when there is no speech. In particular, if the TV on / off operation is also performed by voice recognition, it is necessary to always perform the above voice recognition processing even when the TV is off. , Consumes power.

  On the other hand, a system is disclosed in which a microphone is installed on a remote control of a television, voice recognition is performed inside the remote control, and a recognition result is transmitted to the television (for example, see Patent Document 2). When a microphone is installed on the remote control, the distance between the user's mouth and the microphone is extremely short, and the influence of ambient noise is reduced, so that the voice recognition accuracy can be improved.

  However, this system requires the operation of bringing the remote control to the mouth when speaking. In addition, since it is necessary to include the voice recognition processing unit in the remote controller, battery consumption is increased due to an increase in power consumption. Furthermore, depending on the sound of the television and the level of ambient noise, it is necessary to install a function for removing them in the remote control, so that the power consumption may further increase.

  Further, a system is disclosed in which a wireless communication unit is mounted on a remote control, an audio signal input to a microphone of the remote control is transmitted to a server on the Internet via wireless communication, a recognition result is received, and the recognition result is transmitted from the remote control to the television unit. (For example, refer to Patent Document 3). According to this system, the voice recognition processing unit is not required for the remote controller, and cost and power consumption can be expected to be reduced, but the point that the user has to speak with the remote controller remains unchanged. In addition, since it goes through the Internet server once, real-time performance may be impaired depending on the communication speed and environment.

  Further, a system is disclosed in which a mobile phone is used as a device for voice recognition, a command spoken toward the mobile phone is recognized by the mobile phone, and transmitted to a television (for example, see Patent Document 4). It is an advantage of this device that the mobile phone tends to be placed close to the user and is originally equipped with a microphone, so that the additional cost is low. However, this system also requires an operation of speaking with a mobile phone, so the troublesomeness is not different from the operation of pressing a button on a conventional remote control. In addition, as with other conventional systems, there is a possibility that speech recognition may not be performed well depending on the surrounding noise environment.

Japanese Patent Laid-Open No. 5-22779 JP 2001-318689 A JP 2003-115939 A JP 2005-65156 A

  As described above, in the systems disclosed in Patent Documents 1 to 4, it is necessary to perform troublesome operations such as pressing the voice recognition switch or holding the microphone near the mouth each time a speech is performed. In this case, it is difficult to improve the voice recognition accuracy. Further, even when remote operation by voice recognition is not performed, it is necessary to perform a series of command extraction processing, which increases power consumption.

  The present invention has been made in view of the above circumstances, and an electronic device capable of performing remote operation of the electronic device main body by voice recognition or the like with high accuracy and low power consumption without performing troublesome operations by the user, An object of the present invention is to provide a speech detection device, a speech recognition operation system, a speech recognition operation method, and a program.

In order to achieve the above object, an electronic device according to the first aspect of the present invention provides:
At least one sound collection unit that collects ambient sound and outputs a first sound signal corresponding to the sound;
A wireless communication unit for performing wireless communication with an external device;
Command extraction for extracting an operation command uttered by a user by performing signal processing including voice recognition processing on at least one of the first voice signal and the second voice signal received by the wireless communication unit And
An operation unit for operating the main body according to the extracted operation instructions;
Based on the timing signal received by the wireless communication unit, a control unit for controlling the start and stop of the command extraction unit,
Is provided.

In this case, further comprising a human sensor for detecting the presence of the user in the surroundings,
The wireless communication unit is
Send the sensor output of the human sensor to the external device,
The controller is
Based on the sensor output of the human sensor, control the start and stop of the command extraction unit,
It is good as well.

In addition, the timing signal received by the wireless communication unit includes:
Includes an utterance start signal and an utterance end signal,
The controller is
When the utterance start signal is received by the wireless communication unit, the command extraction unit is activated,
When the utterance end signal is received by the wireless communication unit, the command extraction unit is stopped.
It is good as well.

In this case, the recording apparatus further includes a recording unit that stores the first audio signal,
In the timing signal received by the wireless communication unit,
A save command and a discard command for the recording unit are included,
The controller is
When the storage command is received by the wireless communication unit, the storage of the first audio signal to the recording unit is started,
When the discard command is received by the wireless communication unit, the first audio signal stored in the recording unit is discarded,
Controlling the command extraction unit to extract the operation command using the first audio signal stored in the recording unit;
It is good as well.

In this case, it further includes an output unit for outputting sound,
The controller is
When the wireless communication unit receives the storage command, the volume of the sound output from the output unit is reduced.
It is good as well.

In addition, the control unit
In an operation mode other than the voice recognition mode, the instruction extraction unit is stopped,
When a switching operation to the voice recognition mode is input to the operation unit,
Activating the command extraction unit and transmitting a switching signal to the voice recognition mode to the external device via the wireless communication unit,
It is good as well.

In this case, a display unit for displaying information is further provided.
The controller is
After starting the command extraction unit, a message to the effect that the user refrains from speaking is displayed on the display unit,
It is good as well.

In addition, the control unit
When a wireless connection with the external device is not established,
End the voice recognition mode and stop the command extraction unit,
It is good as well.

In addition, the control unit
When a signal indicating that the start of speech has been detected by the wireless communication unit is not received for a predetermined period of time,
Ending the voice recognition mode, stopping the command extraction unit, and sending an end notification of the voice recognition mode to the external device via the wireless communication unit,
It is good as well.

In addition, the control unit
When the operation command is a command to end the voice recognition mode,
Ending the voice recognition mode, stopping the command extraction unit, and sending an end notification of the voice recognition mode to the external device via the wireless communication unit,
It is good as well.

In addition, the control unit
In the instruction extraction unit,
Causing both the first voice signal and the second voice signal to perform voice recognition processing;
Extracting the operation command using the one with the higher likelihood of the processing result of the voice recognition processing,
It is good as well.

The utterance detection device according to the second aspect of the present invention is:
A sound collection unit that collects surrounding sound and outputs a sound signal corresponding to the sound;
A wireless communication unit that performs wireless communication with an electronic device;
An utterance detection processing unit for detecting the start and end of the user's utterance based on the sound pressure level of the audio signal;
A control unit that transmits a timing signal to the electronic device via the wireless communication unit each time the utterance detection processing unit detects the start and end of the user's utterance;
Is provided.

In this case, the control unit
Control activation and stop of the utterance detection processing unit based on the sensor output of the human sensor,
It is good as well.

In addition, the control unit
When the start of the user's speech is detected by the speech detection processing unit, the speech signal is transmitted to the electronic device together with the speech start signal as the timing signal via the wireless communication unit, and the speech detection processing When the end of the user's utterance is detected by the unit, the utterance end signal as the timing signal is transmitted to the electronic device via the wireless communication unit and the transmission of the audio signal is stopped.
It is good as well.

In this case, it further comprises a recording unit for storing the audio signal,
The utterance detection processing unit
A sound pressure level determination processing unit that determines whether the sound pressure level of the audio signal exceeds a threshold;
The controller is
When it is determined by the sound pressure level determination processing unit that the sound pressure level of the audio signal has exceeded a threshold, the storage of the audio signal to the recording unit is started, and a storage command is transmitted to the electronic device,
When the sound pressure level determination processing unit determines that the sound pressure level of the sound signal has become equal to or lower than a threshold value after a predetermined period of time, the sound signal stored in the recording unit is discarded and a discard command is issued. Send to the electronic device,
When the sound pressure level determination processing unit determines that the sound pressure level of the audio signal has exceeded a threshold value after the predetermined period has elapsed, it is stored in the utterance start signal and the recording unit via the wireless communication unit. Transmitting the audio signal to the electronic device,
It is good as well.

In addition, the control unit
When the switching signal to the voice recognition mode is received from the electronic device via the wireless communication unit, the speech detection processing unit is activated.
It is good as well.

In this case, the control unit
After activating the utterance detection processing unit,
For a predetermined period, the sound pressure level determination processing unit determines whether or not the sound pressure level of the audio signal exceeds a threshold,
Adjusting the threshold so that a ratio of a period during which the sound pressure level of the audio signal with respect to the predetermined period exceeds the threshold is smaller than a predetermined ratio;
It is good as well.

In addition, the control unit
If a wireless connection with the electronic device is not established,
Stopping the utterance detection processing unit;
It is good as well.

In addition, the control unit
When the end notification of the voice recognition mode is received from the electronic device via the wireless communication unit, the speech detection processing unit is stopped.
It is good as well.

A voice recognition operation system according to a third aspect of the present invention is:
An electronic device of the present invention;
An utterance detection device of the present invention;
Is provided.

In this case, the electronic device is a television device.
It is good as well.

Further, the speech detection device is a mobile phone.
It is good as well.

The speech recognition operation method according to the fourth aspect of the present invention is:
A voice collecting step of simultaneously collecting ambient sounds by an utterance detection device and an electronic device placed near the user;
An utterance start detection step of detecting the start of the user's utterance based on an audio signal corresponding to the collected audio using the utterance detection device;
A first transmission step of transmitting an utterance start signal from the utterance detection device to the electronic device by wireless communication when the utterance start detection step detects the start of the user's utterance;
An activation step of activating an instruction extraction unit that extracts an operation instruction uttered by a user from an audio signal input in the electronic device according to the received utterance start signal;
An extraction step of extracting the operation command by the command extraction unit from a voice signal corresponding to the collected voice;
An utterance end detection step of detecting the end of the user's utterance based on an audio signal corresponding to the collected audio using the utterance detection device;
A second transmission step of transmitting an utterance end signal from the utterance detection device to the electronic device by wireless communication when the utterance end detection step detects the end of the user's utterance;
When the electronic device receives an utterance end signal, a stop step of stopping the command extraction unit;
An operation step of operating the main body according to the extracted operation instructions;
including.

A program according to the fifth aspect of the present invention is:
The computer that controls the electronics
At least one voice collecting means for collecting ambient voice and outputting a first voice signal corresponding to the voice;
A wireless communication means for performing wireless communication with an external device;
Command extraction means for performing signal processing including voice recognition processing on at least one of the first voice signal and the second voice signal transmitted from an external device to extract an operation command uttered by the user; ,
Operating means for operating the main body according to the extracted operation instructions;
Based on a timing signal transmitted from the external device, control means for controlling activation and stop of the instruction extraction means,
And make it work.

A program according to the sixth aspect of the present invention is:
A computer that controls an utterance detection device that detects a user's utterance,
Sound collecting means for collecting surrounding sound and outputting sound signals corresponding to the sound;
Wireless communication means for performing wireless communication with an electronic device;
Utterance detection processing means for detecting the start and end of the user's utterance based on the sound pressure level of the audio signal;
Control means for transmitting a timing signal to an electronic device each time the utterance detection processing means detects the start and end of the user's utterance;
And make it work.

The present invention has the following effects.
(1) The electronic device according to the first aspect of the present invention acquires an audio signal based on the collected audio and an audio signal received from an external device. Thereby, the operation command which the user uttered can be extracted using the better one or both of the two audio signals. According to this configuration, the electronic device can stop the command extraction unit when the utterance is not detected based on the timing signal received from the external device. As a result, the remote operation of the electronic device main body by voice recognition or the like can be performed with high accuracy and low power consumption without performing troublesome operations by the user.

(2) The utterance detection device according to the second aspect of the present invention transmits a timing signal to the electronic device every time it detects the start and end of a user's utterance based on the collected audio signals. In this way, the electronic device that has received the timing signal can extract the operation command uttered by the user at an accurate timing, and also stops the command extraction unit that extracts the operation command uttered by the user when unnecessary. I can leave it to you. As a result, the remote operation of the electronic device main body by voice recognition or the like can be performed with high accuracy and low power consumption without performing troublesome operations by the user.

(3) According to the voice recognition operation system according to the third aspect of the present invention, since the electronic device and the utterance detection device of the present invention are provided, an electronic device based on voice recognition or the like can be used without any troublesome operation by the user. Remote operation of the device body can be performed with high accuracy and low power consumption.

(4) According to the speech recognition operation method according to the fourth aspect of the present invention, the speech detection is performed by the speech detection device placed near the user. Further, the extraction of the operation command by voice recognition or the like is performed by using at least one of a voice signal based on the voice collected by the speech detection device and a voice signal based on the voice collected by the electronic device, for example, the better one or A combination of both. Further, it is possible to activate the command extraction unit that extracts the operation command only after the utterance start signal is transmitted until the utterance end signal is transmitted. Thereby, the remote operation of the electronic device main body by voice recognition or the like can be performed with high accuracy and low power consumption without performing troublesome operations by the user.

(5) According to the program according to the fifth aspect of the present invention, the computer acquires an audio signal based on the collected audio and an audio signal transmitted from an external device. In this way, an operation command uttered by the user can be extracted using the better one or both of the two audio signals. In addition, the electronic device can stop the instruction extraction unit based on the timing signal transmitted from the external device when the utterance is not detected. As a result, the remote operation of the electronic device main body by voice recognition or the like can be performed with high accuracy and low power consumption without performing troublesome operations by the user.

(6) According to the program according to the sixth aspect of the present invention, the computer transmits a timing signal to the electronic device every time it detects the start and end of the user's utterance based on the collected audio signal. Accordingly, the electronic device that has received the timing signal can extract the operation command at an accurate timing, and can stop the command extraction unit when it is unnecessary. As a result, the remote operation of the electronic device main body by voice recognition or the like can be performed with high accuracy and low power consumption without performing troublesome operations by the user.

It is a schematic diagram which shows the structure of the speech recognition operation system which concerns on embodiment of this invention. It is a block diagram which shows the structure of the television apparatus which comprises the speech recognition operation system of FIG. It is a block diagram which shows the structure of the mobile telephone which comprises the voice recognition operation system of FIG. FIG. 3 is a state transition diagram according to an operation mode of the television apparatus of FIG. 2. It is a flowchart of a series of whole operation | movement in the voice recognition mode in the voice recognition operation system of FIG. 6 is a subroutine for sound pressure detection threshold calibration of FIG. FIGS. 7A to 7C are timing charts (part 1) for explaining the sound pressure detection threshold calibration. FIG. 8A to FIG. 8C are timing charts (part 2) for explaining the sound pressure detection threshold calibration. It is a subroutine of the speech detection process of FIG. FIGS. 10A to 10F are timing charts for explaining the speech detection process. It is a subroutine of the continuation determination process of FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  As shown in FIG. 1, a voice recognition operation system 300 according to the present embodiment includes a television device 100 and a mobile phone 200. Television apparatus 100 corresponds to the electronic device of voice recognition operation system 300 according to the present embodiment. The mobile phone 200 is placed near the user. In the present embodiment, the mobile phone 200 corresponds to an utterance detection device that constitutes the voice recognition operation system 300.

(Television device)
As shown in FIG. 2, the television apparatus 100 includes a television backbone unit 1, a speaker 2, a display unit 3, a microphone 4, an acoustic processing unit 5, and an acoustic control unit 6.

  The TV backbone 1 collects various general functions of a TV for receiving and playing back broadcast radio waves. The speaker 2 outputs audio corresponding to the audio signal output from the TV backbone unit 1 via the acoustic processing unit 5. The display unit 3 displays a video corresponding to the video signal output from the TV backbone unit 1. For example, the display unit 3 displays a message or the like for performing sound pressure detection threshold calibration described later with high accuracy.

  One or more microphones 4 are provided. The microphone 4 collects sound around the television apparatus 100, converts it into an analog sound signal, and outputs it.

  The acoustic processing unit 5 performs voice input / output processing. The audio signal sent from the microphone 4 is input to the acoustic processing unit 5, subjected to acoustic processing there, and then input to the television backbone unit 1 as necessary. The audio signal output from the TV backbone unit 1 is input to the acoustic processing unit 5, subjected to acoustic processing there, and then output from the speaker 2 as necessary.

  The acoustic control unit 6 mainly controls the acoustic processing unit 5.

  The TV backbone 1 will be described in more detail. In FIG. 2, an operation control unit 10, a human sensor 11, and a wireless communication unit 12 are shown as components constituting the TV backbone 1.

  The operation control unit 10 controls the television apparatus 100 according to a user operation input (for example, an operation input of a remote controller) or an operation command described later.

  The human sensor 11 includes an ultrasonic sensor, an infrared sensor, and the like. These sensor outputs are used to determine whether there is a user around the television apparatus 100.

  The wireless communication unit 12 has general wireless communication such as WLAN, and short-range wireless communication functions such as Bluetooth (registered trademark), low power consumption version Bluetooth (registered trademark), and Zigbee (registered trademark). The output of the human sensor 11 can be transmitted to the outside via the wireless communication unit 12.

  Next, the acoustic processing unit 5 will be described in more detail. The acoustic processing unit 5 includes an AD converter (ADC) 20, a recording area unit 21, an echo canceller 22, a noise removal processing unit 23, an utterance determination processing unit 24, a speech recognition processing unit 25, a speech recognition control unit 26, a DA converter (DAC) 27). The instruction extraction unit 7 is formed by the ADC 20, the recording area unit 21, the echo canceller 22, the noise removal processing unit 23, the speech determination processing unit 24, the speech recognition processing unit 25, and the speech recognition control unit 26.

  The ADC 20 is an AD converter with an amplifier. The ADC 20 amplifies an analog audio signal input from the microphone 4 and converts it into digital audio data.

  The recording area unit 21 stores digital audio data output from the ADC 20. The stored digital audio data is used for extracting operation commands such as the echo canceller 22 and the digital filter of the noise removal processing unit 23.

  The echo canceller 22 cancels the audio component (echo) emitted from the speaker 2 from the digital audio data output from the ADC 20. More specifically, the echo canceller 22 inputs digital audio data corresponding to the audio output from the TV backbone 1 and output from the speaker 2 as a reference signal. The echo canceller 22 outputs data in which the component of the reference signal is suppressed from the digital audio data output from the ADC 20 using an application filter or the like.

  The noise removal processing unit 23 performs digital filter processing to remove and outputs other noise mixed before the user's voice reaches the microphone 4 from the user. When a plurality of microphones 4 are installed, a microphone array technique is applied to the digital filter processing. On the other hand, when only one microphone 4 is installed, a noise removal technique for a single microphone is applied to the digital filter processing.

  The speech determination processing unit 24 determines whether the digital voice data output from the noise removal processing unit 23 or the wireless communication unit 12 is an operation command (whether it is other everyday conversation or the like). If the speech determination processing unit 24 determines that the input digital sound data is an operation command, the speech determination processing unit 24 outputs the digital sound data.

  The voice recognition processing unit 25 performs voice recognition processing on the digital voice data output from the utterance determination processing unit 24. More specifically, the voice recognition processing unit 25 has a command database (DB) (not shown). The voice recognition processing unit 25 refers to the command DB and extracts the feature amount from the input digital voice data or performs pattern matching to specify the content of the operation command included in the digital voice data. The voice recognition processing unit 25 inputs the content of the specified operation command to the operation control unit 10 of the television backbone unit 1. The operation control unit 10 controls the operation of the main body of the television device 100 in accordance with a user operation input or an operation command described later.

  The voice recognition control unit 26 controls the echo canceller 22, the noise removal processing unit 23, the speech determination processing unit 24, and the voice recognition processing unit 25.

  The DAC 27 converts the digital audio data output from the TV backbone unit 1 into an analog audio signal and outputs the analog audio signal to the speaker 2.

  Next, the acoustic control unit 6 will be described in more detail. The acoustic control unit 6 controls the ADC 20, the recording area unit 21, the voice recognition control unit 26, and the DAC 27.

  For example, the acoustic control unit 6 controls the storage of the digital audio data in the recording area unit 21 of the digital audio data output from the ADC 20. The acoustic control unit 6 controls the voice recognition control unit 26 as a host controller. For example, the acoustic control unit 6 controls the input of the digital voice data output from the TV backbone unit 1 to the echo canceller 22 and the voice recognition result output from the voice recognition processing unit 26 via the voice recognition control unit 26. Transmission control to the operation control unit 10 is performed.

  The acoustic control unit 6 inputs the sensor output of the human sensor 11. The sound control unit 6 controls on / off of the command extraction unit 7 based on the sensor output of the human sensor 11. This on / off control includes power on / off control of the amplifier of the ADC 20.

  The acoustic control unit 6 transmits and receives data to and from the mobile phone 200 via the wireless communication unit 12 of the television backbone unit 1. For example, the acoustic control unit 6 receives digital audio data from the mobile phone 200.

(mobile phone)
Next, the mobile phone 200 will be described with reference to FIG.

  As shown in FIG. 3, the mobile phone 200 includes a mobile phone backbone unit 30, a microphone 31, an utterance detection processing unit 32, and an utterance detection control unit 33.

  The mobile phone backbone unit 30 collects functions required for a general mobile phone such as a voice call function. For example, the mobile phone backbone unit 30 includes a wireless communication unit 40. The wireless communication unit 40 has general wireless communication such as WLAM, and short-range wireless communication functions such as Bluetooth (registered trademark), low power consumption version Bluetooth (registered trademark), and ZigBee (registered trademark).

  The microphone 31 collects sound around the mobile phone 200, converts it into an analog sound signal, and outputs it.

  The utterance detection processing unit 32 performs user utterance detection processing based on the analog audio signal output from the microphone 31. The utterance detection processing unit 32 includes an ADC 41, a sound pressure level determination processing unit 42, and a recording area unit 43.

  The ADC 41 has an amplifier. The ADC 41 amplifies the analog audio signal input from the microphone 31 and converts it into digital audio data.

  The sound pressure level determination processing unit 42 monitors ambient sound pressure based on the digital audio data output from the ADC 41. More specifically, the sound pressure level determination processing unit 42 determines whether the sound pressure level of the digital audio data output from the ADC 41 exceeds a certain threshold value.

  In the recording area unit 43, digital audio data output from the ADC 41 is stored.

  The utterance detection control unit 33 includes a timer 44. The utterance detection control unit 33 controls the operation of the utterance detection processing unit 32 using the timer 444 and the like.

  More specifically, when the sound pressure level determination processing unit 42 determines that the sound pressure level of the digital audio data output from the ADC 41 exceeds the threshold value, the utterance detection control unit 33 sets the recording area unit 43. Starts to store the digital audio data and causes the timer 44 to start measuring time. At the same time, the utterance detection control unit 33 transmits a digital audio data storage command to the television apparatus 100.

  When the timer 44 counts a certain time, the speech detection control unit 33 determines again that the sound pressure level of the digital sound data output from the ADC 41 has exceeded the threshold value by the sound pressure level determination processing unit 42. Then, the utterance start signal indicating that the utterance is started and the digital audio data are transmitted to the television apparatus 100 via the wireless communication unit 40.

  On the other hand, the utterance detection control unit 33 receives the sensor output of the human sensor 11 from the television device 100 via the wireless communication unit 40. The utterance detection control unit 33 controls activation and stop of the utterance detection processing unit 32 based on the sensor output of the human sensor 11. At the same time, the utterance detection control unit 33 receives a control command for switching the start and end of the voice recognition mode from the television device 100 via the wireless communication unit 40, and the utterance detection process according to the command. The start and stop of the unit 32 are controlled.

  By the way, in the television apparatus 100, four states are changed by switching the operation mode. With reference to FIG. 4, the state transition by operation mode is demonstrated.

  As shown in FIG. 4, the four operation modes are roughly divided into two modes: a normal mode (state 301 and state 302) that can be operated by a remote controller and a voice recognition mode (state 303 and state 304) that can also be operated by voice recognition. it can.

  A state 301 is a state in which the power of the television apparatus 100 is turned on, an image is received and displayed, and various operations such as power on / off and channel change can be performed by the remote controller. A state 302 is a state in which the power of the television apparatus 100 is turned off and only a startup request from the remote controller is awaited.

  A state 303 is a state in which the television apparatus 100 can be operated by voice recognition in a state where the command extraction unit 7 is activated in addition to a normal remote control operation. A state 304 is a state in which the power of the television apparatus 100 is turned off, but is waiting for either a start request from the remote control or a start request by voice recognition in a state where the command extraction unit 7 is started. is there.

  Switching from the normal mode to the voice recognition mode can be performed by a remote control operation. In addition, it is possible to set a time zone (for example, 9:00 to 24:00, etc.) that the user desires to operate in the voice recognition mode in advance, and automatically switch the mode. Hereinafter, this time zone setting is also referred to as scheduling.

  In FIG. 4, a solid arrow a connecting the two states indicates that the state in the direction of the arrow can be switched by remote control operation or scheduling. A dash-dot line arrow b connecting the two states indicates that the state in the direction of the arrow can be switched by voice operation in addition to remote control operation and scheduling.

  In the present embodiment, schedule management or the like for switching the voice recognition mode is performed by the television device 100. The operation mode of the mobile phone 200 is switched according to a switching signal from the television device 100 in the voice recognition mode.

  In this case, if Bluetooth (registered trademark) or low power consumption Bluetooth (registered trademark) is used for the wireless communication system between the mobile phone 200 and the television apparatus 100, the standby power consumption for signal standby becomes very small. . Switching from the voice recognition mode to the normal mode can be performed by voice operation in addition to scheduling and remote control operation as described above.

  Next, the operation of the voice recognition operation system 300 having the above configuration will be described.

(Operation in voice recognition mode)
First, a series of overall operations after switching to the voice recognition mode in the television device 100 will be described with reference to the flowchart of FIG. In the drawing, the television device 100 is also abbreviated as TV.

  When a switching operation to the voice recognition mode is performed by the above-described remote control operation or scheduling, an operation signal for switching to the voice recognition mode is input from the operation control unit 10 to the acoustic control unit 6. Thereby, the process shown in FIG. 5 is started.

  First, the sound control unit 6 activates the command extraction unit 7 and simultaneously transmits a command to switch to the speech recognition mode to the speech detection control unit 33 of the mobile phone 200 via the wireless communication unit 12, thereby controlling the speech detection control. The utterance detection processing unit 32 is activated by the unit 33 (step S1).

  Subsequently, the acoustic control unit 6 and the utterance detection control unit 33 activate the utterance detection processing unit 32 and perform a sound pressure detection threshold calibration subroutine for calibrating a threshold for sound pressure detection (step S2). Details of the sound pressure detection threshold calibration will be described later.

  When the sound pressure detection threshold calibration is completed, the sound control unit 6 and the speech detection control unit 33 determine whether or not the user is detected by the human sensor 11 (step S3).

  When the user is not detected (step S3; No), the acoustic control unit 6 considers that it is not necessary to perform voice recognition, stops the command extraction unit 7, while the utterance detection control unit 33 performs utterance detection processing. The unit 32 is stopped (step S4).

  When a user is detected (step S3; Yes), the utterance detection control unit 33 of the mobile phone 200 performs an utterance detection process (step S5). Thereby, the speech detection process is started. The details of this utterance detection process will be described later. When an utterance is detected in this utterance detection process, digital audio data corresponding to the voice input from the microphone 31 is stored in the recording area unit 43. Yes.

  Subsequently, as a result of the utterance detection process, the utterance detection control unit 33 determines whether or not the start of the utterance has been detected (step S6). When the start of the utterance is not detected (step S6; No), a voice recognition mode continuation determination process described later is performed (step S13). Details of the continuation determination process will be described later.

  When the start of the utterance is detected (step S6; Yes), the utterance detection control unit 33 transmits a signal (speech start) to the television device 100 via the wireless communication unit 40 that the start of the utterance is detected. Signal) and digital audio data stored in the recording area 43 are started (step S7).

  The acoustic control unit 6 controls the voice recognition control unit 26 of the activated command extraction unit 7, and echoes based on the digital voice data received from the mobile phone 200 and the digital voice data stored in the recording area unit 21. The canceller 22, the noise elimination processing unit 23, the utterance determination processing unit 24, and the speech recognition processing unit 25 are operated to start execution of a series of instruction extraction processes (step S8).

  Subsequently, the utterance detection control unit 33 refers to the determination result of the sound pressure level determination processing unit 42 and waits until the utterance is completed (step S9; No). In the meantime, the transfer of digital audio data from the mobile phone 200 to the television device 100 and a series of instruction extraction processes are continued.

  When the end of the utterance is detected (step S9; Yes), the utterance detection control unit 33 transmits the utterance end signal to the television device 100 and stops the transmission of the digital audio data (step S10). Upon receiving the utterance end signal, the sound control unit 6 stops the command extraction process in the command extraction unit 7.

  Subsequently, the sound control unit 6 determines whether or not the utterance content obtained as a result of the speech recognition exists in the instruction DB of the sound recognition processing unit 25 (step S11). If the utterance content does not exist in the command DB (step S11; No), a speech recognition mode continuation determination process described later is performed (step S13). Details of the continuation determination process will be described later.

  When the utterance content exists in the command DB (step S11; Yes), the operation control unit 10 controls the operation of the television device 100 according to the utterance content (operation command) (step S12).

  Subsequently, the acoustic control unit 6 performs a voice recognition mode continuation determination process (step S13). Details of the continuation determination process will be described later.

  As a result of the continuation determination process, when the speech recognition mode is continued (step S14; Yes), the acoustic control unit 6 determines whether or not the user is detected by the human sensor 11 (step S3). On the other hand, when the voice recognition mode is not continued (step S14; No), the acoustic control unit 6 and the speech detection control unit 33 end the operation during the voice recognition mode.

  The series of overall operations in the voice recognition mode has been described above.

  Next, the details of the sound pressure detection threshold calibration (step S2), the speech detection process (step S5), and the speech recognition mode continuation determination process (step S13) will be described.

(Sound pressure detection threshold calibration)
First, the sound pressure detection threshold calibration in step S2 (see FIG. 5) will be described with reference to FIG.

  As shown in FIG. 6, when the sound pressure detection threshold value calibration subroutine (step S2) is started, first, the sound control unit 6 displays the sound pressure detection threshold value calibration on the display unit 3 to the user. A message to refrain from speaking is displayed (step S21).

  Subsequently, for a certain time (for example, 5 seconds), the speech detection control unit 33 causes the sound pressure level determination processing unit 42 to monitor the sound pressure level of the digital audio data output from the ADC 41 (step S22). The sound pressure of the digital sound data depends on the positional relationship between the television device 100 and the mobile phone 200 at the start of the sound recognition mode and the volume of the television device 100. In this embodiment, in order to detect the utterance depending on whether or not the sound pressure level of the digital sound data exceeds a threshold, the sound pressure level of the digital sound data when the user is not speaking is checked in advance, If necessary, the threshold value is adjusted according to the level.

  At this time, the sound emitted from the speaker 2 of the television apparatus 100 is a sound of a television broadcast or video being selected or a sound for calibration (for example, pink noise). Further, at this time, the adaptive filter in the echo canceller 22 of the television apparatus 100 may be updated based on the sound pressure level of the digital audio data.

  Subsequently, when a monitoring period (for example, 5 seconds) elapses, the utterance detection control unit 33 exceeds the initial threshold and is detected by the sound pressure level determination processing unit 42 during the monitoring period. Sound pressure detection time rate is calculated (step S23). If the sound pressure detection time rate is less than the predetermined rate (5% in the present embodiment) (step S24; No), the utterance detection control unit 33 ends the sound pressure detection threshold calibration. That is, in this case, the initial threshold value is used as it is as a threshold value for sound pressure detection.

  On the other hand, when the sound pressure detection time rate is equal to or greater than a predetermined ratio (5% in the present embodiment) (step S24; Yes), the utterance detection control unit 33 sets the threshold for sound pressure detection during the monitoring period. Is adjusted to a value such that the sound pressure detection time rate is less than 5% (step S25).

  Next, the timing of threshold adjustment in sound pressure detection threshold calibration will be described with reference to the timing charts of FIGS. 7 (A) to 7 (C) and FIGS. 8 (A) to 8 (C).

  FIGS. 7A to 7C show a case where the sound pressure level of the acquired digital voice data has increased to some extent from the start of the voice recognition mode. As shown in FIG. 7A, even when not speaking, if the sound of the speaker 2 and surrounding sounds are large to some extent and the sound pressure level of the digital audio data is large, sound pressure detection based on the initial threshold during the monitoring time, that is, When the sound pressure was monitored, as shown in FIG. 7B, utterances were detected at all times during the monitoring period, and the sound pressure detection time rate was almost 100%. Therefore, as shown in FIG. 7C, the threshold value is adjusted to a larger value (adjusted threshold value) and calibrated so that the utterance is not erroneously detected during non-utterance.

  On the other hand, FIGS. 8A to 8C show a case where the sound pressure level of the digital voice data of the mobile phone 200 has been low since the start of the voice recognition mode. As shown in FIG. 8A, when the sound pressure level of the digital voice data does not exceed the initial threshold during non-speaking, as shown in FIG. 8B, the utterance is not generated at all times during the monitoring period. Since it has not been detected and there is no fear of erroneous detection even if the sound pressure detection threshold value is left as it is, the threshold value remains the initial threshold value as shown in FIG. 8C.

  Note that, when the mobile phone 200 is not necessarily in front of the television device 100 when automatically switched to the power-off state 304 in the voice recognition mode by scheduling, an initial threshold is set as a threshold in this case. The sound pressure detection threshold calibration may be performed after setting and turning on the power of the television apparatus 100.

  As an initial threshold value, a value is set such that an utterance is not erroneously detected on the assumption that the volume of a general television device 100 and the mobile phone 200 are placed at a distance of about 2 to 3 m before shipment from the factory. It is desirable to set. Further, prior to the start of operation of the speech recognition operation system 300 according to the present embodiment, the user may be able to adjust the initial threshold based on the viewing environment and usage status of the television device 100.

  Further, the adjusted threshold value may be stored in the recording area unit 21 of the television device 100 or the recording area unit 43 of the mobile phone 200 so as to be an initial threshold value at the next activation. If the threshold value after adjustment becomes too high and the accuracy of speech detection is poor, the sound pressure detection threshold calibration may be performed by the user at any time, not only when the speech recognition mode is switched.

  Further, after the sound pressure detection threshold calibration, if the volume of the speaker 2 changes due to the volume adjustment of the television device 100, the sound pressure detection threshold calibration is automatically performed according to the amount of change in the volume, You may enable it to adjust a threshold value.

(Speech detection processing)
Next, the utterance detection process in step S5 in FIG. 5 will be described with reference to FIG.

  In the utterance detection process, first, the utterance detection control unit 33 uses the sound pressure level determination processing unit 42 to monitor the sound pressure level of the digital audio data output from the ADC 41 (step S31).

  If the sound pressure level of the digital audio data is below the threshold (step S32; No), the utterance detection control unit 33 sets that no utterance has been detected (step S42), and ends the utterance detection process. .

  When the sound pressure level of the digital audio data exceeds the threshold (step S32; Yes), the utterance detection control unit 33 causes the recording area unit 43 to start storing the digital audio data (step S33). At the same time, the utterance detection control unit 33 records the digital audio data output from the ADC 41 to the television apparatus 100 (more specifically, the acoustic control unit 6) via the wireless communication unit 40. A command to start saving (save command) is transmitted (step S34). The sound control unit 6 receives this save command, and causes the recording area unit 21 to start saving digital audio data output from the ADC 20 (step S35).

  Thereafter, the utterance detection control unit 33 uses the timer 44 to suspend the operation until a certain period (for example, 0.5 seconds) elapses (step S36).

  After a certain period of time has elapsed, the utterance detection control unit 33 causes the sound pressure level determination processing unit 42 to determine again whether or not the sound pressure level of the digital audio data exceeds the threshold (step S37). If the sound pressure level of the digital signal data exceeds the threshold value (step S37; Yes), the utterance detection control unit 33 considers that the sound pressure detected 0.5 seconds ago is an utterance, and has detected the utterance. Is set (step S38), and the speech detection process is terminated.

  On the other hand, if the sound pressure level of the digital audio data does not exceed the threshold (step S37; No), the sound pressure detected 0.5 seconds ago is not utterance but noise, and the utterance detection control unit 33 Then, the digital audio data (stored data) stored in the recording area 43 is discarded (step S39). At the same time, the speech detection control unit 33 transmits a discard command for discarding the digital audio data stored in the television device 100 via the wireless communication unit 40 (step S40). The sound control unit 6 of the television apparatus 100 receives this discard command and discards the digital audio data (stored data) stored in the recording area unit 21 (step S41). Then, the utterance detection control unit 33 sets that no utterance has been detected (step S42).

  As described above, the utterance is detected based on the sound pressure level of the digital sound data output from the ADC 41. For this reason, it is expected that sound that is not an operation command (for example, when the main body of the mobile phone 200 is moved, when a cup or the like is placed nearby) is mixed. For this reason, in this utterance detection process, the utterance is not detected only by sudden sound pressure detection, but the sound pressure continuously exceeds the threshold even after a certain time (for example, 0.5 seconds). Is considered to have detected an utterance. This is because if the sound is an operation command, the sound pressure level is considered to remain high for a certain period of time (at least 1 second or more).

  Therefore, in the present embodiment, since the instruction extraction unit 7 of the television apparatus 100 is not activated at the time of the first sound pressure detection, the instruction extraction process is wasted when the first sound pressure detection is noise. Can be prevented.

  In this embodiment, when the voice is an operation command, theoretically, the speech recognition starts 0.5 seconds after the start of the utterance. Since the contents cannot be specified without completion, starting up 0.5 seconds after the utterance does not adversely affect the performance of speech recognition.

  Further, in this embodiment, the case where the sound pressure is detected again after a certain time from the first sound pressure detection is regarded as an utterance, but the time interval may be shortened or several times (three times or more) ) A case where sound pressure is detected may be regarded as an utterance.

  Next, the operation timing of the above-described speech detection process will be described with reference to the timing charts of FIGS. 10 (A) to 10 (F).

  As shown in FIG. 10 (A), when sudden noise is mixed in at time t1 and the sound pressure level increases and exceeds the threshold value, as shown in FIG. 10 (B), the sound pressure level determination processing unit. The sound pressure detection result of 42 is detected, and as shown in FIG. 10C, storage of digital audio data in the mobile phone 200 is started. Then, as shown in FIG. 10D, a save command is transmitted from the mobile phone 200 to the television apparatus 100, and as shown in FIG. 10E, saving of digital audio data is started in the television apparatus 100. The

  However, at time t2 0.5 seconds later, as shown in FIG. 10 (A), the sound pressure level becomes lower than the threshold value, and no utterance is detected as shown in FIG. 10 (B). As shown in FIG. 10C, the storage of the digital audio data in the recording area unit 43 is stopped, and the stored digital audio data is discarded. Then, a discard command is transmitted from the mobile phone 200 to the television device 100 as shown in FIG. 10D, and storage of the digital audio data in the recording area unit 21 is stopped as shown in FIG. The stored digital audio data is discarded. At this time, as shown in FIG. 10E, the instruction extraction unit 7 of the television apparatus 100 is not activated.

  Further, when the user actually speaks at time t3, as shown in FIG. 10A, the sound pressure level is maintained high even at time t4 after 0.5 seconds. For this reason, as shown in FIG. 10B, the utterance remains detected even at time t4. In this case, as shown in FIGS. 10C and 10E, the storage of the digital audio data including the uttered audio to the recording area portions 43 and 21 is continued. Further, as shown in FIG. 10D, the utterance start signal and the digital audio data are transmitted from the mobile phone 200 to the television apparatus 100 at time t4. As a result, as shown in FIG. 10 (F), the command extraction unit 7 is activated, and a series of command extraction processing is performed using the digital audio data stored in the recording area unit 21 and the transmitted digital audio data. Is called.

  By performing utterance detection processing in this way, only meaningful utterances are detected without reacting to sudden noise, and the instruction extraction unit 7 of the television apparatus 100 is activated and stopped based on the detection result. Can be controlled efficiently.

(Continuation judgment processing)
Next, the voice recognition mode continuation determination process (step S13) according to the present embodiment will be described. In the mobile phone 200, the power consumption in the voice recognition mode is larger than that in the normal standby operation. Therefore, in order to prevent the occurrence of a situation where power consumption increases as much as possible, such as when the user takes the mobile phone 200 outside in the voice recognition mode or continues to detect speech even when there is no user in the room. Then, the voice recognition mode continuation determination process is performed.

  The speech recognition mode continuation determination process will be described with reference to FIG.

  First, the acoustic control unit 6 determines whether or not there is a user around by the human sensor 11 (step S51).

  If it is determined that there is no user (step S51; No), it is determined whether or not the state where there is no user continues for a certain period (for example, 1 hour) (step S52). When the state where there is no user has been continued for a certain period (step S52; Yes), the acoustic control unit 6 ends the voice recognition mode (step S57). Subsequently, the acoustic control unit 6 stops the command extraction unit 7, transmits a stop command to the utterance detection control unit 33, and causes the utterance detection control unit 33 to stop the utterance detection processing unit 32 (step S58).

  On the other hand, when the user is detected (step S51; Yes) or when the state where there is no user has not been continued for a certain period of time (step S52; No), the acoustic control unit 6 is connected to the mobile phone 200 and the television device 100. It is determined whether a wireless connection has been established (step S53). For example, when the user goes out with the mobile phone 200 in the voice recognition mode, the wireless connection is released and communication is disabled because the distance between the television device 100 and the mobile phone 200 is increased. In this case, since the wireless connection is not established (step S53; No), the acoustic control unit 6 ends the voice recognition mode (step S57). Subsequently, the acoustic control unit 6 stops the command extraction unit 7, transmits a stop command to the utterance detection control unit 33, and causes the utterance detection control unit 33 to stop the utterance detection processing unit 32 (step S58).

  When the wireless connection is established (step S53; Yes), the sound control unit 6 determines whether or not there is an operation command for a certain time (for example, 1 hour) (step S54). If no command is issued for a certain period of time (step S54; No), the sound control unit 6 ends the voice recognition mode (step S57). Subsequently, the acoustic control unit 6 stops the command extraction unit 7, transmits a stop signal to the utterance detection control unit 33, and causes the utterance detection control unit 33 to stop the utterance detection processing unit 32 (step S58).

  For example, if the user has not recognized that the voice recognition mode is set, or if the user has fallen asleep on the spot, the power is wasted if the voice recognition mode remains set. Therefore, the voice recognition mode is terminated, and the command extraction unit 7 and the speech detection processing unit 32 are stopped.

  On the other hand, when a command is issued during a certain time (step S54; Yes), the sound control unit 6 determines whether or not a voice recognition mode end command is issued by a remote control operation or an operation command (step S55). ). If the voice recognition mode end command has been issued (step S55; Yes), the acoustic control unit 6 ends the voice recognition mode (step S57). Subsequently, the acoustic control unit 6 stops the command extraction unit 7, transmits a stop signal to the utterance detection control unit 33, and causes the utterance detection control unit 33 to stop the utterance detection processing unit 32 (step S58).

  On the other hand, if the voice recognition mode end command has not been issued (step S55: No), the acoustic control unit 6 performs continuous setting of the voice recognition mode (step S56).

  After step S58 and step S56 are finished, the continuation determination process is finished.

  As described above in detail, according to the present embodiment, television apparatus 100 acquires an audio signal based on the collected audio and an audio signal received from mobile phone 200. Thereby, the operation command which the user uttered can be extracted using the better one or both of the two audio signals. In this way, the television apparatus 100 stops the instruction extraction unit 7 when no utterance is detected based on the timing signals (utterance start signal, utterance end signal) received from the mobile phone 200. I can leave. As a result, the remote operation of the main body of the television apparatus 100 by voice recognition or the like can be performed with high accuracy and low power consumption without performing troublesome operations by the user.

  Further, according to the present embodiment, the mobile phone 200 transmits a timing signal to the television apparatus 100 every time it detects the start and end of the user's utterance based on the collected audio signals. In this way, the television apparatus 100 that has received the timing signal can extract the operation command uttered by the user at an accurate timing, and does not require the command extraction unit 7 that extracts the operation command uttered by the user. It can be stopped at any time. As a result, the remote operation of the main body of the television apparatus 100 by voice recognition or the like can be performed with high accuracy and low power consumption without performing troublesome operations by the user.

  That is, according to the present embodiment, speech detection is performed by the mobile phone 200 placed near the user. Further, the extraction of the operation command by voice recognition or the like is preferable by using at least one of the voice signal based on the voice collected by the mobile phone 200 and the voice signal based on the voice collected by the television device 100. Or a combination of both. In addition, the command extraction unit 7 that extracts the operation command can be activated only during the period from when the utterance start signal is transmitted until when the utterance stop signal is transmitted. Thereby, the remote operation of the main body of the television apparatus 100 by voice recognition or the like can be performed with high accuracy and low power consumption without performing troublesome operations by the user.

  More specifically, in the present embodiment, speech recognition is performed using two speech signals collected at different positions. Thereby, more robust (strong against noise) speech recognition is possible.

  Moreover, noise elimination and voice recognition that require a large amount of power are performed by the television device 100 having relatively large power, and speech detection is performed by the mobile phone 200 placed near the user. For this reason, the speech recognition operation system 300 according to the present embodiment has an optimal system configuration from the viewpoint of accuracy and power.

  Further, by using the mobile phone 200 as an utterance detection device, the addition of hardware can be minimized.

  Further, according to the present embodiment, the presence of the user is detected by the human sensor. If there is no user, the command extraction unit 7 of the television device 100 and the speech detection processing unit 32 of the mobile phone 200 are stopped. Thereby, both power consumption can be reduced.

  Further, according to the present embodiment, in mobile phone 200, instruction extraction unit 7 in television apparatus 100 can be activated only while speech is detected, so that power consumption can be reduced. it can.

  In addition, according to the present embodiment, if the sound pressure level corresponding to the collected voice is not high for a certain period, the command extraction unit 7 is not activated, so that useless power consumption can be avoided. .

  In the present embodiment, the acoustic processing unit 5 and the utterance detection processing unit 32 stop operating in an operation mode other than the voice recognition mode. Thereby, power consumption can be further reduced.

  In this embodiment, when the mode is switched to the voice recognition mode, the sound pressure level threshold value for detecting the utterance is calibrated. Thereby, highly accurate voice recognition according to the surrounding situation becomes possible. Further, when the threshold value is calibrated, the display unit 3 displays a message to refrain from speaking. Thereby, calibration of the threshold value according to the surrounding situation can be performed more appropriately.

  In the present embodiment, when the wireless connection between the television device 100 and the mobile phone 200 is not established, the voice recognition mode is terminated. Thereby, since the instruction extraction unit 7 and the speech detection processing unit 33 are stopped, the power consumption can be further reduced.

  In the present embodiment, the speech recognition mode is terminated when no utterance is made for a certain period. Thereby, since the instruction extraction unit 7 and the speech detection processing unit 32 are stopped, the power consumption can be further reduced.

  In the present embodiment, when the content of the operation command is a voice recognition mode end command, the voice recognition mode is ended. Thereby, since the instruction extraction unit 7 and the speech detection processing unit 32 are stopped, the power consumption can be further reduced.

  Further, in this embodiment, in a series of instruction extraction processes for extracting operation instructions from sound, the television apparatus 100 is provided with functions such as a plurality of microphones 4, an echo canceller 22, a noise removal processing unit 23, and the like. Accurate speech recognition is possible.

  However, depending on the surrounding environment and noise conditions, noise may not be removed successfully. For example, when the position of the user and the television device 100 is very far away, the sound of the speaker 2 is very loud, and the SN ratio of the user's voice reaching the microphone 4 of the television device 100 is extremely small. Such a situation can happen.

  Therefore, in the present embodiment, the mobile phone 200 located near the user is used as an utterance detection device, and the audio data input from the microphone 31 is transmitted to the television device 100 and used to extract an operation command. . For this reason, an operation command can be acquired with a high S / N ratio.

  Since the television device 100 is complicated to operate with a remote controller, the user's work burden is significantly reduced if the operation by voice recognition is possible as in the present embodiment. However, the present invention can be applied to any AV device such as a tuner, an audio, and a recorder, and can be applied to home appliances other than the AV device.

  In the present embodiment, two types of audio data, that is, digital audio data acquired by the television apparatus 100 that has been subjected to echo cancellation and noise removal processing, and digital audio data of the mobile phone 200 having a relatively high S / N ratio are obtained. Can be acquired. Therefore, the voice recognition may be performed independently, and the voice recognition result may be obtained using the one with higher voice recognition accuracy.

  Which voice data is adopted can be based on the likelihood, which is a measure representing the certainty of each voice data. This likelihood is compared, and the higher likelihood may be adopted as the speech recognition result. For example, in a situation where the user is far away from the television apparatus 100 and the mobile phone 200 is placed on a desk or the like and speaking to the mobile phone 200, the voice data acquired by the mobile phone 200 is used. Since the likelihood is higher when it is used, high accuracy of speech recognition can be expected.

  Further, the microphone 31 on the mobile phone 200 side and the microphone 4 of the television device 100 may be combined and regarded as one microphone array, and noise removal processing may be performed. Of the noise removal processing methods using multiple microphones, noise removal using ICA (independent component analysis) is used, so noise removal is possible even if the microphone characteristics and microphone location prior information are unknown. Is possible. In this way, by performing speech recognition using speech data from which noise has been removed with high accuracy, a highly accurate recognition result can be expected.

  In the present embodiment, since the utterance detection process of FIG. 9 can be performed to detect the utterance time, the volume of the television device 100 can be automatically reduced during the utterance, By setting it to zero, it is possible to reduce the influence of the sound emitted from the television device 100 and to allow the sound emitted from the user to reach the microphone 31 with a high SN ratio.

  As described above, according to the present embodiment, each time an utterance is performed, even if the voice recognition switch is not pressed or the microphone 31 is not brought near the mouth, the user can simply speak the surroundings. It is possible to realize the operation of home appliances by high-accuracy voice recognition that is not affected by noise, and to realize the function with low power consumption.

  Note that the various times specified in the present embodiment (sound pressure detection section for speech detection, section in which the speech recognition mode is terminated by the human sensor, etc.) are examples, and are exemplified in the present embodiment. It is not limited to the value. In this embodiment, the digital audio data on the mobile phone 200 side is transmitted to the television apparatus 100 side. However, in order to further reduce power consumption of the mobile phone 200, the audio itself is not transmitted and the speech detection is performed. It can also be limited to signal transmission only. Furthermore, although the mobile phone 200 is used as an utterance detection device in the present embodiment, a module dedicated to utterance detection that realizes an equivalent function may be prepared.

  In the above embodiment, programs executed by the television device 100 and the mobile phone 200 are a flexible disk, a CD-ROM (Compact Disk Read-Only Memory), a DVD (Digital Versatile Disk), an MO (Magneto-Optical). A system that executes the above-described processing may be configured by storing and distributing the program in a computer-readable recording medium such as a disk) and installing the program.

  Further, the program may be stored in a disk device or the like of a predetermined server device on a communication network such as the Internet, and may be downloaded, for example, superimposed on a carrier wave.

  In addition, when the above functions are realized by sharing an OS (Operating System), or when the functions are realized by cooperation between the OS and an application, only the part other than the OS may be stored in a medium and distributed. You may also download it.

  In addition, this invention is not limited by the said embodiment and drawing. It goes without saying that the embodiments and the drawings can be modified without changing the gist of the present invention.

  The present invention is suitable for remote control of electronic devices such as AV devices.

  DESCRIPTION OF SYMBOLS 1 ... Television backbone part, 2 ... Speaker, 3 ... Display part, 4 ... Microphone, 5 ... Sound processing part, 6 ... Sound control part, 7 ... Command extraction part, 10 ... Operation control part, 11 ... Human sensor, 12 DESCRIPTION OF SYMBOLS ... Wireless communication part, 20 ... AD converter (ADC), 21 ... Recording area part, 22 ... Echo canceller, 23 ... Noise removal processing part, 24 ... Speech determination processing part, 25 ... Speech recognition processing part, 26 ... Voice recognition control , 27 ... DA converter (DAC), 30 ... mobile phone backbone, 31 ... microphone, 32 ... utterance detection processing part, 33 ... utterance detection control part, 40 ... wireless communication part, 41 ... AD converter (ADC), 42 ... Sound pressure level determination processing unit, 43 ... Recording area unit, 44 ... Timer, 100 ... Television apparatus, 200 ... Mobile phone, 300 ... Voice recognition operation system, 301, 302, 303, 304 ... State

Claims (25)

At least one sound collection unit that collects ambient sound and outputs a first sound signal corresponding to the sound;
A wireless communication unit for performing wireless communication with an external device;
Command extraction for extracting an operation command uttered by a user by performing signal processing including voice recognition processing on at least one of the first voice signal and the second voice signal received by the wireless communication unit And
An operation unit for operating the main body according to the extracted operation instructions;
Based on the timing signal received by the wireless communication unit, a control unit for controlling the start and stop of the command extraction unit,
Electronic equipment comprising. A human sensor for detecting the presence of the user in the surroundings;
The wireless communication unit is
Send the sensor output of the human sensor to the external device,
The controller is
Based on the sensor output of the human sensor, control the start and stop of the command extraction unit,
The electronic device according to claim 1. In the timing signal received by the wireless communication unit,
Includes an utterance start signal and an utterance end signal,
The controller is
When the utterance start signal is received by the wireless communication unit, the command extraction unit is activated,
When the utterance end signal is received by the wireless communication unit, the command extraction unit is stopped.
The electronic device according to claim 1, wherein the electronic device is an electronic device. A recording unit for storing the first audio signal;
In the timing signal received by the wireless communication unit,
A save command and a discard command for the recording unit are included,
The controller is
When the storage command is received by the wireless communication unit, the storage of the first audio signal to the recording unit is started,
When the discard command is received by the wireless communication unit, the first audio signal stored in the recording unit is discarded,
Controlling the command extraction unit to extract the operation command using the first audio signal stored in the recording unit;
The electronic device according to claim 3. It further includes an output unit for outputting sound,
The controller is
When the wireless communication unit receives the storage command, the volume of the sound output from the output unit is reduced.
The electronic device according to claim 4. The controller is
In an operation mode other than the voice recognition mode, the instruction extraction unit is stopped,
When a switching operation to the voice recognition mode is input to the operation unit,
Activating the command extraction unit and transmitting a switching signal to the voice recognition mode to the external device via the wireless communication unit,
The electronic device according to claim 1, wherein the electronic device is an electronic device. A display unit for displaying information;
The controller is
After starting the command extraction unit, a message to the effect that the user refrains from speaking is displayed on the display unit,
The electronic apparatus according to claim 6. The controller is
When a wireless connection with the external device is not established,
End the voice recognition mode and stop the command extraction unit,
The electronic apparatus according to claim 6 or 7, wherein The controller is
When a signal indicating that the start of speech has been detected by the wireless communication unit is not received for a predetermined period of time,
Ending the voice recognition mode, stopping the command extraction unit, and sending an end notification of the voice recognition mode to the external device via the wireless communication unit,
The electronic device according to claim 6, wherein the electronic device is an electronic device. The controller is
When the operation command is a command to end the voice recognition mode,
Ending the voice recognition mode, stopping the command extraction unit, and sending an end notification of the voice recognition mode to the external device via the wireless communication unit,
The electronic device according to claim 6, wherein the electronic device is an electronic device. The controller is
In the instruction extraction unit,
Causing both the first voice signal and the second voice signal to perform voice recognition processing;
Extracting the operation command using the one with the higher likelihood of the processing result of the voice recognition processing,
The electronic device according to claim 1, wherein the electronic device is an electronic device. A sound collection unit that collects surrounding sound and outputs a sound signal corresponding to the sound;
A wireless communication unit that performs wireless communication with an electronic device;
An utterance detection processing unit for detecting the start and end of the user's utterance based on the sound pressure level of the audio signal;
A control unit that transmits a timing signal to the electronic device via the wireless communication unit each time the utterance detection processing unit detects the start and end of the user's utterance;
An utterance detection device comprising: The controller is
Control activation and stop of the utterance detection processing unit based on the sensor output of the human sensor,
The utterance detection apparatus according to claim 12. The controller is
When the start of the user's speech is detected by the speech detection processing unit, the speech signal is transmitted to the electronic device together with the speech start signal as the timing signal via the wireless communication unit, and the speech detection processing When the end of the user's utterance is detected by the unit, the utterance end signal as the timing signal is transmitted to the electronic device via the wireless communication unit and the transmission of the audio signal is stopped.
The utterance detection apparatus according to claim 12 or 13, characterized in that: A recording unit for storing the audio signal;
The utterance detection processing unit
A sound pressure level determination processing unit that determines whether the sound pressure level of the audio signal exceeds a threshold;
The controller is
When it is determined by the sound pressure level determination processing unit that the sound pressure level of the audio signal has exceeded a threshold, the storage of the audio signal to the recording unit is started, and a storage command is transmitted to the electronic device,
When the sound pressure level determination processing unit determines that the sound pressure level of the sound signal has become equal to or lower than a threshold value after a predetermined period of time, the sound signal stored in the recording unit is discarded and a discard command is issued. Send to the electronic device,
When the sound pressure level determination processing unit determines that the sound pressure level of the audio signal has exceeded a threshold value after the predetermined period has elapsed, it is stored in the utterance start signal and the recording unit via the wireless communication unit. Transmitting the audio signal to the electronic device,
The utterance detection apparatus according to claim 14. The controller is
When the switching signal to the voice recognition mode is received from the electronic device via the wireless communication unit, the speech detection processing unit is activated.
The speech detection device according to any one of claims 12 to 15, wherein The controller is
After activating the utterance detection processing unit,
For a predetermined period, the sound pressure level determination processing unit determines whether or not the sound pressure level of the audio signal exceeds a threshold,
Adjusting the threshold so that a ratio of a period during which the sound pressure level of the audio signal with respect to the predetermined period exceeds the threshold is smaller than a predetermined ratio;
The utterance detection apparatus according to claim 16. The controller is
If a wireless connection with the electronic device is not established,
Stopping the utterance detection processing unit;
The utterance detection apparatus according to claim 16 or 17, The controller is
When the end notification of the voice recognition mode is received from the electronic device via the wireless communication unit, the speech detection processing unit is stopped.
The utterance detection device according to any one of claims 16 to 18. An electronic device according to any one of claims 1 to 11,
The utterance detection device according to any one of claims 12 to 19,
A voice recognition operation system comprising: The electronic device is a television device;
21. The voice recognition operation system according to claim 20, wherein The utterance detection device is a mobile phone;
The voice recognition operation system according to claim 20 or 21, wherein A voice collecting step of simultaneously collecting ambient sounds by an utterance detection device and an electronic device placed near the user;
An utterance start detection step of detecting the start of the user's utterance based on an audio signal corresponding to the collected audio using the utterance detection device;
A first transmission step of transmitting an utterance start signal from the utterance detection device to the electronic device by wireless communication when the utterance start detection step detects the start of the user's utterance;
An activation step of activating an instruction extraction unit that extracts an operation instruction uttered by a user from an audio signal input in the electronic device according to the received utterance start signal;
An extraction step of extracting the operation command by the command extraction unit from a voice signal corresponding to the collected voice;
An utterance end detection step of detecting the end of the user's utterance based on an audio signal corresponding to the collected audio using the utterance detection device;
A second transmission step of transmitting an utterance end signal from the utterance detection device to the electronic device by wireless communication when the utterance end detection step detects the end of the user's utterance;
When the electronic device receives an utterance end signal, a stop step of stopping the command extraction unit;
An operation step of operating the main body according to the extracted operation instructions;
Voice recognition operation method including. The computer that controls the electronics
At least one voice collecting means for collecting ambient voice and outputting a first voice signal corresponding to the voice;
A wireless communication means for performing wireless communication with an external device;
Command extraction means for performing signal processing including voice recognition processing on at least one of the first voice signal and the second voice signal transmitted from an external device to extract an operation command uttered by the user; ,
Operating means for operating the main body according to the extracted operation instructions;
Based on a timing signal transmitted from the external device, control means for controlling activation and stop of the instruction extraction means,
Program to make it work. A computer that controls an utterance detection device that detects a user's utterance,
Sound collecting means for collecting surrounding sound and outputting sound signals corresponding to the sound;
Wireless communication means for performing wireless communication with an electronic device;
Utterance detection processing means for detecting the start and end of the user's utterance based on the sound pressure level of the audio signal;
Control means for transmitting a timing signal to an electronic device each time the utterance detection processing means detects the start and end of the user's utterance;
Program to make it work.

Images