JP2006215418A - Voice input device and voice input method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice input device with which uttering of the will of a user can surely be recognized under very adverse noisy environment and operability is made great and to provide a voice input method. <P>SOLUTION: The voice input device is provided with a voice inputting section 103 which converts inputted voice into audio signals and outputs them, a voice recognition section 105 which converts the audio signals outputted from the voice input section 103 into information signals and an uttering possibility judging section 102 that judges the time interval in which voice inputting of the user is to have high possibility. In the time interval in which possibility of voice inputting by the user is judged to be high by the uttering possibility judging section 102, inputting of the voice of the user is converted by the voice inputting section 103 into audio signals and outputted and the audio signals are converted into information signals by the voice recognition section 105 for a recognition process. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a voice input device and a voice input method.

  2. Description of the Related Art In recent years, some automobiles are equipped with a voice input / recognition device and can operate a navigation device, an audio device, an air conditioner, and the like by voice. Such a voice input device is generally provided with a microphone that takes in a voice signal, and is configured to input and recognize a user's voice. An example of means for recognizing and processing such uttered speech is described in Patent Document 1 below.

  In a general voice input device, for example, it is configured to always wait for voice input, detect a user's speech voice section using the power value of the input voice (the square value of the amplitude of the input signal), etc., and perform recognition. However, if such a configuration is used in the interior of an automobile, not only the audio signal but also the operating sound of the engine and air conditioner of the vehicle, the audio sound, the wind noise associated with traveling, the road noise associated with the contact between the tire and the road surface, etc. Because many noises are input simultaneously, recognition errors increase.

  Therefore, a voice input device mounted on a vehicle often uses a method in which a user sends an input start signal to the voice input device immediately before utterance to accurately specify a voice section to be recognized.

  As examples of the above system, a PTT (Push-To-Talk) switch system, a PTA (Push-To-Activate) switch system, and the like are known. In the PTT method, the input speech for the time interval in which the button is pressed is subject to speech recognition. In the PTA method, the period from the time when the button is pressed until the pause (no voice period) continues for a predetermined time or longer is set as the target of voice recognition. As a method similar to this, a method of using a specific keyword input as a function equivalent to the pressing operation of the PTA switch has been proposed.

  Hereinafter, the means for transmitting the voice input start to the system is referred to as “speech switch”.

JP-A-9-288493

  In the voice input device using such a speech switch, there is a problem that the user has to perform an operation such as pressing the switch every time voice input is performed, and the operation is troublesome.

  In addition, the process of pressing the utterance switch after the user makes a decision to utter and then starting the utterance increases the time required to complete the operation, which is inconvenient for the user. There is.

  A function that teaches feedback such as “Please give a command (pip)” after pressing the utterance switch, or “Please speak if you hear a beep (pip)” (“beep” is a notification sound), etc. It is conceivable that the required time is further increased in an apparatus having a configuration that teaches the above.

  In order to solve the above problems, if a system that always waits for input is used, the number of misrecognitions increases remarkably, especially in the interior of a vehicle in a poor noise environment, and noise is treated as an operation command despite no speech. As a result of interpretation, there is a case where the operation of the device is executed regardless of the intention of the user.

  The present invention has been made in view of the above problems, and the problem to be solved by the present invention is that the user's intention is recognized with high accuracy even in a poor noise environment. To provide a voice input device and a voice input method.

  The speech possibility determination means determines a time interval in which the user's voice input is likely to be performed, and the sound input means converts the voice spoken by the user in the time interval into a voice signal and outputs the voice signal. The speech input device is characterized in that the speech recognition means converts the signal into an information signal.

  By implementing the present invention, it is possible to provide an easy-to-use voice input device and voice input method for recognizing speech uttered by a user's intention with high accuracy even in a poor noise environment.

  Hereinafter, embodiments of the present invention will be described.

(Basic functions and implementation methods)
A voice input device according to the present invention includes a sound input unit that converts an input voice into a voice signal and outputs the voice signal, and a voice recognition unit that performs a recognition process of converting the voice signal output from the sound input unit into an information signal. Utterance possibility determination means for determining a time interval in which the user's utterance is likely to be performed, and the time when the utterance possibility determination means determines that the user's utterance is likely to be performed The voice input means converts the voice input by the user during the section into a voice signal and outputs the voice signal, and the voice recognition means converts the voice signal into an information signal. With such a configuration, the user can perform voice input without operating the speech switch.

  Further, in the voice input device according to the present invention, the utterance possibility determination unit may perform voice input by the user, for example, by detecting a change in the operating state of a device to be operated by voice. A high time interval is determined, and the speech uttered by the user in that time interval is converted into an audio signal by the sound input means and output, and the voice recognition means converts the audio signal into an information signal. With this configuration, the user can perform voice input for operating the device without operating the speech switch.

  In addition to the basic configuration described above, there is a possibility that the user's utterance may be performed by the recognition dictionary including the language dictionary used for collation with the voice signal output from the sound input unit and the utterance possibility determination unit. If a vocabulary that is likely to be uttered in a time interval judged to be high is selected and a language dictionary changing means for changing the language dictionary based on the vocabulary is added, the efficiency and accuracy of speech recognition by the speech recognition means And can be improved. In this case, the language dictionary changing means, as a vocabulary that is likely to be spoken, is a voice operation vocabulary related to the device that is the target of the voice operation, for which a change in the operating state is detected (for operating the device). If the vocabulary to be uttered is selected, the efficiency and accuracy of speech recognition when performing speech input for operating the device can be improved.

  Since the voice input device and the voice input method according to the present invention are particularly effective in the interior of a car, in the following description, the voice input device and the voice input method according to the present invention are used in a car interior. Assume a case.

  When considering the operation of various devices (navigation devices, audio devices, air conditioners, etc.) installed in the interior of an automobile by a passenger user, the user can operate the device by using a change in the operating state of the device. Many scenes can be considered. Such scenes can be selected, for example, on CDs, MDs, DVDs, etc. at the time of title, chapter change, fast forward, rewind, skipping, radio, television program change, radio wave condition change point, etc. Station operation, VICS (road traffic information communication system) data reception or after information display information display, redisplay, reading, route recalculation operations, communication information (charge, usage) when ETC communication is established or after communication is completed (History) display, reading operation, receiving operation when receiving a call in a hands-free system of a mobile phone, temperature resetting operation when the automatic air conditioner operating state (air flow level) changes, TV, radio when receiving time signal information Operation.

  That is, it can be considered that the user is highly likely to operate the device in a predetermined time interval including the change time of the operation state of the device.

  If the above trend is replaced with device operation by voice input, it can be interpreted that the time interval including the time when the operation state of the device changes is a time interval in which the possibility of voice utterance for operating the device is high. it can.

  In the present invention, paying attention to the above-mentioned tendency, the possibility of utterance for device operation is predicted by the change in the operating state of the on-vehicle device, and the utterance switch is not operated for a time interval in which the possibility of utterance is high. The voice input device and the voice input method that can be operated by voice input are configured.

(Embodiment example)
FIG. 1 shows a basic configuration of a voice input device according to the present invention, and FIG. 2 shows a basic embodiment of the voice input device according to the present invention.

  In FIG. 1, an in-vehicle device 101 is a group of various operation devices installed in a vehicle interior. In the present embodiment, in the in-vehicle device 101, the devices to be operated are storage medium playback devices such as CD, MD, DVD, broadcast wave receiving devices such as televisions and radios, telematics systems (navigation system, VICS system). , ETC system, hands-free phone etc.), automatic air conditioner, auto light (indoor / outdoor), auto lock, auto wiper, clock, etc.

  The utterance possibility determination unit 102 as utterance possibility determination means includes the arithmetic device 204, the storage device 205, the timer 206, and the sensor 207 in FIG. As the arithmetic unit 204, a combination of general operation circuits such as a CPU, MPU, DSP, and FPGA is used. As the storage device 205, a general storage medium such as a cache memory, main memory, HDD, CD, MD, DVD, optical disk, FDD, or the like is used.

  The utterance possibility determination unit 102 monitors a change in the operating state of the in-vehicle device 101. When the change is detected, the utterance possibility determination unit 102 determines a predetermined time interval including the time when the change is detected as a user's utterance regarding the operation of the device. The voice input unit 103 converts the voice uttered by the user in this time interval into a voice signal, outputs it, and sends it to the voice recognition unit 105. To do. Along with this, a predetermined utterable time interval measured by the timer 206 (a time interval in which it is determined that the user's utterance is likely to be performed, and the voice uttered by the user within this time interval is converted into an audio signal. The voice recognition unit 105 performs a recognition process for converting the voice signal into an information signal.

  After the above recognition process is completed, the operations of the sound input unit 103 and the voice recognition unit 105 are stopped until the user can speak again. However, if the operation of only one of the sound input unit 103 and the voice recognition unit 105 is stopped, the recognition process in the voice recognition unit 105 is stopped as a result. With such a configuration, it is possible to eliminate the possibility of erroneous recognition due to input of noise or the like outside the utterable time interval.

  The sound input unit 103, which is a sound input unit, includes the microphone 201, the amplification device 202, the AD conversion device 203, and the calculation device 204 in FIG. 2, and the user's utterance voice (see FIG. 1 arrow (a)) is converted into a voice signal and output, and input to the voice recognition unit 105. A general microphone can be used as the microphone 201. The sound input unit 103 validates the input when the utterance possibility determination unit 102 instructs, and when the utterance switch 108 is pressed, converts the input voice into a voice signal, and outputs the voice signal for voice recognition. Send to unit 105. However, if the voice recognition unit 105 operates only in these cases, the sound input unit 103 may continue to operate.

Note that the sound input unit 103 simultaneously outputs noise signals generated from inside and outside the vehicle, sound signals output from the navigation system, and the like in addition to the sound signals. Accordingly, a filter (anti-aliasing filter) is provided between the microphone 201 and the AD conversion device 203 or between the AD conversion device 203 and the arithmetic device 204 that performs speech recognition to weaken non-target components in the sound signal. A voice adjustment (Voice Activity) that adjusts the amount of amplification so that the power (gain) of the input signal is appropriate by providing a gain adjustment mechanism, and accurately cuts out the section that contains the voice in the input signal based on the power change of the input signal, etc. Detection
: Eliminate or weaken noise and sound input signals that are not subject to speech recognition through measures such as providing a (VAD) mechanism and an echo canceling mechanism that cancels sound signals output from audio systems and navigation systems. Is desirable.

  The dictionary changing unit 104, which is a language dictionary changing unit, includes the arithmetic device 204 and the storage device 205 in FIG. 2, and based on the determination result of the utterance possibility determining unit 102, a vocabulary that is highly likely to be input as the next utterance. The language dictionary used for collation with the audio signal output from the sound input unit 103 as sound input means is changed so that the vocabulary can be easily recognized. A specific example of the changing method will be described later. This language dictionary is a component of the recognition dictionary 106.

  The speech recognition unit 105, which is a speech recognition means, includes the arithmetic device 204 and the storage device 205 in FIG. 2, and the acoustic features stored in the recognition dictionary 106 for the speech signals output from the sound input unit 103, and linguistic features Comparison and matching are performed using features, and one or more likely (highly likely) vocabularies are acquired as recognition results. As a general feature amount, time-series vector data combining LPC cepstrum, LPC delta cepstrum, mel cepstrum, logarithmic power, and the like is used for comparison / collation. That is, the speech recognition unit 105 as speech recognition means performs a recognition process for converting the speech signal output from the sound input unit 103 into a vocabulary that is an information signal.

  The recognition dictionary 106 corresponds to the storage device 205 in FIG. 2 and is composed of an acoustic dictionary and a language dictionary. The former records the acoustic features of the language to be collated with the above time-series vector data, for example, in the form of a Hidden Markov Model (HMM), and the latter is a voice input. The vocabulary that can be accepted by the system is recorded, for example, as a word connection (word network). Note that the dictionary changing unit 104, which is a language dictionary changing unit, changes the language dictionary so that a vocabulary with a high utterance possibility is easily recognized.

  The operation command issuing unit 107 includes the arithmetic device 204 and the storage device 205 in FIG. 2, interprets the vocabulary content recognized by the voice recognition unit 105, converts it into operation commands for various devices, and directs the operation target device The operation command is sent out (arrow (b) in FIG. 1). The operation target device can be operated by operation means other than voice (for example, buttons and touch panels arranged on various devices), and information (button press information, etc.) from these operation means is also operated in the same manner as the voice recognition result. It is converted into an operation command by the command issuing unit 107 and sent to various target devices.

  The utterance switch 108 is a switch that is pressed by the user to notify the system of the start of voice input, and corresponds to the switch 208 in FIG. Except when the utterance possibility determination unit 102 determines that the utterance possibility is high, the user starts the utterance after pressing the utterance switch 108 during the utterance.

(Dictionary structure and recognition operation example under normal conditions)
FIG. 4 shows a language dictionary example in the recognition dictionary 106 in the present embodiment.

  The language dictionary shown in FIG. 4A is an example of a network type language dictionary, and forms a hierarchical network. That is, the words in the lower layer have a relationship that refines the word category in the upper layer, and the words in the upper layer and the lower layer can be input for each word or connected. For example, in the dictionary of FIG. 4, it is possible to recognize “destination”, “destination, Kanazawa-ku, Yokohama-shi, Kanagawa”, “TV, Δ × broadcast”, and the like.

  The language dictionary shown in FIG. 4B is a similar network type language dictionary, but a dictionary is provided for each vocabulary category, and these are recognized independently or in parallel as targets for collation. The recognizable vocabulary is the same as (a) in FIG. In the language dictionary shown in FIG. 4 (b), only a single dictionary from “menu dictionary 401” to “..Dictionary 405” can be validated. It is also possible to wait for the included vocabulary in parallel. However, since the recognition rate decreases as the number of vocabularies increases, for example, the menu dictionary 401 (shown in (b) of FIG. 4) is set as an initial standby dictionary. From this result, if the recognized vocabulary is “destination” It is desirable to determine an appropriate usage from the relationship between the number of vocabularies and the recognition performance, such as enabling the destination dictionary 402 (shown in FIG. 4B).

  In the case of excluding batch input (when inputting several words at a time), it is desirable to provide feedback to the user by voice or display as appropriate.

  The following shows an example of interaction between the system and the user when the dictionary shown in FIG. 4 (a) or (b) is used. However, this example is an operation example in a state where it is not determined that the possibility of speech is high (hereinafter referred to as “normal state”). In this case, the user presses the utterance switch 108 to place the voice recognition system in a state waiting for voice input. In the following, U: indicates a user's utterance and S: indicates a system response.

(Dialogue example 1: Destination setting input)
U: (Press utterance switch 108)
S: Command
U: Destination
S: Please give me your destination address
U: Yokohama City, Kanagawa Prefecture
S: Where is Yokohama
U: Rokuura, Kanazawa-ku
S: The destination is Rokuura, Kanazawa-ku, Yokohama, Kanagawa. Is it OK.
U: Yes
S: Route through National Highway No. Drive according to traffic regulations.

(Dialogue example 2: TV channel selection input)
U: (Press utterance switch 108)
S: Command
U: TV
S: Please select a channel
U: △ × Broadcast
That is, in the normal state, the user activates the voice input of the system and starts the voice input by pressing the speech switch 108 in the normal state.

(Dictionary changes and recognition operations in the automatic input state)
On the other hand, in the time interval in which the utterance possibility determination unit 102 detects a change in the operating state of the device and determines that the utterance possibility is high, voice input is validated without pressing the utterance switch 108 (hereinafter referred to as `` automatic input ''). Referred to as "state"). A specific example of a change in the operating state of the device and a method for determining a time interval for validating voice input will be described later.

  In synchronization with this activation processing, the dictionary changing unit 104 is likely to utter the user's speech operation vocabulary (vocabulary uttered to operate the device) related to the device whose operation state has changed. The language dictionary in the recognition dictionary 106 is selected as a vocabulary (next utterance candidate vocabulary) that is likely to be uttered in the time interval determined to be such that the vocabulary is easily recognized based on the next utterance candidate vocabulary. To change dynamically.

  Examples of such dynamic changes are shown in FIGS.

  FIG. 5 shows an example of hierarchy movement. The next utterance prediction vocabulary (next song,..., Song information) is set to the upper hierarchy (first hierarchy) with respect to the original dictionary shown in FIG. ) Moved to dictionaries.

  FIG. 6 shows an example of excluding the utterance prediction vocabulary from the recognition vocabulary. All the vocabularies other than the next utterance candidate vocabulary are invalidated from the original dictionary shown in FIG. It is a dictionary that recognizes only utterance candidate vocabulary (next song, ..., song information).

  FIG. 7 shows an example of adding a bias of appearance / transition probabilities. The next utterance prediction vocabulary (next song,..., Song information) is added to the original dictionary shown in FIG. The dictionary is modified so that the likelihood is increased by giving a bonus to the occurrence probability / transition probability.

  FIG. 8 shows an example of changing the activation dictionary. FIG. 8A shows an example of using the dictionary shown in FIG. 4B in the normal state. In the state, only the “menu dictionary” is activated, and only the vocabulary included in the dictionary can be recognized.

  On the other hand, for example, when a change is detected in the operation state of the television image detector and the automatic input state is entered, a dictionary including the next utterance candidate vocabulary (“TV dictionary” in FIG. 8B). Is activated so that only the vocabulary contained in the dictionary can be recognized.

  By dynamically changing the dictionary state in this way, the next utterance candidate vocabulary can be easily recognized.

  An example of dialogue when the dictionary is changed as shown in FIG.

(Corresponding to dialog example 2 above)
U: △ × Broadcast
S: △ × Broadcasting As can be seen from this interactive example, in the automatic input state, the operation can be completed in a short time without pressing the speech switch 108.

(Basic operation example (voice input validation and dictionary change flow))
The main operation of this embodiment will be described with reference to the flowchart of FIG.
Step S101: The operation information of the in-vehicle device 101 is acquired.
Step S102: obtaining the current time _{T n.}
Step S103: Monitor device operation information change (change from previous time T _n-1 ),
When a change is not detected, it progresses to step S104, and when a change is detected, it progresses to step S107. In _order to monitor the change in the device operation information from the previous time T _n-1 to the current time T _n , the device operation information is buffered for a predetermined time (data is stored in the temporary storage memory and sequentially read out). Good.

Step S104: If the speech switch 108 has been pressed, the process proceeds to step S105, and if not, the process returns to step S101.
Step S105: Voice recognition processing by the voice recognition unit 105 (FIG. 1) is performed. In this case, the recognition operation is performed in a normal state where the language dictionary is not changed.
Step S106: An operation command is issued by the operation command issuing unit 107 (FIG. 1) based on the recognition result.

Step S107: The language dictionary is changed by the dictionary changing unit 104 (FIG. 1) so that the vocabulary (speech operation vocabulary) related to the voice operation of the device is easily recognized based on the type of the device in which the change in the operation state is detected.
Step S108: The voice input by the sound input unit 103 (FIG. 1) is validated, and the input to the voice recognition unit 105 (FIG. 1) is started.
Step S109: The timer 206 (FIG. 2) is initialized to zero. The elapsed count from the time when voice input is enabled is started.
Step S110: If pressing of the utterance switch 108 is detected, that is, if the utterance switch 108 is further pressed while the voice input is automatically enabled, the process proceeds to step S111, and the utterance switch 108 is not pressed. If yes, go to Step S112.

  Step S111: Cancel the change of the language dictionary, return to the original dictionary configuration (illustrated in FIG. 4), and proceed to Step S105.

  Step S112: The presence / absence of voice input is detected. As a detection method, a method such as monitoring the power of the input signal can be used. If a voice input is detected, the process proceeds to step S114. If a voice input is not detected, the process proceeds to step S113.

  Step S113: The timer time, that is, the elapsed time since the voice input is validated is compared with a predetermined time β (corresponding to the “second time” according to claims 3 and 13). If the timer time ≧ β, the process proceeds to step S116.

  As a result, when the time at which the change in the operating state of the device to be operated by voice is detected is set as the reference time, a second time (β) that is determined in advance from the reference time than the reference time. It is determined that the time interval up to the second time later is a time interval in which the user's voice input is highly likely to be performed, and during that time interval, a language dictionary with a high probability of being used for speech recognition It is prepared (in step S107) and the state of recognizing the user's voice input is maintained.

  Here, β determines how many seconds the voice input is validated after detecting a change in the operating state of the device. The value of β is determined in advance before step S108.For example, it is determined by the user to use a fixed value (for example, 5 seconds). The user's utterance timing can be learned and adjusted so as to match this tendency.

Step S114: Perform voice recognition processing. In this case, the recognition operation is performed in an automatic input state in which the language dictionary is changed based on the device operation change. After the recognition operation, the process proceeds to step S115.
Step S115: Based on the recognition result, an operation command is issued by the operation command issuing unit 107 (FIG. 1), and the process proceeds to step S116.
Step S116: The voice input is invalidated and the input of the sound signal to the voice recognition unit 105 is stopped. However, instead of stopping the input of the sound signal to the voice recognition unit 105, the operation of the voice recognition unit 105 may be stopped. That is, after the recognition processing by the voice recognition unit 105 is completed, until the utterance switch 108 is pressed and until there is a possibility that the user will speak again, the sound input unit 103 and the voice recognition unit The operation of at least one of 105 may be stopped. In this way, the speech recognition processing is not performed in the time interval from when the recognition processing by the speech recognition unit 105 is completed until the user tries to input speech again, and erroneous recognition due to the influence of noise or the like. The possibility of this can be reduced.
Step S117: The dictionary changing unit 104 cancels the change of the language dictionary, and returns the language dictionary changed in step S107 before the above time interval to the state before change, that is, the normal state dictionary (illustrated in FIG. 4). This is a procedure corresponding to the case where the utterance in the normal state is performed next.

  This series of processing functions to recognize the input voice after pressing the utterance switch 108 in the normal state, while a predetermined time interval (length β) is detected when a change in the operating state of the device is detected. Thus, the automatic input state is entered, and the operation voice of the device can be input without pressing the speech switch 108.

(Difference in usage between skilled and unskilled personnel)
In the above flow, when the utterance switch 108 is pressed in the automatic input state, the change of the dictionary is canceled (the language dictionary is returned to the normal state dictionary). This is a response to the user who understands the operation of the present embodiment. In other words, since the utterance switch 108 is intentionally pressed in the automatic input state (a state in which an operation utterance of a specific device can be input without pressing the utterance switch 108), it is considered that the user intends to perform the normal operation. ing. In addition, since it can be considered that there is an intention to perform operations other than the operation of a specific device that has changed, the language dictionary should be changed so that other than the operation vocabulary related to the device in which the operation change is detected becomes effective. Also good.

  On the other hand, an unskilled user, that is, a user who does not know a function capable of automatic input, may press the speech button in the automatic input state. As a response to such users, for example, recognition processing and command issuance in a normal state, and if the recognition result is an operation of a device whose operation state has changed, the output of audio or video will be performed later. Thus, it is desirable to teach the mechanism of the present invention. As a result, the user can more smoothly perform input accompanying the change in operation of the device from the next time.

  The following example shows a more specific method for determining the possibility of speech.

  In the voice input device according to the present invention, when the device is installed in an automobile, a change in the operating state of the equipment provided inside and outside the vehicle interior is used as a method for detecting a time interval with a high probability of speaking. . Hereinafter, a speech possibility detection method in a specific vehicle interior device will be described.

(Example 1)
In this embodiment, the device to be operated by voice is a device that reproduces a sound signal or a video signal stored in a storage medium, detects a change in the operating state of the device, and uses the result based on the result. The structure which determines the time interval judged that a person's utterance is likely to be performed is shown. Here, a device that reproduces a sound signal or video signal stored in a storage medium is a device that reproduces a sound signal or video signal stored in a storage medium such as a CD, MD, DVD, HDD, or flash memory. Point to.

  At this time, the utterance possibility determination unit 102 determines the utterance possibility from the time when the operation state of the device changes. The time at which the change in the operating state of the device is detected is the time at which the change in the upper storage medium or the break of the sound signal or the video signal stored in the storage medium is detected. Here, the change of the storage medium corresponds to, for example, replacement of a CD (storage medium) when the apparatus includes a CD changer, and the separation of the sound signal or the video signal stored in the storage medium is, for example, This corresponds to a change in the reproduction data such that the reproduction of the music or video stored in the storage medium shifts to the reproduction of the video of the next music or the next chapter, or the reproduction reaches the end point of the stored signal.

  Such a change can be detected as, for example, a change in power or rhythm in the sound data, or a change in the difference between the previous and next frames in the video signal. Further, when accompanying information (called metadata or the like) is stored in the storage medium, it is desirable to use this to acquire the time at which the operation changes. The metadata includes the total recording time of the storage medium, the start / end time of the song / chapter, the song title, song genre, release date, artist name, chapter caption information, and the like.

  The above-described flowchart of FIG. 3 shows a configuration in which it is determined that there is a high possibility of utterance for a predetermined time interval (length β) starting from the change time. This is shown in (a) of FIG. 9, taking as an example the case where the present embodiment reproduces music.

The waveform shown in the upper part of FIG. 9 shows a part of the sound signal, and at the change point A, the music is switched to the next music. The switching of music, which is a break of such a sound signal, is detected when the sound signal becomes 0 level within a predetermined time. The time at which such a change is detected (corresponding to the “reference time” according to claims 3 and 13) is defined as T _c, and the time interval from T _c to (T _c + β) (from the reference time to the second time) It is determined that the possibility of utterance is high in the time interval (indicated by checkered stripes in the figure).

On the other hand, in (b) of FIG. 9, it goes back from the change time T _c by a predetermined time α (corresponding to “first time” according to claims 3 and 13), and (T _c −α) to (T _c + Β) is determined to have a high utterance possibility in the time interval (the time interval from the first time to the second time, indicated by the checkered pattern in the figure). The change time _Tc can be acquired in advance by a method such as reading the metadata first. When the time has already passed the time _Tc- α, ( _Tc- α) to The acquisition of the utterance in the time interval of (T _c + β) (the time interval from the first time to the second time) is continuously possible by buffering the utterance for a predetermined time. In this case, the sound input unit 103 continues to operate, and it is necessary to continuously buffer the audio signal output by the sound input unit 103 for a predetermined time. The setting method of α, β is fixed time, can be changed by the user, is adjusted according to the usage history (learning of utterance timing), and is changed based on the content of the song (for songs with a long intro part) For example, extending the time interval).

Incidentally, T _c -α~T _c time interval (time interval reaching the reference time from the first time), the utterance of the user may be a high time interval is performed.

  If a change in the operating state is detected, the language dictionary is changed simultaneously with the transition to the automatic input state. For example, the language dictionary is changed so that the vocabulary related to the movement of a song (chapter) back and forth (heading / skip), song information inquiry, next disc, song name search, artist name search, etc. is recognized with priority.

  Furthermore, it is desirable to provide means for notifying the user of the automatic input state. FIG. 10 shows an example of a display screen when the operation state changes during music reproduction. 10A shows a screen in the normal state, and FIG. 10B shows a screen in the automatic input state by detecting a change in the operating state. In FIG. 10B, the voice command (operation of the device by voice) and the type of operation are displayed.

  You may make it sound a notification sound synchronizing with the screen transition to (b) of FIG. At this time, if the configuration is such that a notification sound is prepared for each device so that the notification sound corresponds to the type of device that can be automatically input, the teaching becomes easier to understand for the user.

  With the above configuration, the user can perform an operation utterance of the device without pressing the utterance switch 108.

(Example 2)
In this embodiment, the device to be operated by sound is a broadcast wave receiving device that receives and outputs a sound signal or a video signal transferred from the outside, and the operating state of this broadcast wave receiving device is The structure which detects a change and determines the time interval judged that possibility that a user's speech will be performed highly based on the result is shown. Here, the broadcast wave receiving device refers to, for example, a television receiver, a radio receiver, or the like.

  In this embodiment, as a change in the operating state of the device to be operated by voice, a received sound signal or video signal break (for example, a change in received program), or a sound signal transferred from the outside or A change in the reception intensity of the video signal is used to determine a time interval in which the user's voice input is likely to be performed. That is, a change in received program (start / end of program, CM insertion, etc.) and a change in received intensity are regarded as a change in operation state, and the possibility of speech is determined.

  The change in the program information can be detected using a change in the power of the received sound signal, a change in the reception channel (stereo / monaural), a change in the difference between frames before and after the video signal, and the like. If information (metadata) about a broadcast program can be acquired, it is desirable to use it. The metadata includes the start / end time of the program being broadcast, the start / end time of the currently playing music, the description information of the program and music, and the like. The detection of the possibility of utterance based on the change in the program information is the same as in the first embodiment.

  On the other hand, the reception intensity constantly changes in a broadcast wave receiving device mounted on a mobile body (in this case, an automobile). Therefore, a predetermined threshold is provided, and it is considered that the operation has changed when the reception intensity is lower than the threshold.

  FIG. 11 shows an example of detecting the possibility of speech accompanying a change in reception intensity. The broken line shown in the upper part of FIG. 11 shows the change in reception intensity with time. In this example, the section from time A to A ′, the section from time B to B ′, and the section from time C to C ′. In the section from time D to D ′ and the section from time E to E ′, the reception intensity is lower than the threshold value TH. Let each section be A, B, C, D, E using the same reference numerals as the left end of the section. By the way, at time C ′, it is shown that the reception intensity is increased as a result of the user changing the channel. At time E ′, the user ends the use of the broadcast wave receiving device, and thereafter It shows a state of not receiving (corresponding to a case where the user switches to a device that reproduces a sound signal or a video signal stored in a storage medium).

  In FIG. 11A, it is determined that the utterance possibility is high in all the sections A to E (indicated by checkered stripes in the figure). Therefore, the time interval in which the reception sensitivity falls below the threshold value is always in the automatic input state, and it is possible to input the operation utterance of the broadcast wave receiving device.

  On the other hand, (b) of FIG. 11 provides a predetermined standby time T (shown in gray in the figure), and the elapsed time after the reception sensitivity falls below the threshold value TH exceeds the standby time T (in the figure, It is judged that the utterance possibility is high only for (indicated by checkered stripes). In this case, the sections A, B, and D do not shift to the automatic input state because the sections are shorter than the standby time, and the sections C and E are between the times C ″ to C ′ and the times E ″ to Transition to the automatic input state between E ′.

  In synchronization with the transition to the automatic input state, the language dictionary is changed so that the vocabulary related to operations such as channel selection, acquisition of program information, and TV / radio OFF can be easily recognized.

  Furthermore, it is desirable to provide means for notifying the user of the automatic input state. An example of screen transition in the present embodiment is shown in FIG. FIG. 12A shows a screen in a normal state, and FIG. 12B shows a screen in the case of shifting to an automatic input state because the reception intensity is low (the state of a sandstorm screen). On the screen of FIG. 12B, the voice command (operation of the device by voice) and the type of operation are displayed.

Example 3
In this embodiment, the device to be operated by voice is a vehicle-mounted telematics device that is a communication device that receives information by communication means, detects a change in the operating state of the device, and based on the result, the user The structure which determines the time interval judged that possibility that utterance of will be performed is high is shown. Here, the telematics device refers to, for example, a navigation system, a DSRC (narrow area communication) system, a hands-free system, and the like.

  The time at which the change in the operating state of the communication device is detected is the time at which communication between the communication device and an external communication device other than the communication device is established, or the time at which information from the external communication device arrives. Suppose there is. That is, information exchange by communication of a telematics device or the like is regarded as a change in operation state, and it is determined that the possibility of utterance increases at the time of the change. As specific changes in the operating state, acquisition of traffic jam information, establishment of ETC communication, incoming calls to the hands-free system, and the like can be considered. When such a situation is detected, it is determined that there is a high possibility of speaking for a predetermined time from the detection time, and an automatic input state is set. Alternatively, for a device configured to display the information and output a notification sound immediately after the transmission / reception of information, such as VICS information or ETC information, when a certain time interval elapses from the output of the information and the information output is terminated. It is good also as a structure which transfers to an automatic input state about the predetermined time area from.

  Synchronized with the transition to the automatic input state, for example, the language dictionary has been changed so that vocabulary related to traffic information and ETC fee information display (redisplay), reading operations and hands-free system receiving operations can be easily recognized. The

  Furthermore, it is desirable to provide means for notifying the user of the automatic input state. FIG. 13 shows an example of screen transition when VICS information is received. 13A shows a screen in a normal state immediately after acquiring VICS information, and FIG. 13B shows a screen in a case where a certain time interval has elapsed from the display and the state is shifted to an automatic input state. . In FIG. 13B, a voice command (operation of the device by voice) and the type of the operation are displayed.

Example 4
In this embodiment, the device to be operated by voice is a device whose operation state is actively controlled based on input information from inside or outside the vehicle, and automatically from information such as sensors provided inside and outside the vehicle. FIG. 2 shows a configuration in which a device whose operation state is controlled is detected, a change in the operation state of the device is detected, and a time interval in which it is determined that a user's speech is likely to be performed is determined based on the result. . The devices whose operation state is actively controlled based on input information from the vehicle interior or the exterior of the vehicle include, for example, an automatic air conditioner, an auto light (indoor / outdoor), an auto lock, an auto wiper, a clock, and the like. For example, the automatic air conditioner receives information related to the temperature in the vehicle interior from the temperature sensor in the vehicle interior as input information from the vehicle interior, and actively controls the operation state based on the input information. Further, the auto wiper (automatic wiper) receives information on the rain condition from a sensor outside the vehicle compartment as input information from outside the vehicle compartment, and actively controls the operation state based on the input information.

  If it is an automatic air conditioner, auto lock, auto wiper, auto light, etc., it is judged that the possibility of utterance is high when ON / OFF (locking, unlocking), air volume change, operation speed change, etc. are detected. To do. As for the clock, the user's operation history, etc. is used to learn TV / radio operation trends associated with time signal information. Based on this information, for example, an utterance is detected when a change in time such as ○ hour ○ minute is detected. It is desirable to judge that the possibility is high. The automatic input state is set for a predetermined time interval from the time when the change in the operation state of the device is detected. Synchronously with this, it is related to the operation of adjusting the air volume and set temperature of the automatic air conditioner, locking / unlocking operation of auto-lock, ON / OFF operation of auto wiper / auto light, operation of TV / radio due to time signal, etc. The language dictionary is changed so that the vocabulary to be recognized is easily recognized.

  Furthermore, it is desirable to provide means for notifying the user of the automatic input state. FIG. 14 shows a screen transition when the operating state of the air conditioner changes as such an example. (A) of FIG. 14 is a screen in a normal state, and clearly shows that the air volume of the air conditioner is 3. FIG. 14B shows a screen when the room temperature is lowered and the air-conditioner air volume is changed to 1. With the change in the air conditioner operating state, the automatic input state is entered, indicating that the air conditioner adjustment voice command (operating the device by voice) is possible.

(Common to Examples 1 to 4)
In the above embodiment, there is a possibility that a change in the operation of a plurality of devices that are objects of operation by voice is detected at the same time. An operation example for such a case is shown in FIG.

(A), (b), and (c) of FIG. 15 show changes in the devices A, B, and C in time series, respectively. In this example, the device A at time _{T 1} and _{T 6,} the device B at time _{T 2} and _{T 5,} device C changes at time _{T 3} and _{T 4.} For the device A, the device B changes from the change time T (= T ₂ , T ₅ ) to T + β ₂ for the time interval from T−α to T + β _{1 with} the change time T (= T ₁ , T ₆ ) as a reference. The device C determines that the possibility of speech is high at T + β ₃ from the change time T (= T ₃ , T ₄ ).

Examples of operations under this situation are shown in FIGS. 15 (e), (f), and (g).
(e) A ∪ B ∪ C: An automatic input state in which all devices with high utterance possibility can be operated.
(f) Newest (A, B, C): An automatic input state in which only the device that has detected the latest change can be operated.
(g) A>B> C: Determines the priority order of devices (for example, priority: hands-free telephone> storage media playback device> broadcast wave receiving device> automatic air conditioner), and operation of the highest priority device is possible Set to the automatic input state.

  With the above-described series of configurations, it is possible to input an utterance related to the operation of the device in which a change in the operating state is detected without using the utterance switch, and convenience for the user is improved.

It is a figure which shows the basic composition of the audio | voice input apparatus which concerns on this invention. It is a figure which shows the example of fundamental embodiment of the audio | voice input apparatus which concerns on this invention. It is the flowchart which showed the basic operation | movement of the audio | voice input apparatus which concerns on this invention. It is a figure which shows an example of the language dictionary in the embodiment of this invention. It is a figure which shows the example of dynamic change of a language dictionary (hierarchy movement). It is a figure which shows the example of dynamic change of a language dictionary (except for an utterance prediction vocabulary is excluded from a recognition vocabulary). It is a figure which shows the example of dynamic change of a language dictionary (bias of appearance and transition probability). It is a figure which shows the example of dynamic change of a language dictionary (change of an activation dictionary). It is a figure explaining the speech possibility detection method in the apparatus which reproduces | regenerates the sound signal memorize | stored in the storage medium. It is the figure which showed the screen transition in Example 1. FIG. It is a figure explaining the speech possibility detection method in a broadcast wave receiver. It is the figure which showed the screen transition in Example 2. FIG. It is the figure which showed the screen transition in Example 3. FIG. It is the figure which showed the screen transition in Example 4. FIG. It is the figure which showed the operation example of each apparatus in case there exist two or more apparatuses used as the object of operation by an audio | voice.

Explanation of symbols

  101: In-vehicle device, 102: Speech possibility determination unit, 103: Sound input unit, 104: Dictionary change unit, 105: Speech recognition unit, 106: Recognition dictionary, 107: Operation command issue unit, 108: Speech switch, 201: Microphone, 202: amplification device, 203: AD conversion device, 204: arithmetic device, 205: storage device, 206: timer, 207: sensor, 208: switch, 401: menu dictionary, 402: destination dictionary, 403: television dictionary, 404: CD dictionary, 405: ... dictionary.

Claims (20)

Sound input means for converting input sound into sound signals and outputting; sound recognition means for converting sound signals output from the sound input means into information signals; and time when the user is likely to speak A speech input device having an utterance possibility judging means for judging a section,
The sound input means converts voice input by the user into a voice signal in a time interval determined by the speech possibility determination means to be a time interval in which the user is highly likely to be uttered. The voice input device outputs the voice recognition means for performing a recognition process for converting the voice signal into an information signal.   A recognition dictionary having a language dictionary used for collation with a speech signal output from the sound input means, and a vocabulary that the user is likely to utter in a time interval in which the user is likely to utter The speech input device according to claim 1, further comprising: a language dictionary changing unit that selects the language dictionary and changes the language dictionary based on the vocabulary.   The utterance possibility determining means, when a reference time is a time at which a change in the operating state of a device to be operated by voice is detected, is a first time that is a predetermined time before the reference time. A time interval from one time to the reference time, a time interval from the reference time to a second time after a predetermined second time, or from the first time to the second time The voice input device according to claim 1, wherein the time interval to be reached is determined to be a time interval in which the user is highly likely to speak.   The language dictionary changing unit selects a voice operation vocabulary related to a device to be operated by the voice as a vocabulary that the user is likely to utter in a time interval in which the user is likely to utter. 4. The voice input device according to claim 3, wherein the language dictionary is changed based on the voice operation vocabulary.   The device to be operated by voice is a device that reproduces a sound signal or a video signal stored in a storage medium, and the change in the storage medium is detected at the time when the change in the operation state of the device is detected. The audio input device according to claim 3 or 4, wherein the audio input device is a time at which a break of a sound signal or a video signal stored in the storage medium is detected.   The device to be operated by the voice is a device that receives and outputs a sound signal or a video signal transferred from the outside, and the time when the change in the operation state of the device is detected is the received sound signal. 5. The audio input device according to claim 3, wherein the audio input device is a time at which a video signal break is detected, or a time at which a change in reception intensity of an externally transmitted sound signal or video signal is detected. .   The device to be operated by the voice is a communication device that receives and receives information by communication means, and the time when the change in the operation state of the communication device is detected is the external communication device other than the communication device and the communication device. 5. The voice input device according to claim 3, wherein the voice input device is a time at which communication with the external communication device is established or a time at which information from the external communication device arrives.   The device to be operated by voice is a device whose operation state is actively controlled based on input information from inside or outside the vehicle interior, and the time at which a change in the operation state of the device is detected is 5. The voice input device according to claim 3, wherein the time is a time when a change in the control state is detected.   The operation of at least one of the sound input unit and the voice recognition unit is stopped until the possibility that the user speaks again after the recognition processing by the voice recognition unit is completed. The voice input device according to any one of claims 1 to 8.   After completion of the recognition processing by the voice recognition means, the language dictionary changing means returns the language dictionary changed before the recognition processing to the state before the change before the user may possibly speak again. The voice input device according to any one of claims 2 to 9, Sound input means for converting input sound into sound signals and outputting; sound recognition means for converting sound signals output from the sound input means into information signals; and time when the user is likely to speak A speech input method using speech possibility determination means for determining a section,
The sound input means converts voice input by the user into a voice signal in a time interval determined by the speech possibility determination means to be a time interval in which the user is highly likely to be uttered. A voice input method for outputting, wherein the voice recognition means converts the voice signal into an information signal.   A recognition dictionary having a language dictionary used for collation with a speech signal output from the sound input means, and a vocabulary that the user is likely to utter in a time interval in which the user is likely to utter 12. The speech input method according to claim 11, further comprising: using a language dictionary changing unit that selects the language dictionary and changes the language dictionary based on the vocabulary.   The utterance possibility determining means, when a reference time is a time at which a change in the operating state of a device to be operated by voice is detected, is a first time that is a predetermined time before the reference time. A time interval from one time to the reference time, a time interval from the reference time to a second time after a predetermined second time, or from the first time to the second time The voice input method according to claim 11 or 12, wherein the time interval to be reached is determined as a time interval in which the user's utterance is likely to be performed.   The language dictionary changing unit selects a voice operation vocabulary related to a device to be operated by the voice as a vocabulary that the user is likely to utter in a time interval in which the user is likely to utter. 14. The voice input method according to claim 13, wherein the language dictionary is changed based on the voice operation vocabulary.   The device to be operated by voice is a device that reproduces a sound signal or a video signal stored in a storage medium, and the change in the storage medium is detected at the time when the change in the operation state of the device is detected. 15. The audio input method according to claim 13 or 14, wherein the audio input method is a time at which a break of a sound signal or a video signal stored in the storage medium is detected.   The device to be operated by the voice is a device that receives and outputs a sound signal or a video signal transferred from the outside, and the time when the change in the operation state of the device is detected is the received sound signal. 15. The audio input method according to claim 13 or 14, wherein a time at which a video signal break is detected, or a time at which a change in reception intensity of a sound signal or video signal transferred from the outside is detected. .   The device to be operated by the voice is a communication device that receives and receives information by communication means, and the time when the change in the operation state of the communication device is detected is the external communication device other than the communication device and the communication device. 15. The voice input method according to claim 13, wherein the time at which communication with the external communication device is established or the time at which information from the external communication device arrives is received.   The device to be operated by voice is a device whose operation state is actively controlled based on input information from inside or outside the vehicle interior, and the time at which a change in the operation state of the device is detected is The voice input method according to claim 13 or 14, characterized in that it is a time at which a change in the control state is detected.   The operation of at least one of the sound input unit and the voice recognition unit is stopped until the possibility that the user speaks again after the recognition processing by the voice recognition unit is completed. The voice input method according to claim 11, wherein:   After completion of the recognition processing by the voice recognition means, the language dictionary changing means returns the language dictionary changed before the recognition processing to the state before the change before the user may possibly speak again. The voice input method according to any one of claims 12 to 19, wherein

Images