JP5154363B2 - Car interior voice dialogue system - Google Patents

Description

  The technology disclosed in the present specification relates to a voice interactive apparatus mounted in a vehicle interior.

A car navigation system mounted on a car is widely used having a voice dialogue function (see, for example, Patent Document 1). Conventionally, a speech recognition technology and a speech synthesis technology necessary for realizing a speech dialogue function have been widely studied. The speech recognition technology is a technology for converting a speech waveform input through a microphone into text. The speech synthesis technique is a technique for generating a speech waveform from text. By combining the speech recognition technology and the speech synthesis technology, the car navigation system can talk with the user by voice. Further, since various noises are superimposed on the voice waveform recorded in the passenger compartment, there is a problem that the voice recognition rate is greatly deteriorated. In order to solve this problem, a sound source separation technique that suppresses noise using a microphone array having a plurality of microphone elements and extracts only desired speech has been widely studied.
JP 2004-109323 A

  Since the conventional car navigation system is designed on the assumption that the driver speaks, the entire contents of the speech recognition dictionary to be accepted are read out by speech synthesis. However, since it takes a long time to read all the options, there is a problem that it takes a long time to complete the voice conversation. On the other hand, the option may be displayed on the display instead of being read out. However, although the method of displaying choices on the display has the advantage that the time until the voice conversation is completed is short, it is not preferable to use it when the driver performs the voice conversation. This is because the safety of the driver who is driving the vehicle is impaired by viewing the display.

A representative invention disclosed in the present application includes an occupant detection unit that detects a occupant's boarding position in an automobile, an audio output unit that converts text information into voice, and outputs the text information into an image for display. The image display unit, the plurality of microphones, and the sound output unit or the image display unit are selected based on the detection result of the occupant detection unit, and the selected sound output unit or the image is selected. A switching unit that causes the display unit to output the text information, and when an occupant other than the driver is on board, a first time required to display the text information on the image display unit, and the text Estimating the second time required to output the speech for reading information from the voice output unit, and outputting the text information to the image display unit when the first time is shorter than the second time If the second time is shorter than the first time, select to output the text information to the voice output unit, and if a passenger other than the driver is on board, When a voice prompting a passenger other than the driver to respond is output from the voice output unit and a voice prompting a passenger other than the driver to respond is output, the plurality of microphones receive the voice. Determining the sound source direction of the received voice, determining whether the specified sound source direction is within a predetermined range including a direction from the plurality of microphones to an occupant other than the driver, When the identified sound source direction is within the predetermined range, the received voice is processed as a response to the text information displayed on the image display unit, and no occupant other than the driver is on board. If, and selects that to output said text information to the audio output unit.

  According to one embodiment of the present invention, the option presenting means determination unit determines whether or not there is only a driver in the vehicle, and selects the optimum presenting means according to the determination result. For example, if there is only a driver, the contents of the speech recognition dictionary are read out by speech synthesis, and if there are passengers other than the driver, the passenger is guided to answer to reduce the task time, and then the display The choices are displayed above. As a result, a voice conversation with a short task time can be realized without sacrificing safety, and a quick car navigation system can be operated.

  FIG. 1 is a functional block diagram of the voice interactive apparatus according to the first embodiment of the present invention.

  The voice interactive apparatus of this embodiment is assumed to be used as an application of a car navigation system mounted on a car, for example. Hereinafter, the embodiment will be described under this assumption. In addition, one or more users including one driver (driver) get on the automobile.

  FIG. 2 is a block diagram of a hardware configuration of the voice interactive apparatus according to the first embodiment of this invention.

  The system of the present embodiment includes a microphone array 1201 including at least two microphone elements. The microphone array 1201 measures the sound pressure level at each microphone element position.

  The analog sound pressure value measured by the microphone array 1201 is sampled by the AD converter 1202 and converted into digital data. The AD converter 1202 may sample the sound pressure value after removing a frequency component equal to or more than 0.5 times the sampling rate using an analog low-pass filter (not shown) or the like.

  The sampled digital sound pressure data is sent to the central processing unit 1203. The central processing unit 1203 executes programs such as estimation of a sound source direction in digital sound pressure data, correction of a phase difference, speech recognition, and dialogue processing.

  A program executed by the central processing unit 1203 is stored in the storage medium 1205 as data.

  Temporary data necessary for program execution may be stored in the volatile memory 1204 or the storage medium 1205. In addition, advance data necessary for program execution is stored in advance in the storage medium 1205. The storage medium 1205 is a large-capacity nonvolatile storage medium such as a hard disk drive (HDD) or a flash memory. On the other hand, the volatile memory 1204 is a high-speed storage device such as a dynamic random access memory (DRAM).

  For example, the dialogue script database (DB) 114 and the recognition vocabulary DB 115 shown in FIG. 1 are stored in the storage medium 1205. Each unit other than the database shown in FIG. 1 is realized by the central processing unit 1203 executing a program stored in the storage medium 1205. However, the voice input unit 101 may be realized by the AD conversion device 1202. The above database and program may be stored in the storage medium 1205, and all or a part of them may be copied to the volatile memory 1204 as necessary.

  Analog voice received by the microphone is converted into digital voice by the voice input unit 101. The converted digital sound is sent to the sound source separation unit 102. In the sound source separation unit 102, a noise component included in the digital sound is removed, and a signal in which a desired sound is emphasized is obtained. Whether or not the sound is desired is determined from information on the sound source direction. For example, when the desired sound is a driver driving an automobile, the relative direction of the driver viewed from the microphone is set as the desired sound source direction, and the sound coming from the desired sound source direction is determined as the desired sound. .

  The desired sound source direction is set by the separation range setting unit 105. The separation range setting unit 105 sets a desired sound source direction based on the output result of the dialogue control unit 107.

  The desired speech component extracted by the sound source separation unit 102 is sent to the speech recognition unit 103. The speech recognition unit 103 recognizes the utterance content of the transmitted speech waveform and outputs the converted speech string. The speech recognition processing executed by the speech recognition unit 103 may be any one based on, for example, a hidden Markov model or a dynamic programming method.

  The speech recognition process is a kind of pattern matching process that determines which word in the given vocabulary is the closest to the utterance content of the input speech waveform and outputs the closest word. The vocabulary stored in advance for use in speech recognition is created in advance by the recognition dictionary generation unit 108 before starting the speech recognition process.

  The speech recognition unit 103 further calculates the likelihood of the recognition result. The likelihood of the recognition result is a scale indicating the degree of closeness between the word determined to be closest to the utterance content of the input speech waveform and the utterance content. The likelihood can be calculated by various known methods.

  The utterance content (that is, the recognition result) converted into the character string generated by the speech recognition unit 103 and the likelihood of the recognition result are sent to the recognition result rejection determination unit 104. The recognition result rejection determination unit 104 determines whether to accept or reject the recognition result based on the likelihood information of the recognition result. For example, it may be determined to be accepted when the posterior probability of the recognition result generated from the likelihood information of the recognition result exceeds a threshold value. On the other hand, when the posterior probability is equal to or lower than the threshold value, it is estimated that a word different from any given vocabulary is spoken, and therefore, it may be determined to reject the recognition result.

  The recognition result received by the recognition result rejection determination unit 104 is sent to the dialogue control unit 107. If the recognition result is rejected, the voice recognition process may be continued. If the recognition result is not accepted within a certain time after the start of the speech recognition processing, the recognition result rejection determination unit 104 may transmit information indicating that there is no recognition result to the dialogue control unit 107. The dialogue control unit 107 may determine the next action based on the information. For example, the dialogue control unit 107 may control each unit so that the speech recognition process is executed again after outputting the guidance for uttering “Please speak again”.

  FIG. 3 is a flowchart showing the speech recognition process executed in the first embodiment of the present invention.

  Specifically, FIG. 3 shows the flow of specific speech recognition processing executed by the sound source separation unit 102, speech recognition unit 103, and recognition result rejection determination unit 104.

  In the target sound range setting S <b> 901, the sound source separation unit 102 sets the target sound existence range based on the range set by the separation range setting unit 105. For example, the target sound existence range is set for each of the azimuth angle and the elevation angle. For example, the azimuth angle may be set from −30 degrees to +30 degrees, and the elevation angle may be set from −90 degrees to 90 degrees.

  In the sound source separation S902, the sound source separation unit 102 extracts a sound in the target sound direction based on the set information on the target sound presence range.

  In speech recognition S903, the speech recognition unit 103 recognizes the utterance content of the sound in the target sound direction extracted in the sound source separation S902 using the speech recognition dictionary and the sound model.

  In the reliability check S904, the recognition result rejection determination unit 104 sets a threshold in which a scale indicating the reliability of the speech recognition result (for example, a posterior probability calculated from the acoustic likelihood associated with the speech recognition result) is set in advance. It is determined whether or not it exceeds. If the measure of confidence is above the threshold, the recognized speech is estimated to be uttered to enter a command. For example, as will be described later, when an option is presented to the user, the user utters one of the options accordingly. Such an utterance is an example of an utterance for inputting a command. In this case, the process proceeds to the direction check S905.

  On the other hand, if the measure of reliability is below the threshold, the recognized speech is not spoken to enter the command (for example, speech entered without regard to the command, such as noise) )It is estimated to be. In this case, the process proceeds to the time limit check S907.

  In the direction check S905, the recognition result rejection determination unit 104 calculates the sound source direction of the speech recognition waveform by Equation (1), where T is the time length of the speech recognition waveform, and whether the direction is within a given target sound range. Determine whether.

  θ (f, τ) is the sound source direction of the frequency f at time τ, and the calculation method will be described later. fmax is the maximum frequency component of the speech recognition waveform.

  If the sound source direction is within the target sound range, the recognition result rejection determination unit 104 returns the recognition result and ends the process. As will be described later, when the process shown in FIG. 3 is executed after the option is presented to the user, the recognition result returned as described above is a response to the presentation of the option (that is, the presented option). As one of the utterances).

  If the sound source direction is outside the target sound range, the process proceeds to a time limit check S907. In the time limit check S907, the recognition result rejection determination unit 104 determines whether or not the time elapsed since the start of speech recognition is within a given time limit. If the elapsed time is within the given time limit, the process returns to the sound source separation S902, and the sound in the target sound direction is extracted again. If the elapsed time exceeds the time limit, the process ends without returning the recognition result.

  The sound source separation S902 and the speech recognition S903 are processes for a real-time speech waveform acquired by the speech input unit 101. That is, the above processing is performed on new speech waveforms that are input every moment.

  FIG. 4 is an explanatory diagram illustrating a flow of detailed processing executed by the sound source separation unit 102 according to the first embodiment of this invention.

  The process of FIG. 4 is executed each time the voice input unit 101 acquires a certain amount of voice data (for example, about several tens of ms).

  A speech waveform recorded by a plurality of microphone elements is subjected to discrete Fourier transform in the DFT 1001 for each microphone element. The sound pressure data for the sampling time (t) for each microphone element (i) is expressed by Equation (2).

  A discrete Fourier transform may be performed after a hamming window or a hanning window of a speech waveform is previously multiplied by a signal in the time domain. By multiplying the Hamming window or the window function of the Hanning window, a highly accurate time-frequency domain signal can be obtained.

  The conversion to the time-frequency domain signal by the discrete Fourier transform is performed by Equation (3). The converted signal is expressed by Equation (4).

  Here, τ is called a frame index and is equal to the number of times of conversion into a time-frequency domain signal. w (n) is the window function of the Hanning window or the Hamming window. Frame size for Fourier transform.

  In the per-frequency vectorization 1002, the sound source separation unit 102 generates a vector X (f, τ) defined by Expression (5) by collecting the signals for each microphone belonging to the same time-frequency domain after conversion. M is the number of microphone elements.

  In the sound source localization 1003, the sound source separation unit 102 gives the maximum value of the inner product of the steering vectors a (θ, f) and X (f, τ) defined by Equation (6) for each time frequency. Is calculated by Equation (7). c is the speed of sound.

  FIG. 5 is a flowchart illustrating processing executed for each time frequency in the sound source separation 1004 according to the first embodiment of this invention.

  In step S1101, the sound source separation unit 102 determines whether the sound source direction θ for each time frequency is within a predetermined target sound range. When the sound source direction θ is within the target sound range, the sound source separation unit 102 sets the vector n (f, τ) to the zero vector and sets the vector s (f, τ) to X (f, τ). Thereafter, the process proceeds to the target sound covariance update S1102. If it is outside the target sound range, the sound source separation unit 102 sets the vector n (f, τ) to X (f, τ), and proceeds to the noise covariance update S1103.

  In the target sound covariance update S1102, the sound source separation unit 102 updates the covariance matrix Rs (f) using Equation (8) using the vector s (f, τ).

  In noise covariance update S1103, the sound source separation unit 102 updates the covariance matrix R (f) as shown in Expression (9) using the vector n (f, τ). Here, α is a given update rate.

  The sound source separation unit 102 obtains a sound source separation filter w (f, τ) using Equation (10) using the covariance matrices Rs (f) and R (f). Let eig_vector be a function that gives the eigenvector that is the maximum eigenvalue.

  In filtering S1105, the sound source separation unit 102 obtains the noise suppression signal s (f, τ) from the input signal X (f, τ) and the sound source separation filter w (f, τ) by Expression (11).

  In post-filtering S1106, the sound source separation unit 102 suppresses the residual noise component by applying a Wiener filter or spectral subtraction processing to the noise suppression signal s (f, τ). Then, the sound source separation unit 102 outputs the time frequency signal after residual noise suppression, and ends the processing.

  In inverse DFT 1005, sound source separation section 102 generates a time domain signal by performing inverse discrete Fourier transform on the obtained noise suppression signal for each frequency, and then outputs the time domain signal.

  The dialogue control unit 107 controls voice dialogue with the user based on the dialogue script held in the dialogue script DB 114.

  FIG. 6 is a flowchart showing the dialogue based on the dialogue script in the first embodiment of the present invention.

  In the example of the dialogue flow shown in FIG. 6, after the dialogue with the user is started, first, in the command name S301, the dialogue control unit 107 recognizes the name of the command that the user wants to execute. Upon recognition, the dialogue control unit 107 may output guidance prompting the user to speak such as “Please give command name”. Furthermore, the dialog control unit 107 may select a command to be executed from a command list displayed on the screen together with a comment such as “Please select from these” or a voice that reads out the contents of the command list. May be generated using a speech synthesis system or the like, and the speech may be output.

  In the present embodiment, whether to display the command list on the screen or to read the command list is switched based on the seating information of the user. This switching may be executed in the same manner as selection of an option presentation method described later. For example, when only the driver is on board, the command list may be read out. On the other hand, if a passenger (ie, a user other than the driver) is present in the car, the passenger will see a screen such as “Passenger, please select from here” in order to shorten the time required for voice conversation. It may be controlled to display the command list on the screen after the guidance for prompting and answering is given. As a result, quick command input is realized while ensuring the safety of driving of the automobile.

  In FIG. 6, after the command name S301 ends, the process is switched according to the recognized command (that is, the command selected from the command list by the user). For example, if the recognized command is destination setting, then destination setting S302 for recognizing a specific destination is executed.

  In the destination setting S302, guidance that prompts the user to speak the destination may be output, such as “Please tell the destination”, or “Please select a destination from this”. After the audio is output, the destination list may be displayed on the screen, or the destination list may be read out. An example of the destination list displayed on the screen will be described later with reference to FIG.

  As in the case of the command name S301 described above, the presenting means that minimizes the time required for destination setting is selected from the presenting means that do not impair safety, depending on whether or not there is a passenger.

  When the command recognized in the command name S301 is an automobile device operation, an automobile device operation S303 is executed. In the car equipment operation S303, the dialogue control unit 107 recognizes a user command for operating the car internal equipment such as control of air conditioner On / Off or music. At this time, the dialogue control unit 107 may cause the voice output unit 112 to output a guidance such as “Please tell us the name of the device you want to operate and the details of the operation” or display a command list that can be operated with voice commands on the screen. In this manner, the image display unit 113 may be controlled. Similar to the above-described destination setting S302, a presentation means is selected such that the time required for operation of the automobile internal device is the shortest.

  When the command recognized in the command name S301 is a surrounding facility search, a surrounding facility search S304 is executed. In the peripheral facility search S304, the dialogue control unit 107 searches for a peripheral facility desired by the user, and executes a process of setting the searched facility as a destination using a voice interface. Specifically, the dialogue control unit 107 may cause the voice output unit 112 to output a guidance that prompts the user to speak, such as “Please tell the surrounding facility” or “Select from these” guidance. May be displayed, a list of surrounding facilities may be displayed on the screen, and the user may be prompted to speak while viewing the screen. In the same manner as in the above-described automobile device operation S303, a presentation unit that selects the time required for the operation of the automobile internal device most is selected.

  After any processing from S302 to S304 is completed, the dialogue control unit 107 ends the dialogue.

  The above is an example of the dialogue based on the dialogue script in the car navigation system. The interactive script may be described in any format as long as it is described in a language that can be described in a format that holds conditional branching and system execution command information. For example, an interactive script may be described in an XML format such as Voice XML, or an interactive script may be described as a program code in a script language that is a kind of program language.

  The dialogue control unit 107 controls operations of the voice recognition unit 103, the sound source separation unit 102, the voice output unit 112, and the image display unit 113 based on the dialogue script. For example, in the dialog for recognizing the command list described above, the recognition dictionary generation unit 108 switches the recognition dictionary used for recognition before starting speech recognition. The recognition dictionary generation unit 108 generates a speech recognition dictionary described in a finite automaton format.

  FIG. 7 is an explanatory diagram showing an example of a speech recognition dictionary used in the first embodiment of the present invention.

  In this example, the recognition dictionary is expressed in the form of a network of six nodes and six arcs. When the recognition dictionary of this example is used, it is assumed that the user utterance is either a sentence in which “Destination” and “End of Word” are continuous, or a sentence in which “Phone Number” and “End of Word” are continuous Is done. “Destination”, “End of word”, and “Telephone number” are labels that collectively represent a plurality of words called arcs. Each arc is further expanded into a word list.

  FIG. 8 is an explanatory diagram illustrating an example of a word list for each arc according to the first embodiment of this invention.

  FIG. 8 shows an example of a word list in which the arc of “destination” is expanded. In the example of FIG. 8, proper names of facilities such as “Central Research Laboratory” and “Mechanical Research Laboratory” are included in the list of words indicating the destination. As other destinations, for example, a general noun of a facility such as “restaurant” may be included (see FIG. 12), or a place name such as “Tokyo” may be included.

  In this way, each arc is expanded into a word list. Thus, for example, the number of sentences in which “Destination” and “End of Word” are consecutive is obtained by multiplying the number of words included in the word list of “Destination” by the number of words included in the word list of “End”. Become a number. The word list is stored in the recognition vocabulary DB 115, and necessary vocabulary is extracted according to the recognition dictionary.

  The speech recognition unit 103 divides each word into phonemes or phonemes, and outputs a pattern closest to the input speech using a pattern in which sound models corresponding to each phoneme or phoneme are arranged. The sound model is expressed by a mixed normal distribution with a feature amount such as LPC cepstrum, MFCC, their difference value (Δ), their ΔΔ, or power time difference value for each phoneme or phoneme. . Further, the distortion of the transmission system may be corrected by subtracting the average value (cepstrum average value subtracting process) or the like when calculating the feature amount.

  The pattern closest to the input speech is calculated by a Viterbi algorithm approximated to calculate only the state transition path that gives the maximum likelihood in the maximum likelihood estimation based on the forward / backward algorithm. The likelihood is defined based on the distance between the input speech and the pattern. The speech recognition unit 103 calculates a state transition path that gives the maximum likelihood, and reverses the word sequence from the state transition path. As a result, a single character string is obtained and the character string is output. Further, the maximum likelihood itself or the posterior probability p (O | X) obtained by processing the maximum likelihood is output. Here, p (O | X) is the posterior probability of the recognition result O on the condition of the input speech X (that is, a value indicating the probability that the recognition result O for the input speech X is a correct result).

  A user utterance is not necessarily a sentence expressed in a speech recognition dictionary. In addition, since there is noise such as running sound other than the desired user utterance in the passenger compartment, the recognition target voice is often noise. Even in such a case, since the voice recognition unit 103 outputs the pattern closest to the input voice, it is not desirable to be confident and accept all the output voice recognition results.

  If the likelihood or posterior probability of the output speech recognition result is smaller than a predetermined threshold, it is highly possible that the speech to be recognized was noise or an utterance other than a sentence expressed in the speech recognition dictionary was made. Such speech recognition results should be rejected. The recognition result rejection determination unit 104 determines whether to accept or reject the recognition result based on the likelihood or posterior probability of the recognition result output by the speech recognition unit 103 (reliability check S904). Furthermore, the recognition result rejection determination unit 104 estimates the sound source direction of the waveform subjected to speech recognition, and if the sound source direction is outside the range of the desired sound source direction (that is, outside the target sound range), the recognition result May be rejected.

  In general, the user does not know in advance information about a sentence (text) that can be received by the speech recognition unit 103. Therefore, it is necessary to present to the user what sentences are acceptable before the start of speech recognition. As a method of presenting options, a part of an acceptable sentence is read out using speech synthesis technology, or a part of an acceptable sentence is displayed on a display screen of a car navigation system. And so on.

  FIG. 9 is a block diagram of a hardware configuration of the car navigation system including the voice interactive apparatus according to the first embodiment of the present invention.

  The car navigation system of this embodiment includes a central processing unit 1603, a speaker 1601, a storage medium 1602, a seat sensor 1604, a display 1605, a microphone 1606, and a speed sensor 1607 connected to the central processing unit.

  The central processing unit 1603 executes software processing such as speech recognition and sound source separation.

  Information such as a recognition dictionary is held in the storage medium 1602.

  A reproduction sound such as a guidance sound is output from the speaker 1601. The speaker 1601 may be a super-directional speaker such as an ultrasonic speaker.

  It is determined whether or not a passenger is present by a seat sensor 1604 installed in each seat such as a driver seat, a passenger seat, and a rear seat. As long as the seat sensor 1604 outputs information indicating whether or not the user is in each seat, the seat sensor 1604 may be of any type, such as a weight sensor or an ultrasonic sensor having a beam in each seat direction. May be.

  The display 1605 displays information related to the recognized vocabulary such as a command list, a map, and the like.

  Information indicating the speed of the vehicle acquired by the in-vehicle speed sensor 1607 is taken into the central processing unit 1603 and used to determine a traveling state (for example, whether the vehicle is traveling).

  The microphone 1606 is used for recording user utterances. The voice recognition unit 103 recognizes the voice recorded through the microphone 1606. Instead of the microphone 1606, a microphone array including a plurality of microphone elements (for example, the microphone array 1201 shown in FIG. 2) may be used.

  By using a microphone array, it is possible to obtain information on the direction of a sound source that is difficult to obtain with a single microphone, or to apply a beam to the direction of the target speaker and extract only the speech uttered by the speaker in that direction. can do. The sound source separation unit 102 may extract only the voice uttered by a speaker in a specific direction using a microphone array.

  Note that the microphone 1606 and the central processing unit 1603 shown in FIG. 9 correspond to the microphone array 1201 and the central processing unit 1203 shown in FIG. 2, respectively. A storage medium 1602 illustrated in FIG. 9 corresponds to at least one of the volatile memory 1204 and the storage medium 1205 illustrated in FIG.

  The option presenting means determination unit 106 selects a task end time (that is, the user presenting device gives options to the user from the presenting means using the user presenting device attached to the car navigation system under the condition that safety is not impaired. Presenting, selecting the means with which the user understands the option, and having a short time required for the user to speak any of the option. The car navigation system described in FIG. 9 includes a speaker 1601 and a display 1605 as a user presentation device. In this case, there are two possible user presentation means: a method of outputting a voice that reads out an option using speech synthesis from the speaker 1601 and a method of displaying the option on the display 1605.

  FIG. 10 is a flowchart illustrating processing executed by the option presenting means determination unit 106 according to the first embodiment of this invention.

  In passenger determination S701, the option presenting means determination unit 106 determines whether a passenger other than the driver is on board. For example, based on the output of the seat sensor 1604 for the passenger seat (or the rear seat), it may be determined whether or not the passenger is on board.

  In the next task end time determination S702, the option presenting means determination unit 106 estimates the task end time when the options are presented to the user using each presenting means. The estimated value of the task end time may be obtained by adding a preset average speech recognition end time to the time required for the presentation device to present an option, or a voice interaction device using each presentation device. May be the average task end time measured when the subject has previously used.

  In the shortest end time means selection S703, the option presenting means determination unit 106 estimates in the task end time determination S702 among the presentation devices determined based on the result of the passenger determination S701 and does not impair the safety of the in-vehicle environment. The presentation device with the shortest task end time is selected. Information indicating which presentation device has been selected is output from the option presentation means determination unit 106.

  If there is only one safe presentation means, the task end time determination S702 may not be executed. In that case, in the shortest end time means selection S703, only one of the safe presentation means is selected.

  The selection of the presentation device that does not impair the safety of the in-vehicle environment based on the passenger determination result will be specifically described below. Here, two presentation apparatuses described in FIG. 9, that is, a speaker 1601 that reads an option using speech synthesis and a display 1605 that displays the option will be described as an example.

  The act of the driver looking at the display 1605 may distract the driver's attention while driving (ie, it may compromise safety). For this reason, it is not preferable that the driver selects an option while viewing the option. Therefore, reading out the choices by speech synthesis is considered as one of the safe presentation methods.

  Therefore, as a result of the passenger determination S701, when it is determined that there is no passenger other than the driver, the method of displaying options on the display 1605 is selected from the presentation methods using the presentation device shown in FIG. Instead, the method of reading out the options using speech synthesis is selected. In other words, not the display 1605 but the speaker 1601 is selected as a presentation device to be used.

  On the other hand, as a result of the passenger determination S701, when it is determined that there is a passenger other than the driver, the act of the passenger other than the driver selecting an option while viewing the display 1605 does not impair safety. Therefore, in this case, the presentation method with the shortest estimated task end time is selected from the method of reading out the choices using speech synthesis and the method of displaying the choices on the display 1605.

  As described above, the task end time is the time required for the user presentation device to present the options to the user, and the user understands the presented options, and the user speaks one of those options. It is the total time required for.

  When the user presentation device is the speaker 1601, the time required for it to present the option to the user is approximately the sum of the time required for synthesizing the speech that reads out the option and the time required for outputting the synthesized speech. Equivalent to. This estimated time can be calculated based on, for example, the processing performance of hardware such as the central processing unit 1603 and the length of text to be presented as an option.

  On the other hand, when the user presentation device is the display 1605, the time required for it to present the options to the user is approximately the time required to generate the data of the image to be displayed on the display 1605. This corresponds to the total time required to display the displayed image on the display 1605. This estimated time value can be calculated based on, for example, the processing performance of hardware such as the central processing unit 1603 and the data amount of the generated image.

  A predetermined value may be held in advance as the time required for the user to understand the presented options and for the user to speak any of those options. For example, the time required when the subject uses each presentation device may be actually measured in advance, and the measured time may be held by the voice interaction device.

  However, in practice, the task end time depends largely on the time it takes for the user presentation device to present the options to the user rather than the time it takes for the user to understand the options and speak any of them. It is thought that it is done. In that case, only the time required for the user presentation device to present the option to the user may be calculated and compared as the task end time. In this case, if it is determined that there is a passenger, the time required for displaying the option on the display 1605 and the time required for reading the option are calculated and compared. As a result, for example, when it is determined that the time required for displaying the option on the display 1605 is short, the display 1605 is selected as the presentation device.

  Further, in practice, when the same option is presented, the time required for displaying the option on the display 1605 is generally shorter than the time required for reading the option. Therefore, when it is determined that neither of the above two presentation methods impairs safety, there is always a presentation method in which options are displayed on the display 1605 (ie, without calculating the estimated task end time). The presentation method with the shortest estimated task end time may be selected.

  When a method for displaying options on the display 1605 is selected, when a directional display is used as the display, the directivity may be set to be directed toward a passenger other than the driver. When a directional speaker is used as the audio reproduction speaker 1601, directivity may be directed toward the user who inputs an option from now on (ie, a passenger other than the driver).

  The process shown in FIG. 10 is executed in order for the dialog control unit 107 to select a method for interacting with the user. For example, the process shown in FIG. 10 may be executed before the dialog control unit 107 starts the process shown in FIG. 6, or may be executed every time the dialog control unit 107 processes each command.

  FIG. 11 is an explanatory diagram showing an example of guidance voice output in the first embodiment of the present invention.

  Specifically, FIG. 11 shows an example of guidance voice output when a passenger is present and not present. When there is no passenger, a guidance voice that guides the user to select from the choices read out by speech synthesis (in the example of FIG. 11, a guidance sentence “speech to read from the name of the facility to be read”) is output. The

  On the other hand, when there is a fellow passenger, a guidance voice for guiding the user to choose from the choices displayed on the display 1605 (in the example of FIG. 11, the guidance sentence “Please answer by the fellow passenger. “Please select from the list of facility names” is output. Even if there are passengers, it is not preferable that the driver induces the driver to look at the display 1605. Therefore, it is also necessary to induce the passengers to answer.

  FIG. 12 is an explanatory diagram illustrating an example of options displayed on the display 1605 according to the first embodiment of this invention.

  Specifically, FIG. 12 shows a display example of a screen displayed when a method of displaying options on the display 1605 is selected. In FIG. 12, as an example, destination options (for example, “restaurant” and “home”) to be set are displayed on the screen. By displaying the options on the screen as shown in FIG. 12, it is possible to shorten the time until the user grasps the options as compared to the method of reading the options using speech synthesis.

  The question sentence generation unit 110 generates a question sentence based on the determination result of the option presenting means determination unit 106 and the presence or absence of a passenger. The question sentence may be generated in real time every time it is necessary to output it, or the sentence may be preset for each condition. Further, by including the passenger's name in the question text, a specific passenger may be prompted to answer. By doing so, it is only necessary to extract the voice of a specific passenger, so that the recognition performance can be improved. In order to include the passenger's name in the question sentence, it is necessary to register the passenger's name in advance.

  FIG. 13 is a flowchart illustrating an example of processing executed for presenting options in the first embodiment of the present invention.

  First, in the passenger seat determination S1701, the option presenting means determination unit 106 determines whether or not a passenger is in the passenger seat. This determination is executed by the same method as the passenger determination S701 in FIG.

  When it is determined that a passenger is in the passenger seat, in the guidance output 1_S1702, the option presenting means determination unit 106 outputs a guidance voice when there is a passenger (see FIG. 11).

  Next, in the option display S 1704 on the screen, the option presenting means determination unit 106 displays the options on the display 1605. Specifically, based on an instruction from the option presenting means determination unit 106, the display image generation unit 111 generates an image to be displayed, and the image display unit 113 causes the display 1605 to display the generated image.

  On the other hand, when it is determined that the passenger is not in the passenger seat, in the guidance output 2_S1703, the option presenting means determination unit 106 outputs a guidance voice when there is no passenger (see FIG. 11).

  Next, in the option reading aloud in step S1705, the option presenting means determination unit 106 outputs a voice for reading the option. Specifically, based on an instruction from the option presenting means determination unit 106, the question sentence generation unit 110 generates a question sentence including the options to be output, and the voice output unit 112 reads out the voice that reads out the generated question sentence. Causes the speaker 1601 to output.

  After the option display S1704 or the option reading aloud S1705 is executed on the screen, the voice recognition S1706 is executed. Specifically, as described with reference to FIG. 3 and the like, the voice input unit 101 receives a voice input from the user, and the sound source separation unit 102 and the voice recognition unit 103 process the input voice. Thus, the input voice is recognized.

  At this time, the target sound range may be set based on the result of the passenger seat determination S1701 (target sound range setting S901 in FIG. 3). For example, when it is determined that there is a passenger, in the target sound range setting S901, a predetermined range including the direction from the microphone 1606 to the passenger (for example, a user seated in the passenger seat) is set as the target sound range. May be. In that case, the voice from the range of the target sound (that is, the voice uttered by the passenger) is accepted. The accepted voice is processed as a response to the choice presentation. On the other hand, since the voice (for example, the voice uttered by the driver) from outside the target sound range is rejected, it is not processed as a response to the presentation of the option.

  11 and 13 show an example in which selection is read out as an option presentation method when there is no passenger, and display of options is selected as an option presentation method when there is a passenger. As shown. However, for example, in the passenger seat determination S1701, processing similar to that shown in FIG. 10 may be executed. As a result, even when a passenger is in the passenger seat, it may be selected to read out the options as an option presentation method. In that case, guidance output 2_S1703 and option reading aloud S1705 are executed.

  FIG. 14 is a flowchart showing a passenger name registration process according to the first embodiment of the present invention.

  This process is executed immediately after the car navigation system is activated. In the passenger seat sensor check S801, whether or not there is a person in the passenger seat is determined based on the information of the seat sensor 1604.

  If there is a person in the passenger seat, the process proceeds to name recognition S802. In name recognition S802, after the speech recognition dictionary is set to accept all syllable sequences, speech recognition is executed. In this way, the name of any person can be recognized. The name recognized by speech recognition is stored in the storage medium 1602 after it is labeled as a result of the passenger seat. When the question sentence is generated, the name recognition result stored in the storage medium 1602 is referred to.

  If there is no person in the passenger seat, or after the name recognition S802 is executed, the process proceeds to the rear seat sensor check S803 to determine whether there is a person in the rear seat. If there is a person, the process proceeds to name recognition S804. In name recognition S804, after the speech recognition dictionary is set to accept all syllable sequences, speech recognition is executed. The name recognized by speech recognition is stored in the storage medium 1602 after it is labeled as a result of the backseat. When the question sentence is generated, the name recognition result stored in the storage medium 1602 is referred to.

  If there are no people in the backseat, the process ends. Further, after the name recognition S804, the process is terminated.

  The voice output unit 112 converts the question sentence generated by the question sentence generation unit 110 into a voice synthesized sound, and outputs the converted voice synthesized sound from the speaker 1601. The display image generation unit 111 generates a display image when it is necessary to display information on the screen based on the determination result of the option presenting means determination unit 106 and the presence or absence of a passenger. The image generated here may be one in which options are displayed as text information on the screen as illustrated in FIG. 12, or an image preset in advance for each option is displayed on the screen. May be. In the latter case, for example, a restaurant image included in the options may be displayed on the screen. The image display unit 113 displays the image generated by the display image generation unit 111 on the display 1605.

  The function control unit 109 controls the vehicle interior device based on the voice recognition result.

  FIG. 15 is a flowchart illustrating processing executed by the function control unit 109 according to the first embodiment of this invention.

  In the route setting command S1501, the function control unit 109 determines whether or not the voice recognition result relates to route setting. This determination is realized by setting in advance a flag indicating whether or not the result of the speech recognition result is related to route setting.

  When the voice recognition result relates to route setting, the function control unit 109 calls the route setting processing of the car navigation system and ends the processing.

  If the voice recognition result is not related to the route setting, the function control unit 109 next determines in the air conditioner operation command S1502 whether or not the voice recognition result relates to the air conditioner operation command. This determination is realized by setting in advance a flag indicating whether or not the result relates to an air conditioner operation command for each speech recognition result candidate.

  When the voice recognition result relates to the air conditioner operation command, the function control unit 109 calls the air conditioner operation process on the car navigation system and ends the process.

  If the voice recognition result is not related to the air conditioner operation process, then in the speaker control command S1503, the function control unit 109 determines whether or not the voice recognition result is related to speaker control. If the voice recognition result relates to speaker control, the function control unit 109 calls speaker control processing on the car navigation system and ends the processing.

  As described above, according to the first embodiment of the present invention, it is determined whether or not a user other than the driver is on the vehicle, and an information presentation method for the user is selected based on the determination result. As a result, a short voice dialogue is realized without sacrificing safety.

  Next, a second embodiment of the present invention will be described. The second embodiment is realized by hardware similar to that of the first embodiment (see FIGS. 2 and 9). Further, in the second embodiment, the same processing as that of the first embodiment is executed except for differences described below.

  FIG. 16 is a functional block diagram of the voice interactive apparatus according to the second embodiment of this invention.

  Among the units shown in FIG. 16, the voice input unit 201, the sound source separation unit 202, the voice recognition unit 203, the recognition result rejection determination unit 204, the separation range setting unit 205, the option presenting means determination unit 206, the dialogue control unit 207, the recognition dictionary. The generation unit 208, the function control unit 209, the recognized vocabulary DB 217, the dialogue script DB 216, the question sentence generation unit 210, the display image generation unit 211, the audio output unit 214, and the image display unit 215 are each used as the audio input in the first embodiment. Unit 101, sound source separation unit 102, speech recognition unit 103, recognition result rejection determination unit 104, separation range setting unit 105, option presenting means determination unit 106, dialogue control unit 107, recognition dictionary generation unit 108, function control unit 109, recognition Vocabulary DB 115, dialogue script DB 114, question sentence generation unit 110, display image generation unit 111, voice output unit 112 and It has the same function as the image display unit 113.

  The voice interactive apparatus according to the second embodiment further includes a question sentence output timing determination unit 212, an image display timing determination unit 213, and a traveling state determination unit 218. These are also realized by the central processing unit 1203 executing the program stored in the storage medium 1205.

  The traveling state determination unit 218 determines the current traveling state, for example, whether the vehicle is currently driving or stationary based on the vehicle speed information. Furthermore, the traveling state determination unit 218 may determine, using the map information of the car navigation system, information that the current location is on the expressway and a state that the current location is stationary at the intersection.

  Specifically, the car navigation system shown in FIG. 9 includes a positioning device (not shown) that acquires the current position information of the automobile. The traveling state determination unit 218 can determine whether or not the current location is on an expressway by referring to the current position information acquired by the positioning device and the map information stored in the storage medium 1602. .

  Based on the determined driving situation, the option presenting means determining unit 206 determines the task end time (that is, the user) from the presenting means using the user presenting device attached to the car navigation system under the condition that safety is not impaired. The presentation device presents the option to the user, the user understands the option, and selects a means that requires a short time) until the user speaks one of the options. The act of the driver viewing the display 1605 may distract the driver's attention (ie, it may compromise safety). For this reason, it is not preferable that the driver selects an option while looking at the option while driving.

  For this reason, when it is determined by the passenger determination S701 that there is no passenger other than the driver, the option presenting means is determined based on the determined traveling situation. Specifically, when the traveling state is stationary (that is, the vehicle is stopped), a method of displaying options on the display 1605 is selected. On the other hand, when the driving situation is driving, the guidance for instructing the driver to stop the vehicle at a place where the driver can stop is reproduced, and after confirming that the driver has stopped the vehicle by the driving situation determination, the display 1605 The method of displaying the options on is selected.

  The question sentence output timing determination unit 212 controls the timing of outputting the voice for reading out the question sentence according to the traveling situation. Specifically, when the driving situation is driving, the question message output timing determination unit 212 reproduces the guidance for instructing the driver to stop the vehicle at a place where the driver can stop, and then at the timing when the driver enters the stationary state. , Control to output a question sentence that guides the user to choose from the choices on the screen.

  Similarly, the image display timing determination unit 213 controls the timing at which the options are displayed on the display 1605 in accordance with the traveling state. Specifically, the image display timing determination unit 213 performs control so that options are displayed on the screen of the display 1605 at the timing when the image becomes stationary.

  FIG. 17 is a timing chart of the driving situation and voice conversation in the second embodiment of the present invention.

  In the example of FIG. 17, since the vehicle was running at a speed of 40 km / h at the time of starting the voice dialogue, the question sentence output timing determination unit 212 reproduces the guidance that prompts the driver to stop. 214 is controlled. Thereafter, the question sentence output timing determination unit 212 controls the voice output unit 214 to output the question sentence after confirming that the vehicle has stopped.

  After confirming that the car has stopped, the image display timing determination unit 213 controls the image display unit 215 to display the options on the screen of the display 1605. Thereafter, voice recognition by the voice recognition unit 203 or the like is started.

  FIG. 18 is a flowchart illustrating an example of processing executed for presenting options in the second exemplary embodiment of the present invention.

  Hereinafter, an example in which the processing of FIG. 18 is executed when a passenger other than the driver is not in the vehicle and the vehicle is traveling will be described.

  First, in the situation determination S1801, the traveling situation determination unit 218 determines whether or not the vehicle can be stopped. For example, the traveling state determination unit 218 may determine that the vehicle cannot be stopped when the current location is on an expressway, and may determine that the vehicle can be stopped when the current location is on a general road.

  When it is determined that the vehicle cannot be stopped, in the guidance output 1_S1802, the option presenting means determination unit 106 outputs a guidance sound when the vehicle cannot be stopped. For example, the option presenting means determination unit 106 may control the question sentence generation unit 210 to output a voice for reading out the question sentence when there is no passenger shown in FIG.

  Next, in the option reading aloud in step S1804, the option presenting means determination unit 106 outputs a voice for reading the option. Specifically, based on an instruction from the option presenting means determination unit 106, the question sentence generation unit 110 generates a question sentence including the options to be output, and the voice output unit 112 reads out the voice that reads out the generated question sentence. Causes the speaker 1601 to output.

  On the other hand, when it is determined that the vehicle can be stopped, in the guidance output 2_S1803, the option presenting means determination unit 106 outputs a guidance sound when the vehicle can be stopped. For example, the option presenting means 106 may control the question sentence generation unit 210 so as to output a guidance for guiding the driver to stop the vehicle at a place where the driver can rest.

  Next, in stop determination S1805, the question sentence output timing determination unit 212 determines whether or not the vehicle has stopped. If it is determined that the vehicle has not yet stopped, the process returns to guidance output 2_S1803, and guidance for instructing to stop the vehicle is output again.

  On the other hand, when it is determined that the vehicle has stopped, the question message output timing determination unit 212 outputs a guidance voice when the vehicle stops in guidance output 3_S1806. For example, the question sentence output timing determination unit 212 controls the voice output unit 214 to output a voice that is read out by the question sentence generation unit 210 when a passenger is present (see, for example, FIG. 11). May be. In the above example, there is actually no passenger, but since the vehicle is stopped, safety is not impaired even if the driver looks at the display 1605.

  Further, in stop determination S1805, the image display timing determination unit 213 determines whether or not the vehicle has stopped. If it is determined that the vehicle has not yet stopped, the process returns to guidance output 2_S1803, and guidance for instructing to stop the vehicle is output again.

  On the other hand, when it is determined that the car has stopped, the image display timing determination unit 213 displays the options on the screen of the display 1605 in the guidance output 3_S1806. For example, the image display timing determination unit 213 may control the image display unit 215 so as to display a display image of options generated by the display image generation unit 211 (see, for example, FIG. 12).

  After the option reading aloud S1804 or the guidance output 3_S1806 is executed by voice, the voice recognition S1807 is executed. This step is the same as the speech recognition S1706 of the first embodiment.

  Note that FIG. 18 illustrates an example in which a passenger other than the driver is not in the vehicle and the vehicle is running. For example, also in the present embodiment, as in the first embodiment, the passenger seat determination S1701 shown in FIG. 13 is executed, and the processing shown in FIG. 18 may be started when it is determined that there is no passenger. On the other hand, if it is determined that there is a passenger, the guidance output 1_S 1702 and the option display S 1704 may be executed on the screen as in the first embodiment.

  Further, FIG. 18 may be executed when the vehicle is not traveling. For example, in the situation determination S1801, it may be first determined whether or not the vehicle is traveling, and if it is determined that the vehicle is traveling, it may be determined whether or not the vehicle can be stopped. If it is determined that the vehicle is not traveling, the guidance output 2_S1803 and the stop determination S1805 may be omitted, and the guidance output 3_S1806 may be executed.

  As described above, according to the second embodiment of the present invention, even when only a driver is in a car, an information presentation method that can perform voice conversation in a short time without sacrificing safety is selected. Is done.

  In the first and second embodiments of the present invention described above, the text information indicating the option is presented to the user, and the process in which the user utters one of the options as a response to the presentation has been described as an example. However, these embodiments can be applied to handle the presentation of arbitrary text information and the response from the user to it.

It is a functional block diagram of the voice interactive apparatus of the 1st Embodiment of this invention. It is a block diagram of the hardware constitutions of the voice interactive apparatus of the 1st Embodiment of this invention. It is a flowchart which shows the speech recognition process performed in the 1st Embodiment of this invention. It is explanatory drawing which shows the flow of the detailed process which the sound source separation part of the 1st Embodiment of this invention performs. It is a flowchart which shows the process performed for every time frequency in the sound source separation of the 1st Embodiment of this invention. It is a flowchart which shows the dialogue based on the dialogue script in the first embodiment of the present invention. It is explanatory drawing which shows an example of the speech recognition dictionary used in the 1st Embodiment of this invention. It is explanatory drawing which shows an example of the word list | wrist for every arc in the 1st Embodiment of this invention. 1 is a block diagram of a hardware configuration of a car navigation system including a voice interaction apparatus according to a first embodiment of the present invention. It is a flowchart which shows the process which the option presentation means determination part of the 1st Embodiment of this invention performs. It is explanatory drawing which shows the example of the guidance audio | voice output in the 1st Embodiment of this invention. It is explanatory drawing which shows the example of the option displayed on a display in the 1st Embodiment of this invention. It is a flowchart which shows an example of the process performed for the presentation of the choice in the 1st Embodiment of this invention. It is a flowchart which shows the passenger's name registration process in the 1st Embodiment of this invention. It is a flowchart which shows the process which the function control part of the 1st Embodiment of this invention performs. It is a functional block diagram of the voice interactive apparatus of the 2nd Embodiment of this invention. It is a driving | running | working condition in the 2nd Embodiment of this invention, and a timing chart of voice dialogue. It is a flowchart which shows an example of the process performed for the presentation of the choice in the 2nd Embodiment of this invention.

Explanation of symbols

101, 201 Voice input unit 102, 202 Sound source separation unit 103, 203 Speech recognition unit 104, 204 Recognition result rejection determination unit 105, 205 Separation range setting unit 106, 206 Option presentation means determination unit 107, 207 Dialog control unit 108, 208 Recognition dictionary generation unit 109, 209 Function control unit 110, 210 Question sentence generation unit 111, 211 Display image generation unit 112, 214 Audio output unit 113, 215 Image display unit 114, 216 Dialogue script DB
115, 217 Recognition vocabulary DB
212 Question sentence output timing determination unit 213 Image display timing determination unit 218 Travel condition determination unit S301 Command name S302 Destination setting S303 Car equipment operation S304 Neighboring facility search S305 Time-frequency decomposition S701 Passenger determination S702 Task end time determination S703 Shortest end time Means selection S801 Passenger seat sensor check S802 Name recognition S803 Rear seat sensor check S804 Name recognition S901 Target sound range setting S902 Sound source separation 1001 DFT
1002 Vectorization per frequency 1003 Sound source localization 1004 Sound source separation 1005 Inverse DFT
S1101 Target sound range S1102 Target sound covariance update S1103 Noise covariance update S1104 Filter generation S1105 Filtering S1106 Post filtering 1201 Microphone array 1202 AD converter 1203, 1603 Central processing unit 1204 Volatile memory 1205, 1602 Storage medium S1501 Path setting Command S1502 Air-conditioner operation command S1503 Speaker control command 1601 Speaker 1604 Seat sensor 1605 Display 1606 Microphone 1607 Speed sensor

Claims (10)

An occupant detection unit for detecting the position of the occupant in the vehicle;
An audio output unit for converting text information into audio and outputting it;
An image display unit for converting the text information into an image and displaying the image,
Multiple microphones,
A switching unit that selects either the voice output unit or the image display unit based on the detection result of the occupant detection unit and outputs the text information to the selected voice output unit or the image display unit. and, with a,
When an occupant other than the driver is in the vehicle, a first time required for displaying the text information on the image display unit, and a second time required for outputting from the voice output unit a voice that reads out the text information. Estimate time
When the first time is shorter than the second time, it is selected to output the text information to the image display unit, and when the second time is shorter than the first time, the text information is output to the voice output unit. Select to output
When an occupant other than the driver is on board, further, a voice prompting the occupant other than the driver to respond is output from the audio output unit,
After the voice prompting the crew member other than the driver to respond is output, when the plurality of microphones receives the voice, the sound source direction of the received voice is specified,
Determining whether the identified sound source direction is within a predetermined range including a direction from the plurality of microphones to an occupant other than the driver;
When the specified sound source direction is within the predetermined range, the received voice is processed as a response to the text information displayed on the image display unit,
When a passenger other than the driver is not in the vehicle , the voice interaction device is selected to output the text information to the voice output unit . The automobile further includes a speed sensor that outputs information indicating whether or not the automobile is running,
  The voice interaction device
  Based on the output from the speed sensor, it is determined whether the automobile is running,
  When it is determined that the automobile is running, a voice prompting the driver to stop the automobile is output from the voice output unit,
  The voice interactive apparatus according to claim 1, wherein when it is determined that the automobile is stopped, an image including the text information is displayed on the image display unit. The spoken dialogue device determines whether or not the vehicle is running when no passenger other than the driver is on board. When the passenger other than the driver is on board, the voice vehicle travels. The voice interactive apparatus according to claim 2, wherein it is not determined whether or not it is in the middle. The automobile further includes a positioning device that acquires current position information of the automobile,
  The voice interaction device
  Based on the current position information of the vehicle acquired by the positioning device, determine whether the vehicle can be stopped,
  When it is determined that the vehicle can be stopped, a voice prompting the driver to stop the vehicle is output from the voice output unit,
  The voice interactive apparatus according to claim 2, wherein when it is determined that the automobile cannot be stopped, a voice that reads out the text information is output from the voice output unit. 5. The voice dialogue apparatus according to claim 4, wherein when it is determined that the automobile is traveling on an expressway based on the current position information, the voice interactive apparatus determines that the automobile cannot be stopped. The spoken dialogue apparatus described. In a voice dialogue method by a voice dialogue device comprising: a voice output unit that converts text information into voice and outputs; an image display unit that converts the text information into an image and outputs; and a plurality of microphones.
  An occupant detection procedure for detecting the position of the occupant in the vehicle;
  Based on the position of the occupant detected by the occupant detection procedure, either the voice output unit or the image display unit is selected, and the selected voice output unit or the image display unit outputs the text information. A switching procedure to
  In the switching procedure, when an occupant other than the driver is on board, the first time required to display the text information on the image display unit, and the voice output unit that reads out the text information from the voice output unit A second time required for output is estimated, and when the first time is shorter than the second time, it is selected to output the text information to the image display unit, and the second time is shorter than the first time. A case in which the text information is selected to be output to the voice output unit, and when an occupant other than the driver is not in the vehicle, the text information is selected to be output to the voice output unit.
  The voice interaction method further includes:
  When it is determined that an occupant other than the driver is on board, a procedure for outputting a voice prompting the occupant other than the driver to respond from the audio output unit;
  After the voice prompting the passengers other than the driver to respond is output, when the plurality of microphones receive the voice, a procedure for specifying the sound source direction of the received voice;
  Determining whether the identified sound source direction is within a predetermined range including a direction from the plurality of microphones to an occupant other than the driver;
  And a procedure for processing the received voice as a response to the text information displayed on the image display unit when the specified sound source direction is within the predetermined range. . The automobile further includes a speed sensor that outputs information indicating whether or not the automobile is running,
  The voice interaction method further includes:
  A procedure for determining whether or not the vehicle is running based on an output from the speed sensor;
  When it is determined that the automobile is running, a procedure for outputting a voice prompting the driver to stop the automobile from the voice output unit;
  The voice interaction method according to claim 6, further comprising: displaying an image including the text information on the image display unit when it is determined that the automobile is stopped. The procedure for determining whether or not the vehicle is running is executed when an occupant other than the driver is not on board, and is not executed when an occupant other than the driver is on board. The voice interaction method according to claim 7, wherein: The automobile further includes a positioning device that acquires current position information of the automobile,
  The voice interaction method further includes:
  A procedure for determining whether or not the automobile can be stopped based on the current position information of the automobile acquired by the positioning device;
  When it is determined that the vehicle can be stopped, a procedure for outputting a message prompting the driver to stop the vehicle from the voice output unit;
  The voice interaction method according to claim 7, further comprising: a step of outputting a voice for reading the text information from the voice output unit when it is determined that the automobile cannot be stopped. The procedure for determining whether or not the vehicle can be stopped is based on the current position information, and when it is determined that the vehicle is traveling on a highway, the vehicle cannot be stopped. The voice interaction method according to claim 9, further comprising a determination procedure.

Images