JP6558180B2 - Information providing apparatus and information providing control program - Google Patents

Description

  The present invention relates to an information providing apparatus that provides information to a user by voice, and an information provision control program that controls information provision by voice.

  Conventionally, as a kind of information providing device, for example, Patent Document 1 discloses a navigation device that provides information such as the current position, traveling direction, and route of a vehicle to a driver as a user by voice. This navigation device determines whether or not the user is speaking, and does not perform voice guidance when the user is speaking.

JP 2002-156241 A

  Now, since the navigation device of Patent Document 1 only controls the start of voice guidance based on whether or not the user is speaking, for example, even when a break occurs in the user's conversation, voice guidance is started. It will be. Therefore, when the conversation of the user who has been excited is accidentally interrupted, the voice guidance by the navigation device is immediately reproduced at the break. As a result, the user who was blocked from the conversation was easily discomforted with the voice guidance by the navigation device.

  The present invention has been made in view of such problems, and an object thereof is to provide an information providing apparatus and an information providing control program capable of providing information by voice without causing discomfort to the user. There is to do.

In order to achieve the above object, one disclosed invention is an information providing apparatus that provides information to a user by using sound output from the sound output device (30), and acquires information provided to the user. An information acquisition unit (62), a conversation analysis unit (70) that determines whether or not the user is in conversation, and detects the end of the conversation when the user is in conversation, and a conversation analysis unit And a provision control unit (80) for starting provision of information by voice of the voice output device after waiting for the waiting time (T) to elapse after the end of the conversation is detected by the conversation analysis unit, estimating the swelling degree of conversation that the user is performed, it provides the control unit, as are raised conversation being conducted, and the information providing device to set a longer waiting time.
Moreover, one disclosed invention is an information providing device that provides information to a driver as a user who drives a vehicle by means of sound output from the sound output device (30), and is provided to the user. An information acquisition unit (62) for acquiring a message, a conversation analysis unit (70) for determining whether the user is in conversation, and detecting the end of the conversation when the user is in conversation, A provision control unit (80) that waits for the elapse of a waiting time (T) after the conversation analysis unit detects the end of the conversation, and then starts providing information by voice from the voice output device, and obtains information The unit obtains the random information that determines whether or not the driver is in a random state from the random determination unit (51), and the provision control unit determines that the driver is in the random state by the random determination unit To set the waiting time short based on The test device.
Moreover, one disclosed invention is an information providing device that provides information to a driver as a user who drives a vehicle by means of sound output from the sound output device (30), and is provided to the user. An information acquisition unit (62) for acquiring a message, a conversation analysis unit (70) for determining whether the user is in conversation, and detecting the end of the conversation when the user is in conversation, A provision control unit (80) that waits for the elapse of a waiting time (T) after the conversation analysis unit detects the end of the conversation, and then starts providing information by voice from the voice output device, and obtains information The unit acquires load information that estimates the height of the driving load of the driver from the load estimation unit (52), and the provision control unit sets the standby time longer as the driving load estimated by the load estimation unit increases. Information providing device to be used.

  According to the present invention, even when a break occurs in the user's conversation, the provision of information by voice is not started immediately, but is started after waiting time has elapsed. Therefore, when the conversation of the user who has been excited is accidentally interrupted, the situation where the voice is immediately reproduced at the break can be prevented. Based on the above, the information providing apparatus can provide information by voice without causing discomfort to the user.

Another disclosed invention is an information provision control program for controlling provision of information to a user using a voice output from the voice output device (30), wherein information provided to the user is stored. An information acquisition step (S101) to be acquired, a conversation analysis step (S107) for determining whether or not the user is in conversation, and detecting the end of the conversation when the user is in conversation, and a conversation After the end of the conversation is detected by the analysis step, a provision control step (S108, S109) for starting provision of information by voice from the voice output device after waiting for the waiting time (T) to elapse, at least one processor (60a), in the conversation analysis step, the degree of excitement of the conversation being performed by the user is estimated, and in the provision control step, the ongoing conversation is excited The more I, the information provision control program to set a longer waiting time.
Another disclosed invention is an information provision control program for controlling provision of information to a user using a voice output from the voice output device (30), wherein information provided to the user is stored. An information acquisition step (S101) to be acquired, a conversation analysis step (S107) for determining whether or not the user is in conversation, and detecting the end of the conversation when the user is in conversation, and a conversation After the end of the conversation is detected by the analysis step, a provision control step (S10) that starts providing information by voice from the voice output device after waiting for the waiting time (T) to elapse
8, S109) is executed by at least one processor (60a), and in the information acquisition step, the random information that determines whether or not the driver as the user who drives the vehicle is in a random state is acquired and provided. In the control step, an information provision control program for setting the standby time short is based on the random information determined that the driver is in the random state.
Another disclosed invention is an information provision control program for controlling provision of information to a user using a voice output from the voice output device (30), wherein information provided to the user is stored. An information acquisition step (S101) to be acquired, a conversation analysis step (S107) for determining whether or not the user is in conversation, and detecting the end of the conversation when the user is in conversation, and a conversation After the end of the conversation is detected by the analysis step, a provision control step (S10) that starts providing information by voice from the voice output device after waiting for the waiting time (T) to elapse.
8 and S109), and at least one processor (60a) is executed, and in the information acquisition step, load information obtained by estimating the height of the driving load of the driver as the user driving the vehicle is acquired, and the providing control step Then, it is set as the information provision control program which sets standby time long, so that the driving | running load which load information shows is high.

  According to the information provision control program described above, it is possible to prevent a situation in which voice is immediately reproduced at an accidental break between conversations, and to provide information by voice without causing discomfort to the user.

  Note that the reference numbers in the parentheses are merely examples of correspondences with specific configurations in the embodiments to be described later in order to facilitate understanding of the present invention, and limit the scope of the present invention. It is not a thing.

It is a block diagram which shows the whole structure of the information provision apparatus by one Embodiment. It is a figure explaining the functional block and subblock constructed | assembled in a control circuit. It is a flowchart which shows the provision execution process implemented in a control circuit. It is a flowchart which shows the process which adjusts standby | waiting time according to the duplication rate of conversation. It is a flowchart which shows the process which adjusts waiting time according to the sentence end expression of conversation. It is a flowchart which shows the process which adjusts standby | waiting time according to a driver | operator's casual state. It is a flowchart which shows the process which adjusts standby | waiting time according to a driver | operator's driving load.

  An information providing apparatus 100 according to an embodiment of the present invention shown in FIG. 1 is mounted on a vehicle, and can provide information to a driver as a user who drives the vehicle by voice. The information providing apparatus 100 can present news information acquired from the Internet to the driver in addition to normal vehicle information such as route guidance and fuel remaining amount warning. The information providing apparatus 100 can play a role of recovering the awakening level of a driver who is likely to fall into a general state of awakening by providing information by voice.

  The information providing apparatus 100 is electrically connected to the microphone 20, the audio output device 30, the in-vehicle ECU group 40, the navigation device 41, the communication device 42, the state detection device 50, and the like.

  The microphone 20 is an audio input device provided in the vehicle interior. The microphone 20 converts the voice of the conversation uttered by the vehicle occupant into an electrical signal, and outputs it as voice data to the information providing apparatus 100. The microphone 20 may be configured for a call provided in a communication device such as a smartphone and a tablet terminal. Further, the audio data collected by the microphone 20 may be wirelessly transmitted to the information providing apparatus 100.

  The audio output device 30 has a function of an output interface that outputs information to the driver. The audio output device 30 includes an amplifier 31 and a speaker 32. The amplifier 31 acquires audio data from the information providing apparatus 100. The amplifier 31 drives the speaker 32 based on the acquired audio data. The speaker 32 is provided in the passenger compartment, and reproduces voice guidance so that it can be heard by passengers of the vehicle including the driver. The driver can acquire information from the sound output from the speaker 32. The audio output device 30 may be, for example, an acoustic device provided in the vehicle interior, or may be a communication robot, a smartphone, a tablet terminal, or the like.

  The in-vehicle ECU group 40 is configured mainly with a microcomputer or the like, and includes an integrated control ECU, an engine control ECU, a brake control ECU, and the like. From the in-vehicle ECU group 40, information indicating the fuel remaining amount, information indicating the traveling speed of the host vehicle, and operation information indicating the operation amount of the accelerator pedal, the steering wheel, and the like are output to the information providing device 100 and the state detection device 50. Is done.

  The navigation device 41 can create a planned travel route to the destination input by the driver or the like, and can provide guidance to the driver according to the planned travel route. The navigation device 41 includes a map database that stores road shape information and the like, a GNSS (Global Navigation Satellite System) receiver that acquires current position information of the host vehicle, and the like. The navigation device 41 instructs the information providing device 100 to perform voice route guidance. As a result, voice guidance for route guidance such as “700 m ahead, right” is output from the voice output device 30. Further, the navigation device 41 can provide the state detection device 50 with information indicating the road shape ahead of the vehicle. In addition, the navigation device 41 can reproduce sound in the passenger compartment by outputting a part of the sound data to the sound output device 30.

  The communicator 42 is configured to have an antenna for mobile communication. The communication device 42 may be a vehicle communication module such as a DCM (Data Communication Module), or may be a communication terminal such as a smartphone brought into the vehicle by the driver. The communication device 42 transmits / receives information to / from a base station outside the vehicle via the antenna. The communication device 42 can be connected to the Internet through a base station, and can acquire various information through the Internet. For example, the communicator 42 can acquire e-mail addressed to passengers of the vehicle, news information, traffic information indicating the degree of congestion around the current location where the host vehicle is traveling, and the like.

  The state detection device 50 is mainly configured by a microcomputer having a processor, a RAM, a flash memory, and an input / output interface. The state detection device 50 acquires information related to the driver's state from the configuration including the in-vehicle ECU group 40, the navigation device 41, and an in-vehicle camera such as a driver status monitor, and estimates the driver's state. The state detection device 50 constructs the random determination unit 51 and the load estimation unit 52 as functional blocks by executing a predetermined state detection program by the processor.

  The random determination unit 51 is a functional block that determines whether or not the driver is in a random state. Based on the transition of the operation information of the accelerator pedal and the steering wheel acquired from the in-vehicle ECU group 40, the casual determination unit 51 detects a slow operation of these operation systems and a large correction operation that is occasionally input. It is determined that the person is in a random state.

  Further, the casual determination unit 51 acquires information such as the line-of-sight direction of the driver's eyes and the degree of eye opening from the image around the driver's face photographed by the driver status monitor. When the parallax determination unit 51 is unstable in the parallax of both eyes or is not suitable for the perception of an object in the traveling direction, or when the low eye opening state continues, the driver Is determined to be in a random state.

  The load estimation unit 52 is a functional block that estimates whether or not the driver's driving load is high on the road on which the vehicle is currently traveling. When the load estimation unit 52 estimates that the driver is busy operating at least one of the accelerator pedal and the steering wheel operation information acquired from the in-vehicle ECU group 40, the current driving load is Judge as high. On the other hand, when the fluctuations in the operation amounts of the steering and the accelerator pedal are slight, the load estimation unit 52 determines that the driving load is low.

  In addition, the load estimation unit 52 can acquire, from the navigation device 41 and the communication device 42, the shape information of the road to be traveled and the traffic information indicating the degree of congestion around the host vehicle, respectively. The load estimation unit 52 determines that the current driving load is high when the road in the traveling direction has a curved shape and when it is estimated that the vehicle is traveling in a traffic jam. On the other hand, when the vehicle is traveling on a substantially straight road and there are few other vehicles traveling around, the load estimation unit 52 determines that the current driving load is low.

  Next, the configuration of the information providing apparatus 100 will be described. The information providing apparatus 100 includes a control circuit 60 mainly composed of a processor 60a, a RAM, a flash memory, and a microcomputer having an input / output interface. The control circuit 60 is a circuit that integrally controls the provision of information to the driver using the sound output from the speaker 32. The control circuit 60 constructs the voice recognition unit 61, the information acquisition unit 62, the conversation analysis unit 70, and the provision control unit 80 as functional blocks by executing a predetermined information provision control program by the processor 60a. Hereinafter, details of each functional block constructed in the control circuit 60 will be described with reference to FIGS. 1 and 2.

  The voice recognition unit 61 is capable of voice recognition and acquires the contents of the utterances of the vehicle occupants including the driver. The voice recognition unit 61 is connected to the microphone 20. The voice recognition unit 61 collects voices uttered by each occupant of the vehicle, such as a driver who is a user and a passenger in a passenger seat as a conversation person who talks with the driver, by the microphone 20, and voice data of the utterances To get. The voice recognition unit 61 reads the acquired voice data and converts it into text data. In addition, the voice recognition unit 61 also extracts voice volume information indicating the volume of each passenger's voice in the utterance from the voice data. The voice recognition unit 61 converts words spoken by each user in the passenger compartment, such as a driver's monologue and a conversation between passengers, into text data and provides the voice analysis information to the conversation analysis unit 70.

  The information acquisition unit 62 acquires various provision information that needs to be provided to the driver from the in-vehicle ECU group 40, the navigation device 41, the communication device 42, and the state detection device 50. The provided information includes, for example, user support information, alert information for warning of a failure, a gas shortage, etc., driving support information related to ACC (Adaptive Cruise Control), etc., and travel support information such as intersection guidance of the navigation device 41. Furthermore, in addition to these normal vehicle information, system trigger information that reads out new mail, new news information, and the like is also included in the provided information. The user trigger information is information presented as feedback for the driver operating or attempting to operate the vehicle-mounted device. In addition, the information acquisition unit 62 includes the random information that determines whether or not the driver is in a random state and the driving load information that estimates the height of the driving load of the driver, respectively. 52.

  The conversation analysis unit 70 acquires the content of the utterance performed between the passengers of the vehicle from the information acquisition unit 62, and analyzes the content of the utterance of each passenger. The conversation analysis unit 70 includes a conversation detection unit 71, a duplication rate calculation unit 73, a sentence end determination unit 74, an inflection determination unit 75, and a participation rate calculation unit 76 as sub-blocks.

  The conversation detection unit 71 determines whether each occupant including the driver is in conversation. When the information acquisition unit 62 acquires the information to be provided to the driver, the conversation detection unit 71 determines whether each occupant is in conversation. And when crew members are in conversation, the end of this conversation is detected. The determination result and the detection result by the conversation detection unit 71 are provided to the provision control unit 80.

  The duplication rate calculation unit 73, the sentence end determination unit 74, the inflection determination unit 75, and the participation rate calculation unit 76 are blocks for estimating the degree of excitement of conversation performed in the vehicle. The duplication rate calculation unit 73 calculates the duplication rate of words spoken by each occupant. First, an example of a method for calculating the overlap rate calculated by the overlap rate calculation unit 73 will be described.

A word that appears n times per unit time or per specific number of utterance turns is defined as C _n . Furthermore, the total number of words appearing n times or specific utterances per turn number per unit time is defined as C _N. Note that C _N = n · C _n . Using these, the number X of word types that appear per unit time or per specific number of utterance turns, the total number Y of words that appear, the number A of word types that appear more than once, and the appearance more than once The total number B of the types of words to be performed is expressed by the following formulas 1 to 4, respectively.

Furthermore, by using the above X, Y, A, and B, the overlapping rate can be defined as the following formulas 5 to 7, for example.

The above R ₁ is a numerical value of 1 or more, and is a value indicating the ratio of how many times the word is used. R ₂ is a value indicating the ratio of the total number of words that have appeared twice or more of the total number of words that have appeared. This R _2, the word that appeared more than once is the more, gradually approaches to 1. Further R _3, of the number of occurrences words, a value that indicates the ratio of the number of words that appeared more than once. This R ₃ also approaches 1 as the number of words that appear more than once increases.

Using the above mathematical formulas 1 to 7, for example, the following conversation overlap rate is calculated.
User C: Do you know tomorrow's weather?
User D: It looks fine tomorrow.
User C: That's right.
User D: Jua, is it sunny next week?
User C: How about it? I want the weather to be sunny recently.

  First, morphological analysis is performed on each user's utterance in the conversation. As a result, the first utterance was divided into meaningful minimum units (words), such as "Tomorrow (noun)", "no (particle)", "weather (noun)", "understand (verb)". Then, each part of speech is discriminated. The second to fifth sentences are processed in the same way.

Through the above processing, the words that appear in the conversation and the number of appearances of each word are acquired. Specifically, information is extracted such that the words “da” and “sunny” each appear three times in the conversation, and “ha”, “weather”, and “tomorrow” each appear twice in the conversation. The As a result, C ₃ = 2, C ₂ = 3, and C ₁ = 19.

Furthermore, based on the above formulas 1 to 4, the number of types of words that appear; X = 19 + 3 + 2 = 24, the total number of words that appear; Y = 1 × 19 + 2 × 3 + 3 × 2 = 31, the number of words that appear more than once Number of types: A = 3 + 2 = 5. Then, the total number of words appearing twice or more; B = 2 × 3 + 3 × 2 = 12. Therefore, the duplication rate is based on the above formulas 5 to 7, respectively. R ₁ = Y / X = 31/24 = 1.29, R ₂ = B / Y = 12/31 = 0.39, R ₃ = A / X = 5/24 = 0.21.

The duplication rate calculation unit 73 estimates the degree of excitement of conversation by comparing at least one of the above with a preset threshold value. Specifically, the value of R ₁ is 1 or more, and increases as the overlap rate increases. Therefore, when the value of R ₁ is greater than the threshold, overlap rate calculation unit 73 estimates and conversation is raised. Similarly, when each value of R ₂ and R ₃ is larger than each threshold value set as each comparison target and close to 1, the duplication rate calculation unit 73 estimates that the conversation is exciting. .

  The sentence end determination unit 74 determines whether or not an expression suggesting continuation of the conversation is included at the end of the utterance in the conversation between the passengers. The sentence end determination unit 74 extracts the sentence end expression of the utterance and compares it with a preset sentence end expression. When the sentence end expression of the utterance matches or resembles a preset sentence end expression, the sentence end determination unit 74 estimates the continuation of the conversation.

  The intonation determining unit 75 determines the amount of inflection in each passenger's utterance based on the volume information output from the speech recognition unit 61. The intonation determining unit 75 determines that the intonation is large, for example, when the overall utterance volume is larger than usual, or when there is a point where the voicing volume increases during utterance.

  The participation rate calculation unit 76 calculates the participation rate of each occupant in the conversation. The participation rate is the ratio of the occupant's utterance time in the utterance time of each occupant. The smaller the difference between the calculated participation rates of each passenger, the more actively speaking with each other, and it is considered that the conversation between the passengers is exciting.

  The provision control unit 80 obtains provision information to be provided to the driver from the information acquisition unit 62, and causes the voice output device 30 to reproduce the guidance voice for guiding the provision information at an optimal timing. The provision control unit 80 includes a priority determination unit 81, a time limit setting unit 82, a sound generation unit 83, an output control unit 84, and a standby time setting unit 85 as sub blocks.

  The priority determination part 81 determines whether the information acquired by the information acquisition part 62 is the priority provision information which should be alert | reported immediately to a driver | operator. The priority provision information with high priority is notified to the driver without adjusting the provision timing. The priority provision information includes the above-described user trigger information, alert information that requires an emergency such as warning and attention, and the like. Here, when the presentation of the user trigger information is delayed, the user feels uncomfortable due to the poor responsiveness of the system or misunderstands that the system is not operating. Therefore, it is desirable that the user trigger information, which is feedback to the user operation, be presented quickly. On the other hand, the priority of provision information mainly related to user convenience, such as driving support information, travel support information, and system trigger information, is set low.

  Moreover, the priority determination part 81 changes the level of the priority to set according to a condition, even if it is the same information. For example, when the planned route is guided, two pieces of provision information “700 m ahead, right” and “300 m ahead, right” are acquired. If the information “700 m ahead, right” is transmitted to the driver, the priority of the information “300 m ahead, right” may be set low. However, when the information “700 m ahead, right” is not transmitted to the driver, the information “300 m ahead, right” is set to the priority provision information with a high priority and is immediately provided to the driver.

  The deadline setting unit 82 sets a presentation deadline for each provision information provided from the information acquisition unit 62. The deadline setting unit 82 sets, for example, a spatial presentation deadline based on a point such as a specific intersection, and a temporal presentation deadline based on time such as the date and time or the elapsed time since the provision became possible. Is possible.

  For example, a spatial presentation deadline is set for the movement support information for guiding an intersection. As a result, this movement support information becomes expired information when it passes through the intersection and is rejected. On the other hand, the temporal presentation deadline is set longer for information with lower priority. For example, the driving support information related to ACC is information that can be provided until the ACC stops operating. Therefore, a long presentation deadline can be set in the driving support information related to ACC. The time limit setting unit 82 provides the output control unit 84 with the set presentation time limit associated with each provision information.

  The sound generation unit 83 generates sound data that is reproduced by the speaker 32. When normal vehicle information such as user trigger information, alert information, and driving support information is provided to the driver, the voice generation unit 83 notifies vehicle information from a plurality of voice data for guidance stored in advance. Select audio data corresponding to.

  On the other hand, when system trigger information that reads out new mail and news information is provided to the driver, the voice generation unit 83 synthesizes voice data of the guidance using each text data of the mail and news information. More specifically, the voice generation unit 83 generates prosodic data for utterance from text data of mail and news information. Then, the voice generation unit 83 connects the voice waveform data in accordance with the prosodic data from the voice waveform database stored in advance. Through the above process, the voice generation unit 83 can convert text data of mail and news information into voice data. When the amount of information of the mail and news information is large, the voice generation unit 83 can summarize the original information and convert it to voice data.

  The output control unit 84 outputs the voice data selected or generated by the voice generation unit 83 toward the voice output device 30, thereby causing the speaker 32 to speak the guidance. The output control unit 84 can adjust the timing of providing information by voice. For example, when priority providing information with a high priority is acquired, the output control unit 84 starts providing priority providing information by voice even if the passengers are in conversation.

  On the other hand, when provision information with low priority is acquired, the output control unit 84 first determines whether or not passengers are in conversation with each other. When the passengers are talking with each other, the output control unit 84 refrains from reporting the provided information by voice. Then, after the end of the conversation is detected by the conversation detection unit 71, the output control unit 84 causes the speaker 32 to reproduce the guidance voice for notifying the provided information after waiting for the standby time T to elapse. Further, when the conversation detecting unit 71 detects the resumption of the conversation before the elapse of the waiting time T, the output control unit 84 can hold the start of providing information by voice and wait for the detection of the end of the conversation again.

  The standby time setting unit 85 sets the length of the standby time T used by the output control unit 84. The waiting time T is adjusted, for example, between several hundred milliseconds to several seconds. The standby time setting unit 85 adjusts the standby time T based on the estimation result of the degree of excitement of conversation conducted by each occupant including the user. The waiting time T is set longer as the ongoing conversation increases. The standby time setting unit 85 estimates the degree of excitement by combining a plurality of methods.

  Specifically, the waiting time setting unit 85 sets the waiting time T longer as the word duplication rate calculated by the duplication rate calculation unit 73 increases. In addition, the standby time setting unit 85 sets the standby time T to be longer when the sentence end determination unit 74 determines that there is an expression suggesting continuation of the conversation at the end of the sentence. The standby time setting unit 85 sets the standby time T longer as the inflection in each passenger's utterance determined by the inflection determination unit 75 increases. Furthermore, the standby time setting unit 85 sets the standby time T to be longer as the participation rate of each occupant calculated by the participation rate calculation unit 76 is nearly equal.

  In addition, the standby time setting unit 85 can adjust the standby time T in consideration of the state of the driver. Specifically, the waiting time setting unit 85 sets the waiting time T to be short based on the fact that the driver determining unit 51 determines that the driver is in a maneuvering state. Furthermore, the standby time setting unit 85 sets the standby time T to be shorter as the driving load of the driver estimated by the load estimation unit 52 is lower.

  Details of each process performed by the control circuit 60 will be further described. First, the details of the provision execution process for controlling the provision of information to the driver will be described with reference to FIGS. 1 and 2 based on FIG. Each step of the providing execution process shown in FIG. 3 is mainly performed by the output control unit 84. The providing execution process is started based on the fact that the power source of the vehicle is turned on and the power supply to the information providing apparatus 100 is started, and is repeated until the power source of the vehicle is turned off.

  In S101, it is determined whether or not provision information to be provided to the driver has been acquired. If it is determined in S101 that the provided information has not been acquired, the determination in S101 is repeated to wait for the provided information. And if provision information is acquired, it will progress to S102.

  In S102, the priority of the provided information acquired in S101 is determined. If it is determined in S102 that priority providing information with a high priority has been acquired, the process proceeds to S104. In S104, the reproduction of the guidance for notifying the driver of the priority provision information acquired in S101 is started by the speaker 32, and the process returns to S101. On the other hand, if it is determined in S102 that the priority of the acquired provided information is low, the process proceeds to S103.

  In S103, it is determined whether or not an occupant including a driver is talking. If it is determined in S103 that the occupants are not in conversation with each other, the process proceeds to S104, and guidance for notifying the provided information is started. On the other hand, if it is determined in S103 that the drivers are talking, the process proceeds to S105.

  In S105, the presentation deadline of the provision information set by the deadline setting unit 82 is read, and it is determined whether or not it is within the presentation deadline. In S105, when the provision time limit of the provided information has expired, the process proceeds to S106. In S106, the provision information acquired in S101 is rejected, and the process returns to S101. On the other hand, if it is determined in S105 that the provided information is within the presentation deadline, the process proceeds to S107.

  In S107, it is determined whether or not the conversation detecting unit 71 detects the end of the conversation. If it is determined in S107 that the end of the conversation has not been detected, the end of the conversation is awaited by repeating S105 and S107. If the end of the conversation is detected before the presentation deadline expires, the process proceeds to S108.

  In S108, the elapsed time from the end of the conversation is measured, and it is determined whether or not this elapsed time exceeds the waiting time T. If it is determined in S108 that the standby time T has elapsed, the process proceeds to S109. In S109, similarly to S104, the reproduction of the guidance for notifying the driver of the provided information acquired in S101 is started by the speaker 32, and the process returns to S101. On the other hand, if it is determined in S108 that the standby time T has not elapsed, the process proceeds to S110.

  In S110, it is determined whether the conversation has been resumed before the waiting time T has elapsed. If it is determined in S110 that the silent state continues and the conversation has not been resumed after the talks of the drivers, the process waits for the waiting time T to elapse by repeating S108 and S110. On the other hand, when the conversation is resumed before the waiting time T has elapsed, the process returns to S105 and waits for the resumed conversation to end.

  Next, a plurality of time setting processes for setting the standby time T used for the determination in S108 will be described based on FIGS. 4 to 7 and with reference to FIG. Each time setting process is performed by linking the sub-blocks (73 to 76) of the conversation analysis unit 70 and the standby time setting unit 85. Each time setting process is started along with the start of power supply to the information providing apparatus 100 (see FIG. 1), and is repeated until the power supply is stopped, similarly to the provision execution process.

Note that the waiting time T for use in the determination of S108 may be the maximum value of T _4, etc. from the mean value in a at best, or T ₁ of the such T ₄ from T ₁ of the following. Furthermore, a value obtained by adding the sum of the following ΔT ₁ to ΔT ₄ to the reference standby time may be used as the standby time T used for the above determination.

The flowchart of FIG. 4, details of the waiting time setting process of adjusting the standby time T ₁ in accordance with the overlap ratio of the conversation is shown. In S121, the waiting time _{T 1} is initialized, and the process proceeds to S122. In S122, the utterances of each occupant converted into text data by the voice recognition unit 61 are acquired, and the process proceeds to S123. In S123, the overlap rate calculation unit 73 performs the morphological analysis and the overlap rate calculation described above, and the process proceeds to S124.

In S124, it is determined whether or not the duplication rate calculated in S123 is lower than a preset threshold value. When the calculated duplication rate is lower than the threshold value, the process proceeds to S125. In S125, the waiting time T1, and it sets the preset reference value (overline ¯ in Figure 4, _{T 1),} the flow returns to S122. On the other hand, when it is determined in S124 that the duplication rate is larger than the threshold value, the process proceeds to S126. In S126, a waiting time _{T 1,} is set in advance has been greater value by [Delta] T ₁ than the reference value, the flow returns to S122.

The flowchart of FIG. 5 shows the details of the time setting process for adjusting the waiting time T2 according to the expression at the end of the conversation. In S141, the waiting time _{T 2} is initialized, and the process proceeds to S142. In S142, the speech of each occupant converted into text data by the voice recognition unit 61 is acquired, and the process proceeds to S143. In S143, the sentence end determination unit 74 extracts a sentence end expression by morphological analysis.

In S144, it is determined whether or not the sentence end expression extracted in S143 corresponds to a specific expression that suggests continuation of the conversation. If it is determined in S144 that the sentence end is not the end of the sentence suggesting continuation of the conversation, the process proceeds to S145. In S145, a waiting time _{T 2,} set preset reference value (overline ¯ in FIG 5 _{T 2),} the flow returns to S142. On the other hand, if it is determined in S144 that the end of the sentence suggests continuation of the conversation, the process proceeds to S146. In S146, a waiting time _{T 2,} is set in advance has been greater value by [Delta] T ₂ than the reference value, the flow returns to S142.

  Here, the standby time T can be adjusted by the same process for the inflection of the utterance and the participation rate of the driver in the conversation. When the inflection determination unit 75 determines that the inflection is large, the standby time setting unit 85 sets a standby time T obtained by adding ΔT to a preset reference value. Similarly, when the participation rate of the driver's conversation calculated by the participation rate calculation unit 76 exceeds a predetermined threshold value, the standby time setting unit 85 adds a standby time by adding ΔT to a preset reference value. Set T.

The flowchart of FIG. 6, the driver absentminded state whether details of the time setting process of adjusting the standby time T ₃ according to is shown. In S161, the waiting time _{T 3} is initialized, and the process proceeds to S162. In S162, the driver's casual information is acquired from the casual determination unit 51, and the process proceeds to S163. In S163, it is determined whether or not the driver is in a sloppy state based on the sloppy information acquired in S162. If it is determined in S163 that the driver is not in a sloppy state, the process proceeds to S164. In S164, a waiting time _{T 3,} set preset reference value (overline ¯ in FIG 6 _{T 3),} the flow returns to S162. On the other hand, if it is determined in S163 that the driver is in a sloppy state, the process proceeds to S165. In S165, a waiting time _{T 3,} set in advance has been smaller by [Delta] T ₃ than the reference value, the flow returns to S162.

The flowchart of FIG. 7, detailed time adjusts the standby time T ₄ in accordance with the operating load of the driver setting process is shown. In S181, the waiting time _{T 4} is initialized, and the process proceeds to S182. In S182, the driving load information of the driver is acquired from the load estimation unit 52, and the process proceeds to S183. In S183, based on the driving load information acquired in S182, it is determined whether or not the current driving load is high. If it is determined in S183 that the operation load is high, the process proceeds to S184. In S184, a waiting time _{T 4,} set preset reference value (overline ¯ in FIG. 7 _{T 4),} the flow returns to S182. On the other hand, when it determines with driving | running | working load being low in S183, it progresses to S185. In S185, a waiting time _{T 4,} set in advance has been smaller by [Delta] T ₄ than the reference value, the flow returns to S182.

  In the present embodiment described so far, the provision of information by voice is not started immediately even when a break occurs in a conversation of an occupant or the like including the user, and is started after waiting for the waiting time T to elapse. Therefore, when conversation between crew members who have been excited is accidentally interrupted, it is possible to prevent a situation where sound is immediately reproduced at the break. Based on the above, the information providing apparatus 100 can provide information by voice without causing discomfort to the user.

  In addition, in the present embodiment, the provision of information is started at the timing when the conversation between the passengers is reliably ended. According to the above, since it is easy to listen to information provided by voice, it becomes possible to improve the cognition of information for the user (driver). As a result, the occurrence of replay by the user is reduced, so that the user's discomfort can be further reduced.

  In addition, in the present embodiment, when the conversation is exciting, the standby time T is set long. The higher the conversation, the higher the probability that it will be resumed even if it is interrupted. Therefore, if the waiting time T is set longer according to the degree of excitement of the conversation, the information providing apparatus 100 can reliably wait for the interruption to the conversation until the end of the conversation. As a result, it is possible to prevent a situation in which the guidance voice is reproduced so as to overlap the resumed conversations of the users. Therefore, the possibility that the user may feel uncomfortable is further reduced.

  In this embodiment, the standby time T is set longer as the word duplication rate increases. It is estimated that the word duplication rate tends to increase as the conversation gets more exciting. Therefore, by adjusting the waiting time T in accordance with the word duplication rate, the information providing apparatus 100 avoids timing that easily overlaps with the utterances of the driver who is a user or another occupant who is a talker, and information by voice You can start offering.

  Furthermore, in the present embodiment, the standby time T is set to be long when an expression suggesting continuation of the conversation is included at the end of the spoken word. As described above, if the standby time T is set long by paying attention to the end of sentence expression of the conversation, the information providing apparatus 100 can start providing information by voice while avoiding a timing that easily overlaps with each passenger's utterance. it can.

  In addition, in the present embodiment, paying attention to the fact that the inflection of the utterance increases in response to the excitement of the conversation, the waiting time T is set longer according to the amount of inflection in the utterance. As a result, the information providing apparatus 100 can start providing information by voice while avoiding a timing that easily overlaps with the user's utterances.

  In the present embodiment, the waiting time T is adjusted based on the participation rate in the conversation, focusing on the participation rate in the conversation between the driver who is the user and another occupant who is the conversation person. If the conversation is exciting, the difference in participation rate in the conversation will be small. On the other hand, when either the driver or the other occupant is unilaterally speaking, the difference in participation rate increases. According to such adjustment, the information providing apparatus 100 can start providing information by voice while avoiding a timing that easily overlaps with the utterances of the users.

  Further, in the present embodiment, when the driver is in a muffled state, the waiting time T is set shorter than in a case where the driver is not muddy. In addition, in the present embodiment, the standby time T is set to be short even in a low driving load state in which a casual state is likely to occur. According to the above settings, information presentation by the information providing apparatus 100 is likely to occur. As a result, the driver's situation is changed by the voice of the guidance, and the effect of increasing the driver's arousal level is easily exhibited.

  In the present embodiment, when the conversation is resumed before the waiting time T elapses, the start of providing information by voice is suspended. Therefore, it is possible to prevent a situation in which the voice is reproduced so as to overlap the conversations of the resumed users. Therefore, the possibility that the user may feel uncomfortable is further reduced.

  Furthermore, according to the present embodiment, when the priority providing information to be notified to the user immediately is acquired, the information providing apparatus 100 starts to notify the priority providing information by voice even when the user is in conversation. Therefore, the user can acquire highly urgent information without delay. In addition, if the conversation is blocked by highly urgent information, the user is less likely to feel discomfort. Therefore, it is desirable that the priority provision information is notified immediately.

  In the present embodiment, S101 in the providing execution process corresponds to an “information acquisition step”, S107 corresponds to a “conversation analysis step”, and S108 and S109 correspond to a “providing control step”.

(Other embodiments)
As mentioned above, although one embodiment by the present invention was described, the present invention is not interpreted limited to the above-mentioned embodiment, and is applied to various embodiments and combinations within the range which does not deviate from the gist of the present invention. be able to.

  The waiting time T in the above embodiment has been adjusted by combining factors such as conversation overlap rate, conversation end-of-sentence expression, intonation, driver participation rate, and driver's casual state and load state. However, the waiting time T may not be determined based on all these factors, and can be determined by any one of these factors or any combination of two or more. Further, the waiting time T may be a predetermined length of time.

  In the above embodiment, the standby time T is adjusted depending on whether or not the predetermined ΔT is added. However, the adjustment method of the standby time T can be changed as appropriate. For example, the standby time T that gradually increases as the overlap rate increases may be set by adding the calculated overlap rate value to the reference value. Alternatively, a plurality of ΔTs having different values may be set in advance, and ΔT corresponding to the amount of inflection may be added to the reference value.

  In the above-described embodiment, the standby time T is adjusted by subtracting ΔT from the reference value when the state of casualness and the driving load are low. However, the waiting time T may be adjusted by a process of adding ΔT to a reference value that is set to a low value in advance when the casual state and the driving load are high.

  In the above embodiment, the priority provision information with a high priority is provided to the immediate user. The target of such priority provision information can be changed as appropriate. Further, the determination of priority may be omitted.

  In the above embodiment, each function related to information provision provided by the processor 60a of the information provision apparatus 100 may be realized by a dedicated integrated circuit, for example. Alternatively, a plurality of processors may cooperate to execute each process related to execution of information provision. Furthermore, each function may be provided by hardware and software different from those described above, or a combination thereof.

  The present invention is also applicable to an information provision control program installed on a server or the like outside the vehicle. The server acquires vehicle status information, in-vehicle conversation status information, driver status information, and the like from the vehicle side via the Internet. Then, the server can output sound for providing information from the speaker at an optimal timing based on the acquired state information.

  Furthermore, the present invention can also be applied to an information providing device mounted on a vehicle that performs automatic driving. In such an autonomous vehicle, a driver (operator) needs to maintain a standby state for backup of driving operation. At this time, the driver whose driving load has been reduced can easily concentrate on conversation with other passengers. Therefore, the information providing apparatus to which the present invention is applied is suitable as a configuration that reliably provides information while avoiding discomfort to the driver in a standby state as a backup of the automatic driving system.

  In addition, the configuration and the program to which the present invention is applied can be effective even in uses different from the provision of information to the driver. For example, the present invention can be applied to an application that can be executed by a processor of a communication terminal such as a smartphone. In this case, the communication terminal can detect the end of the user according to the application, and can provide the acquired various information to the user at an appropriate timing after the end of the conversation. Furthermore, the present invention can also be applied to devices having a function of having a conversation with a user, such as toys, communication robots, reception robots, and nursing robots.

30 speech output unit, 51 random determination unit, 52 load estimation unit, 60a processor, 61 speech recognition unit, 62 information acquisition unit, 70 conversation analysis unit, 80 provision control unit, 81 priority judgment unit, 100 information provision device, T Waiting time

Claims (15)

An information providing device for providing information to a user by sound output from the sound output device (30),
An information acquisition unit (62) for acquiring information provided to the user;
A conversation analysis unit (70) for determining whether or not the user is in a conversation and detecting the end of the conversation when the user is in a conversation;
A provision control unit (80) that waits for the waiting time (T) to elapse after the conversation analysis unit detects the end of the conversation, and then starts providing information by voice of the voice output device ;
The conversation analysis unit estimates the degree of excitement of the conversation that the user is conducting,
Wherein providing the control unit, as are raised conversation being conducted, the information providing apparatus to set longer the waiting time. A speech recognition unit (61) for recognizing speech uttered by each of the user and a talker who has a conversation with the user;
The information providing apparatus according to claim 1 , wherein the conversation analysis unit acquires contents of utterances of the user and the talker from the voice recognition unit. The conversation analysis unit calculates a duplication rate of words spoken by the user and the talker,
The information providing apparatus according to claim 2 , wherein the provision control unit sets the waiting time longer as the word duplication rate calculated by the conversation analysis unit increases. The conversation analysis unit determines whether an expression suggesting continuation of the conversation is included at the end of the utterance by the user and the talker,
Wherein providing the control unit, the conversation when suggesting representing the continuation of the conversation is determined to be the end of the sentence by the analyzer, the information providing apparatus according to claim 2 or 3 is set longer the waiting time. The conversation analysis unit obtains the amount of inflection in each utterance of the user and the talker from the speech recognition unit,
The information providing apparatus according to any one of claims 2 to 4 , wherein the provision control unit sets the waiting time longer as an inflection in the utterance of the user or the talker becomes larger. The conversation analysis unit calculates a participation rate in the conversation of each of the user and the talker,
Wherein providing the control unit is closer evenly participation rate to the conversation analysis unit of the user and the conversations who were calculated by conversation to any one of claims 2-5 to set the waiting time long The information providing apparatus according to the description. An information providing apparatus for providing information to a driver as the user who drives a vehicle,
The information acquisition unit acquires, from the random determination unit (51), random information that determines whether or not the driver is in a random state.
The information providing device according to any one of claims 1 to 6 , wherein the provision control unit sets the waiting time to be short based on the fact that the driver is determined to be in a disordered state by the random determination unit. . An information providing device that provides information to a driver as a user who drives a vehicle by means of sound output from the sound output device (30),
An information acquisition unit (62) for acquiring information provided to the user;
A conversation analysis unit (70) for determining whether or not the user is in a conversation and detecting the end of the conversation when the user is in a conversation;
A provision control unit (80) that waits for the waiting time (T) to elapse after the conversation analysis unit detects the end of the conversation, and then starts providing information by voice of the voice output device ;
The information acquisition unit acquires, from the random determination unit (51), random information that determines whether or not the driver is in a random state.
Wherein providing the control unit, the aimless based on said driver is determined to be in the absentminded state by the determination unit, the information providing apparatus to set the waiting time shortened. An information providing apparatus for providing information to a driver as the user who drives a vehicle,
The information acquisition unit acquires the load information obtained by estimating the height of the driving load of the driver from the load estimation unit (52),
The information providing apparatus according to any one of claims 1 to 8, wherein the provision control unit sets the standby time longer as the driving load estimated by the load estimation unit is higher. An information providing device that provides information to a driver as a user who drives a vehicle by means of sound output from the sound output device (30),
An information acquisition unit (62) for acquiring information provided to the user;
A conversation analysis unit (70) for determining whether or not the user is in a conversation and detecting the end of the conversation when the user is in a conversation;
A provision control unit (80) that waits for the waiting time (T) to elapse after the conversation analysis unit detects the end of the conversation, and then starts providing information by voice of the voice output device ;
The information acquisition unit acquires load information obtained by estimating the height of the driver's driving load from the load estimation unit (52),
Wherein providing the control unit, the load the higher the operation load estimated by the estimation unit, the information providing apparatus to set the waiting time longer. When the conversation analysis unit detects that the conversation is restarted before the waiting time elapses, the provision control unit holds the start of information provision by voice, and the conversation analysis unit detects the end of the conversation again. The information providing apparatus according to any one of claims 1 to 10 , which waits. A priority determination unit (81) for determining whether or not the information acquired by the information acquisition unit is priority provision information to be immediately notified to the user;
The information provision according to any one of claims 1 to 11 , wherein the provision control unit starts provision of the priority provision information even when the conversation analysis unit determines that the user is in a conversation. apparatus. An information provision control program for controlling provision of information to a user using voice output from the voice output device (30),
An information acquisition step (S101) for acquiring information provided to the user;
A conversation analysis step (S107) for determining whether or not the user is in a conversation and detecting the end of the conversation when the user is in a conversation;
After the end of the conversation is detected by the conversation analysis step, a provision control step (S10) for starting provision of information by voice from the voice output unit after waiting for the waiting time (T) to elapse.
8, S109)
Is executed by at least one processor (60a) ,
In the conversation analysis step, the degree of excitement of conversation conducted by the user is estimated,
Wherein provided in the control step, as has raised the conversation in the embodiment, the information provision control program to set the waiting time longer. An information provision control program for controlling provision of information to a user using voice output from the voice output device (30),
An information acquisition step (S101) for acquiring information provided to the user;
A conversation analysis step (S107) for determining whether or not the user is in a conversation and detecting the end of the conversation when the user is in a conversation;
After the end of the conversation is detected by the conversation analysis step, a provision control step (S10) for starting provision of information by voice from the voice output unit after waiting for the waiting time (T) to elapse.
8, S109)
Is executed by at least one processor (60a) ,
In the information acquisition step, the information as to whether or not the driver as the user driving the vehicle is in a state of disorder is acquired,
Wherein provided in the control step, based on the aimless information the driver is determined to be in the absentminded state, the information provision control program to set shorter the waiting time. An information provision control program for controlling provision of information to a user using voice output from the voice output device (30),
An information acquisition step (S101) for acquiring information provided to the user;
A conversation analysis step (S107) for determining whether or not the user is in a conversation and detecting the end of the conversation when the user is in a conversation;
After the end of the conversation is detected by the conversation analysis step, a provision control step (S10) for starting provision of information by voice from the voice output unit after waiting for the waiting time (T) to elapse.
8, S109)
Is executed by at least one processor (60a) ,
In the information acquisition step, load information obtained by estimating the height of the driving load of the driver as the user driving the vehicle is acquired,
Wherein provided in the control step, the higher the operation load indicated by the load information, the information provision control program to set the waiting time longer.

Images