JP6862797B2 - Conversation assist device - Google Patents

Description

The present invention relates to a technique for assisting conversation.

Patent Document 1 describes an in-vehicle conversation support system that collects the sound emitted by the speaker with a microphone, delays the sound for a certain period of time (5 to 20 ms), and outputs the sound from the speaker. According to this in-vehicle conversation support system, the receiver can easily hear the speaker's remarks as compared with the case where the sound (sound collected by the microphone) is not output from the speaker.

Japanese Unexamined Patent Publication No. 2008-42390

In the in-vehicle conversation support system described in Patent Document 1, the delay time added to the sound collected by the microphone is constant. Here, if the loud noise sound is long enough to overlap both the sound emitted by the speaker and the sound from the speaker in time, the receiver will hear the sound emitted by the speaker and the sound from the speaker. In either case, it may be difficult to hear. Therefore, if this constant delay time is lengthened and the volume of the noise sound is reduced according to the state of the vehicle, the receiver will hear the sound from the speaker much later than the sound emitted by the speaker. , There is a risk that the sound from the speaker may feel strange. In addition, since the speaker also hears his / her own sound from the speaker much later than the time when the voice is uttered, it is easy for the speaker to feel a sense of discomfort in the sound from the speaker.

The present invention has been made in view of the above circumstances, and has solved the problem of providing a technique that makes it easier for the speaker or the speaker to feel a sense of discomfort in the sound from the speaker while making it easier to hear the speaker's remarks in the passenger compartment. Make it an issue.

One aspect of the conversation assist device of the present invention depends on the state of the sound processing unit that delays the sound collected by the microphone in the vehicle interior and emits the sound by the speaker in the vehicle interior, and the state of the vehicle having the vehicle interior. , A control unit that controls the delay time, and the like.

It is a figure which showed the conversation assist device 100 which concerns on 1st Embodiment. It is a figure which showed an example of the supply part 4. It is a figure which showed an example of noise-related information. It is a flowchart for demonstrating operation of a conversation assist device 100. It is a figure which showed the conversation assist device 100A of the 2nd Embodiment. It is a figure which showed an example of speed relation information. It is a figure which showed other example of noise-related information. It is a figure which showed still another example of noise-related information. It is a figure which showed other example of speed relation information. It is a figure which showed still another example of speed relation information.

Hereinafter, embodiments according to the present invention will be described with reference to the drawings. In the drawings, the dimensions and scale of each part are appropriately different from the actual ones. Moreover, the embodiment described below is a preferable specific example of the present invention. For this reason, the present embodiment is provided with various technically preferable limitations. However, the scope of the present invention is not limited to these forms unless it is stated in the following description that the present invention is particularly limited.

<First Embodiment>
FIG. 1 is a diagram showing a conversation assist system 1 including a conversation assist device 100 according to the first embodiment of the present invention. The conversation assist system 1 is used in the vehicle C.
In the passenger compartment R of the vehicle C, in addition to the conversation assist device 100, four seats 51 to 54 arranged in a rectangle, a ceiling 6, a front right door 71, a front left door 72, and a rear right door 73 are provided. And the rear left door 74 is arranged. The seat 51 is a driver's seat. Seat 52 is a passenger seat. Seat 53 is the rear right seat. Seat 54 is the rear left seat. Each of the seats 51 to 54 is made of cloth or leather and has sound absorption. Seats 51-54 face in a common direction.

The conversation assist system 1 includes microphones 11 to 14 and 81, speakers 21 to 24, and a conversation assist device 100.
Each of the microphones 11 to 14 and 81 outputs an output signal according to the collected sound. The microphone 11 is arranged with respect to the seat 51. In the example shown in FIG. 1, the microphone 11 is arranged in the area 61 of the ceiling 6 facing the seat 51. The microphone 12 is arranged with respect to the seat 52. In the example shown in FIG. 1, the microphone 12 is arranged in the area 62 of the ceiling 6 facing the seat 52. The microphone 13 is arranged with respect to the seat 53. In the example shown in FIG. 1, the microphone 13 is arranged in the area 63 of the ceiling 6 facing the seat 53. The microphone 14 is arranged with respect to the seat 54. In the example shown in FIG. 1, the microphone 14 is arranged in the area 64 of the ceiling 6 facing the seat 54. The microphone 81 collects sound in the passenger compartment R and outputs an output signal M8 corresponding to the collected sound. The output signal M8 is used to identify the noise volume in the passenger compartment R. The noise includes engine noise, tire noise, wind noise, air conditioner noise in the vehicle interior, environmental noise from the outside of the vehicle, and the like when the vehicle is running.

The speaker 21 is arranged with respect to the seat 51. In the example shown in FIG. 1, the speaker 21 is arranged on the front light door 71 facing the seat 51. The speaker 22 is arranged with respect to the seat 52. In the example shown in FIG. 1, the speaker 22 is arranged on the front left door 72 facing the seat 52. The speaker 23 is arranged with respect to the seat 53. In the example shown in FIG. 1, the speaker 23 is arranged on the rear light door 73 facing the seat 53. The speaker 24 is arranged with respect to the seat 54. In the example shown in FIG. 1, the speaker 24 is arranged on the rear left door 74 facing the seat 54.

The conversation assist device 100 includes sound processing units 31 to 34, a supply unit 4, a storage unit 82, and a control unit 83.
The sound processing unit 31 adds a delay to the output signal M1 of the microphone 11 to generate the sound signal A1. In the process of adding a delay to the output signal M1, for example, a digital signal corresponding to the output signal M1 is stored in the internal memory of the sound processing unit 31, and when the delay time elapses from the storage time, the digital signal is read from the memory. It is a process to output. In this case, a memory having a capacity equal to or larger than the data amount of the digital signal corresponding to the output signal M1 may be used. The sound processing unit 32 adds a delay to the output signal M2 of the microphone 12 to generate the sound signal A2. The sound processing unit 33 adds a delay to the output signal M3 of the microphone 13 to generate the sound signal A3. The sound processing unit 34 adds a delay to the output signal M4 of the microphone 14 to generate the sound signal A4. The process of adding a delay to the output signal in the sound processing units 32 to 34 is a process similar to the process of adding a delay to the output signal by the sound processing unit 31. The delay time added to the output signals M1 to M4 is controlled according to the state of the vehicle C (for example, the noise volume in the vehicle interior R).

The supply unit 4 supplies the sound signal A1 to the speaker 24 arranged with respect to the seat (diagonal seat) 54 at a position diagonal to the seat 51. Further, the supply unit 4 supplies the sound signal A2 to the speaker 23, supplies the sound signal A3 to the speaker 22, and supplies the sound signal A4 to the speaker 21.
FIG. 2 is a diagram showing an example of the supply unit 4. The supply unit 4 shown in FIG. 2 includes a wiring 41 that electrically connects the sound processing unit 31 and the speaker 24, a wiring 42 that electrically connects the sound processing unit 32 and the speaker 23, and a sound processing unit 33. Includes a wiring 43 that electrically connects the speaker 22 and the speaker 22, and a wiring 44 that electrically connects the sound processing unit 34 and the speaker 21.

The storage unit 82 is, for example, a recording medium of any known type such as a semiconductor recording medium, a magnetic recording medium, or an optical recording medium, or a recording medium in which these recording media are combined. In the present specification, the “non-transient” recording medium is readable by all computers except a recording medium such as a transmission line that temporarily stores a transient propagation signal (transitory, propagating signal). It does not exclude volatile recording media, including various recording media. The storage unit 82 stores a program that defines the operation of the control unit 83 and noise-related information that indicates the correspondence between the noise volume and the delay time. The noise-related information may be a table in which the noise volume and the delay time are related to each other, or a mathematical formula in which the noise volume is an independent variable and the delay time is a dependent variable.
FIG. 3 is a diagram showing an example of the correspondence relationship between the noise volume and the delay time indicated by the noise-related information. In the noise-related information shown in FIG. 3, the larger the noise volume, the longer the delay time. An upper limit value Tref (for example, 50 ms) is set for the delay time. If the delay time becomes long and the sound from the speaker overlaps with the mora next to the speaker's live voice, the front and rear mora cannot be distinguished, making it difficult to hear. The upper limit value Tref is set to a time shorter than the delay time at which voice recognition becomes difficult in consideration of this point. The upper limit value Tref is not limited to 50 ms and can be changed as appropriate.

The control unit 83 is a computer such as a CPU (Central Processing Unit). The control unit 83 operates as follows, for example, by reading and executing the program stored in the storage unit 82.
The control unit 83 controls the delay time added to the output signals M1 to M4 according to the state of the vehicle C (for example, the noise volume in the vehicle interior R). For example, the control unit 83 identifies the noise volume in the vehicle interior R by using the output signal M8 of the microphone 81. The noise volume in the passenger compartment R is the volume of sounds other than conversation in the passenger compartment R (for example, the running sound and the environmental sound of the vehicle C). The control unit 83 determines the delay time corresponding to the noise volume in the vehicle interior R by using the noise-related information of the storage unit 82. The control unit 83 sets the delay time added by the sound processing units 31 to 34 to the delay time determined by using the noise-related information.

Next, the operation will be described. FIG. 4 is a flowchart for explaining the operation of the conversation assist device 100.
The control unit 83 first identifies the noise volume in the vehicle interior R (step S1). In step S1, the control unit 83 extracts a sound signal indicating noise in the vehicle interior R by subtracting the output signals M1 to M4 from the output signal M8, and specifies the noise volume based on the sound signal. For example, the control unit 83 receives the output signal M1 from the sound processing unit 31, the output signal M2 from the sound processing unit 32, the output signal M3 from the sound processing unit 33, and the output signal M4 from the sound processing unit 34. ..

Subsequently, the control unit 83 controls the delay time added to the output signals M1 to M4 according to the noise volume in the vehicle interior R (step S2). In step S2, the control unit 83 determines the delay time corresponding to the noise volume in the vehicle interior R by using the noise-related information of the storage unit 82. In the noise-related information of the storage unit 82, the larger the noise volume, the longer the delay time. Therefore, the control unit 83 increases the delay time as the noise volume in the vehicle interior R increases. Subsequently, the control unit 83 outputs the delay time information indicating the determined delay time to the sound processing units 31 to 34. The sound processing unit 31 adds the delay time indicated in the delay time information to the output signal M1 to generate the sound signal A1, and the sound processing unit 32 adds the delay time to the output signal M2 to generate the sound signal A2. The sound processing unit 33 adds the delay time to the output signal M3 to generate the sound signal A3, and the sound processing unit 34 adds the delay time to the output signal M4 to generate the sound signal A4. To do.

Subsequently, the supply unit 4 supplies the sound signal A1 to the speaker 24, supplies the sound signal A2 to the speaker 23, supplies the sound signal A3 to the speaker 22, and supplies the sound signal A4 to the speaker 21. The speaker 24 outputs a sound corresponding to the sound signal A1, the speaker 23 outputs a sound corresponding to the sound signal A2, the speaker 22 outputs a sound corresponding to the sound signal A3, and the speaker 21 outputs a sound corresponding to the sound signal A4. Output the sound (step S3).

According to this embodiment, the delay time of the output signal of the microphone is controlled according to the noise volume in the vehicle interior R. Therefore, the delay time is controlled according to the noise volume in the passenger compartment R so that the speaker can easily hear the speaker's speech and the speaker or the speaker does not feel uncomfortable with the sound from the speaker. Will be possible.
For example, if the delay time is lengthened as the noise volume in the passenger compartment R is increased, both the sound emitted by the speaker and the sound output by the speaker are less likely to overlap with the sudden loud noise sound in time. , Makes it easier for the receiver to hear the speaker's remarks. Further, the smaller the noise volume in the passenger compartment R, the shorter the time difference between the sound emitted by the speaker and the sound from the speaker, and the speaker or the speaker is less likely to feel discomfort with respect to the sound from the speaker.

<Second Embodiment>
It is considered that the volume of noise in the passenger compartment R has a strong correlation with the speed of the vehicle C. For example, as the speed of the vehicle C increases, the noise volume in the vehicle interior R tends to increase. Therefore, in the second embodiment of the present invention, the delay time for the output signal of the microphone is controlled according to the speed of the vehicle C.

FIG. 5 is a diagram showing the conversation assist device 100A of the second embodiment. In the conversation assist device 100A, the storage unit 82 stores speed-related information indicating the correspondence between the speed and the delay time instead of the noise-related information, and has the control unit 83a instead of the control unit 83, and the control unit 83a has the control unit 83a. It differs from the conversation assist device 100 of the first embodiment in that it receives speed information indicating the speed of the vehicle C from the vehicle control device 9. Further, the microphone 81 that was present in the vehicle interior R in the first embodiment is omitted in the second embodiment. Hereinafter, the conversation assist device 100A will be described with a focus on the above-mentioned differences.

The vehicle control device 9 controls the state of the vehicle C (for example, the speed of the vehicle C), and outputs speed information to the control unit 83a.
The storage unit 82 stores speed-related information as shown in FIG. The speed-related information may be a table in which the speed and the delay time are related to each other, or a mathematical formula in which the speed is an independent variable and the delay time is a dependent variable. In the speed-related information shown in FIG. 6, the higher the speed, the longer the delay time. The above-mentioned upper limit value Tref (for example, 50 ms) is set for the delay time.
The control unit 83a controls the delay time added to the output signals M1 to M4 according to the speed of the vehicle C. Specifically, the control unit 83a determines the delay time corresponding to the speed indicated by the speed information by using the speed-related information of the storage unit 82. According to the speed-related information of the storage unit 82, the faster the speed, the longer the delay time. Therefore, the control unit 83a increases the delay time as the speed of the vehicle C increases. Subsequently, the control unit 83a outputs the delay time information indicating the determined delay time to the sound processing units 31 to 34.

According to this embodiment, the delay time of the output signal of the microphone is controlled according to the speed of the vehicle C, which has a strong correlation with the noise volume in the vehicle interior R. Therefore, it is possible to control the delay time according to the speed of the vehicle C so that the speaker can easily hear the speaker's speech and the speaker or the speaker does not feel uncomfortable with the sound from the speaker. become.
For example, the faster the speed of vehicle C (that is, the louder the noise volume in the passenger compartment R), the longer the delay time, the suddenly large both the sound emitted by the speaker and the sound output by the speaker. It becomes difficult for the speaker to overlap with the noise sound of the volume in terms of time, and it becomes easier for the receiver to hear the speaker's remarks. Further, the slower the speed of the vehicle C (that is, the smaller the noise volume in the passenger compartment R), the shorter the time difference between the sound emitted by the speaker and the sound from the speaker, and the sound from the speaker becomes shorter. It becomes difficult for the speaker to feel a sense of discomfort.

<Modification example>
Each of the above embodiments can be modified in various ways. A specific mode of modification is illustrated below. Two or more embodiments arbitrarily selected from the following examples can be appropriately merged as long as they do not conflict with each other.

<Modification example 1>
An amplification unit that amplifies the sound signal A1, an amplification unit that amplifies the sound signal A2, an amplification unit that amplifies the sound signal A3, and an amplification unit that amplifies the sound signal A4 may be added. Without adding these amplification units, the sound processing unit 31 amplifies the sound signal A1, the sound processing unit 32 amplifies the sound signal A2, and the sound processing unit 33 amplifies the sound signal A3 for sound processing. The unit 34 may amplify the sound signal A4.

<Modification 2>
Noise-related information can be changed as appropriate. For example, noise-related information indicating the relationship between the noise volume and the delay time as shown in FIG. 7 or FIG. 8 may be used. In the noise-related information shown in FIG. 7, when the noise volume is included in the predetermined range Nr, the larger the noise volume, the longer the delay time. In the noise-related information shown in FIG. 8, the delay time increases stepwise as the noise volume increases.
In addition, the speed-related information can be changed as appropriate. For example, speed-related information indicating the relationship between the speed and the delay time as shown in FIG. 9 or FIG. 10 may be used. In the speed-related information shown in FIG. 9, when the speed is included in the predetermined range Vr, the higher the speed, the longer the delay time. In the speed-related information shown in FIG. 10, the higher the speed, the longer the delay time becomes stepwise.
Here, the predetermined ranges Nr and Vr may be set for each vehicle C. In this case, noise-related information and speed-related information can be set according to the sound transmission characteristics of the vehicle C.
Further, as shown in FIGS. 8 and 10, when the delay time changes stepwise according to the noise volume or speed, the delay time is delayed as compared with the case where the delay time changes linearly according to the noise volume or speed. The frequency of the process of changing the time can be reduced, and the process can be simplified.

<Modification example 3>
The supply unit 4 may appropriately change the supply destinations (speakers) of the sound signals A1 to A4. For example, the supply unit 4 may supply a plurality of sound signals to one speaker.

<Modification example 4>
One or more seats may be arranged between the seat 51 and the seat 53. Further, one or more seats may be arranged between the seat 52 and the seat 54. Further, the seat 53 and the seat 54 may be integrally formed. Further, one or more seats may be arranged between the seat 53 and the seat 54. In this case, the seat between the seat 53 and the seat 54, the seat 53, and the seat 54 may be integrally formed.

<Modification 5>
At least one of the control units 83 and 83a receives the output signal M1 from the microphone 11, receives the output signal M2 from the microphone 12, receives the output signal M3 from the microphone 13, and receives the output signal M3 from the microphone 14, without going through the sound processing units 31 to 34. The output signal M4 may be received from.

<Modification 6>
At least one of the control units 83 and 83a may change the delay time for each speaker.

<Modification 7>
An operation unit (for example, an operation switch) for setting the on / off of the control of the delay time is provided, and when the user operates the operation unit to set the control of the delay time to on, the control units 83 and 83a May perform delay time control.

<Modification 8>
In the first embodiment, the noise volume in the passenger compartment R is specified by subtracting the output signals M1 to M4 from the output signal M8, but instead, the output signal M8 is passed through a filter to have a frequency corresponding to the noise. Only the signal may be taken out, and the noise volume in the passenger compartment R may be specified based on the signal of the frequency.

<Modification example 9>
It is considered that the volume of noise in the passenger compartment R has a strong correlation with the rotation speed of the engine of the vehicle C and the rotation speed of the fan of the air conditioner of the vehicle C. For example, as the rotation speed of the engine of the vehicle C or the rotation speed of the fan of the air conditioner of the vehicle C increases, the noise volume in the vehicle interior R tends to increase. Therefore, in the second embodiment, as the state of the vehicle C, the rotation speed of the engine of the vehicle C or the rotation speed of the fan of the air conditioner of the vehicle C may be used instead of the speed of the vehicle C. In this case, the control unit 83a receives the information on the engine speed of the vehicle C or the information on the rotation speed of the fan of the air conditioner of the vehicle C from the vehicle control device 9, and the larger the engine speed of the vehicle C, or The higher the rotation speed of the fan of the air conditioner of the vehicle C, the longer the delay time. At this time, the control unit 83a has information related to the engine speed indicating that the higher the engine speed of the vehicle C is, the longer the delay time is, and the higher the speed of the fan of the air conditioner of the vehicle C is, the longer the delay time is. The delay time may be determined by using the fan speed relation information indicating that.

<Aspects grasped from at least one of each embodiment and each modification>
The following aspects are grasped from at least one of each of the above-described embodiments and modifications.
One aspect of the conversation assist device of the present invention depends on the state of the sound processing unit that delays the sound collected by the microphone in the vehicle interior and emits the sound by the speaker in the vehicle interior, and the state of the vehicle having the vehicle interior. , A control unit that controls the delay time, and the like.
According to this aspect, it is possible to control the delay time according to the state of the vehicle so that the receiver can easily hear the speaker's speech. Therefore, the receiver can easily hear the speaker's remarks as compared with the case where the delay time is constant regardless of the state of the vehicle.
For example, if the delay time is lengthened when the noise volume increases according to the state of the vehicle, both the sound emitted by the speaker and the sound from the speaker are less likely to overlap with the loud noise sound in time. It makes it easier for the receiver to hear the speaker's remarks. In addition, if the delay time is shortened when the noise volume decreases according to the state of the vehicle, the time difference between the sound emitted by the speaker and the sound from the speaker becomes shorter, and the speaker or speaker regarding the sound from the speaker. However, it becomes difficult to feel a sense of discomfort.

In one aspect of the conversation assist device described above, it is desirable that the state of the vehicle is a noise volume in the vehicle interior. According to this aspect, it is possible to control the delay time according to the noise volume in the vehicle interior so that the receiver can easily hear the speaker's speech.

In one aspect of the conversation assist device described above, it is desirable that the control unit prolongs the delay time as the noise volume increases. According to this aspect, both the sound emitted by the speaker and the sound output by the speaker are less likely to overlap with the sudden loud noise sound in time, and the speaker can easily hear the speaker's speech. Further, when the noise volume in the passenger compartment is low, the time difference between the sound emitted by the speaker and the sound from the speaker becomes short, and the speaker or the speaker does not feel uncomfortable with the sound from the speaker.

In one aspect of the conversation assist device described above, it is desirable that the state of the vehicle is the speed of the vehicle. It is known that the speed of the vehicle C has a strong correlation with the noise volume in the vehicle interior R. According to this aspect, it is possible to control the delay time according to the noise volume in the vehicle interior, that is, according to the noise volume in the vehicle interior so that the receiver can easily hear the speaker's speech.

In one aspect of the conversation assist device described above, it is desirable that the control unit increase the delay time as the speed of the vehicle increases. The faster the vehicle speed, the louder the noise volume in the passenger compartment tends to be. According to this aspect, as the speed of the vehicle increases, the delay time increases, so that both the sound emitted by the speaker and the sound output by the speaker are unlikely to overlap with the sudden loud noise sound in time. This makes it easier for the receiver to hear the speaker's remarks. Further, when the speed of the vehicle C is slow, the time difference between the sound emitted by the speaker and the sound from the speaker becomes short, and the speaker or the speaker does not feel uncomfortable with the sound from the speaker.

In one aspect of the conversation assist device described above, the microphone is arranged for any of four seats arranged in a rectangular shape, there are a plurality of the speakers, and the microphone is arranged among the four seats. It is arranged for a plurality of seats including a diagonal seat diagonally to the seat, and supplies the sound collected by the microphone to a speaker arranged for the diagonal seat. It is desirable to include the supply section further. According to this aspect, the sound collected by the microphone is emitted from the speaker of the seat having the longest distance from the seat where the microphone is arranged. Therefore, it is possible to reduce the occurrence of howling during conversation. Further, according to this aspect, the seat is located between the microphone and the speaker. Therefore, among the sounds output by the speaker, the sound (sound wave) that reaches the seat is easily absorbed by the seat. Therefore, it is possible to reduce the occurrence of howling.

100 ... Conversation assist device, 11-14 and 81 ... Microphone, 21-24 ... Speaker, 31-34 ... Sound processing unit, 82 ... Storage unit, 83 ... Control unit.

Claims (3)

A sound processing unit that delays the sound collected by the microphone in the vehicle interior and emits it from the speaker in the vehicle interior.
A control unit that controls the delay time according to the state of the vehicle having the passenger compartment is included.
The control unit prolongs the delay time when the noise volume in the vehicle interior increases.
Kai story assist device characterized in that. A sound processing unit that delays the sound collected by the microphone in the vehicle interior and emits it from the speaker in the vehicle interior.
A control unit that controls the delay time according to the state of the vehicle having the passenger compartment is included.
The control unit increases the delay time as the speed of the vehicle increases .
Kai story assist device characterized in that. The microphone is arranged for any of the four seats arranged in a rectangle.
There are a plurality of the speakers, and the speakers are arranged for a plurality of seats including the diagonal seats which are diagonal to the seat where the microphone is arranged among the four seats.
The conversation assist device according to claim 1 or 2 , further comprising a supply unit that supplies the sound collected by the microphone to a speaker arranged with respect to the seat at the diagonal position.

Images