DE112019007580T5 - Sound field control device and sound field control method - Google Patents

Abstract

It is intended to provide a technology by which a sound field in a vehicle can be appropriately controlled with respect to the voice of any occupant in the vehicle. The sound field control device includes an acquisition section that acquires face information of the plurality of occupants based on a captured image of a plurality of occupants in a vehicle, a determination control section that, based on the face information acquired by the acquisition section and a sound emitted by a microphone in received inside the vehicle, speakers and listeners are determined from among the plurality of occupants, and a sound field control section that controls a sound field at the position of the plurality of occupants based on the result of determination by the determination control section.

Description

[Technical Field]

The present invention relates to a sound field control device and a sound field control method for controlling a sound field generated by a speaker in a vehicle at a specific location inside the vehicle.

[background technology]

Systems for providing a comfortable conversational environment in a vehicle have been proposed. For example, in a hands-free system including a microphone (microphone) and a speaker, a comfortable conversation environment is realized using noise canceling technology for eliminating noise in a vehicle and echo canceling technology for eliminating an echo in a vehicle. Furthermore, a technology is proposed in which a microphone and a speaker are arranged near each occupant in order to perform smooth communication of the occupants in a vehicle, and the noise of an occupant is picked up by the microphone and is picked up from the speaker near one issued to other inmates.

As a technology for controlling a sound field in a vehicle using a microphone and a speaker, e.g. For example, the technology proposed in Patent Document 1 and Patent Document 2. In Patent Document 1, a technology for controlling a sound field according to the rotation of a head is proposed. In Patent Document 2, a technology for controlling a sound field according to the position of a seat is proposed.

[Prior Art Documents]

[patent documents]

• [Patent Document 1] Patent Publication No. JP 2009-253526 A
• [Patent Document 2] Patent Publication No. JPH 07-184298 A

[Outline of the Invention]

[Problem to be solved by the invention]

The above prior art technologies only assume adequate control of a sound field with respect to the voice of a specific occupant, namely the driver. Therefore, there is a problem that an appropriate control of a sound field in a vehicle is not possible with respect to the voice of any occupant in the vehicle.

The present invention has been made with this problem in mind, and has as an object to provide a technology by which a sound field in a vehicle can be appropriately controlled with respect to the voice of any occupant in the vehicle.

[means for solving the task]

The sound field control device according to the present invention includes an acquisition section that acquires face information of the plurality of occupants based on a captured image of a plurality of occupants in a vehicle, a determination control section that, based on the face information acquired by the acquisition section and a sound, the received by a microphone inside the vehicle, determines speakers and listeners from among the plurality of occupants, and a sound field control section that controls a sound field at the position of the plurality of occupants based on the result of determination by the determination control section .

[Effects of the invention]

According to the present invention, speakers and listeners are determined from among the plurality of occupants based on acquired face information and a sound received by a microphone, and a sound field at the position of the plurality of occupants is controlled based on the result of the speaker and listener determination. With such a configuration, a sound field in a vehicle can be appropriately controlled with respect to the voice of any occupant in the vehicle.

The object, characteristics, aspects and advantages of the present invention will become clearer from the following detailed explanation and attached figures.

character list

• [ 1 ] 1 14 is a block diagram showing the structure of a sound field control device according to a first embodiment.
• [ 2 ] 2 14 is a block diagram showing the structure of a sound field control device according to a second embodiment.
• [ 3 ] 3 14 is a flowchart showing the operation of the sound field control device according to the second embodiment.
• [ 4 ] 4 12 is a view for explaining the operation of the sound field control device according to the second embodiment.
• [ 5 ] 5 14 is a view for explaining the operation of the sound field control device according to a third embodiment.
• [ 6 ] 6 14 is a block diagram showing the hardware configuration of a sound field control device according to another modification.
• [ 7 ] 7 14 is a block diagram showing the hardware configuration of a sound field control device according to another modification.
• [ 8th ] 8th 14 is a block diagram showing the structure of a server according to another modification.
• [ 9 ] 9 14 is a block diagram showing the construction of a communication terminal device according to another modification.

[Forms for carrying out the invention]

<First Embodiment>

1 14 is a block diagram showing the structure of a sound field control device 1 according to a first embodiment of the present invention. A DMS (Driver Monitoring System), for example, can be used as the sound field control device 1 . The sound field control device 1 in 1 is connected to a microphone (microphone) 51 and a speaker 52 in a vehicle wirelessly or by wire.

The microphone 51 receives sounds from the mouths of a variety of occupants in the vehicle. As the microphone 51, for example, a super directional microphone, etc., capable of receiving only the sounds of the respective occupants is used. The plurality of occupants includes, for example, the driver, a passenger in the front passenger seat, and a passenger in the back seat.

The speaker 52 generates a sound field in the vehicle under the control of the sound field control device 1 . As such a speaker 52, for example, a super directional speaker such as a parametric array speaker capable of generating different sound fields at the positions of the respective occupants is used.

Next, the sound field control device 1 will be explained. As explained below, the sound field control device 1 can control a sound field generated by the speaker 52 at a specific location inside the vehicle.

The sound field control device 1 in 1 comprises an acquisition section 11, a determination control section 12 and a sound field control section 13.

The acquisition section 11 acquires facial information of the plurality of passengers based on a captured image of a plurality of passengers in a vehicle. The face information is information including feature points of an occupant's face and state, for example, information on the position of the occupant's mouth, the position of the occupant's ears (e.g. both ears) and the movement of the mouth. For the acquisition component 11, at least one of a device that enables recognition processing of an image of the occupants captured by a capturing device such as a camera, etc. and an interface is used.

The determination control section 12 determines speakers and listeners from among the plurality of passengers based on the face information acquired by the acquisition section 11 and the sound received through the microphone 51 . For example, the determination control section 12 determines that an occupant is a speaker when the opening and closing of the mouth of the occupant is detected from the face information of a time series and also it is detected that the waveform of the through the sound received by the micro 51 is a vowel waveform or a consonant waveform, and designates the other occupants as listeners. Further, the determination control section 12 determines that an occupant is a speaker when the face information of a time series is assumed to be the occupant's speech by a lip reading function, and also at the microphone 51, a sound pressure equal to or higher than a certain level was established, and it designates the other occupants as listeners.

By controlling the speaker 52 based on the result of determination by the determination control section 12, the sound field control section 13 can provide not only a sound field at a specific position but also sound fields in a vehicle control the position of a variety of occupants. For example, the sound field control section 13 performs control by selectively changing the voice of the speaker based on the position of the speaker's mouth determined by the determination control section 12 and the position of the listener's ear(s) determined by the determination control section 12 speaker and a voice drowning out the speaker's voice.

<Summary of First Embodiment>

According to the sound field control device 1 according to the above present first embodiment, speakers and listeners are determined from among a plurality of occupants based on acquired face information and sounds received by the microphone 51, and a sound field is applied based on the result of the speaker and listener determination the position of the plurality of occupants. With such a configuration, any occupant whose voice was raised inside the vehicle can be designated as a speaker and the other occupants as listeners, so that sound fields at the position of a plurality of occupants can be appropriately controlled with respect to the voice of any occupant. This is particularly effective with ridesharing, which has been offered in recent years.

<Second embodiment>

2 14 is a block diagram showing the structure of a sound field control device 1 according to a second embodiment of the present invention. In the following, among the components according to the second embodiment, components that are the same as or similar to the components described above will be given the same or similar reference numerals, and components that are different therefrom will be mainly explained.

In the case of the sound field control device 1 in 2 a directional microphone 51a, a loudspeaker group 52a and an image recognition device 53 are communicably connected to each other. The concept of the microphone 51 and the speaker 52 in 1 contains the directional microphone 51a and the loudspeaker group 52a. More specifically, like the mic 51 explained in the first embodiment, the directional mic 51a receives sound from the mouths of a plurality of occupants in a vehicle. The speaker group 52a generates a sound field in the vehicle under the control of the sound field control device 1 like the speaker 52 explained in the first embodiment. The image recognition device 53 performs recognition processing on a captured image of a plurality of occupants in a vehicle, and thereby generates facial information of the plurality of occupants.

Next, the sound field control device 1 will be explained. The sound field control device 1 in 2 comprises a face information acquisition section 11a, a speaking voice determination section 12a, a speaker/listener recognition section 12b, a noise muting section 13a, a noise drive section 13b, a drive control section 13c and a micro control section 14. The face information acquisition section 11a is in the concept of FIG acquisition section 11 in 1 included, and the speaking voice determination section 12a and speaker/listener recognition section 12b are included in the concept of the determination control section 12 in 1 contain. The noise muting section 13a, the noise driving section 13b and the driving control section 13c are in the concept of the sound field control section 13 in FIG 1 contain.

The face information acquiring section 11a acquires face information generated by the image recognition device 53 . The face information is information including feature points of an occupant's face and the state, like the first embodiment.

The micro control section 14 controls the direction of the directional mic 51a based on the face information acquired by the face information acquiring section 11a so that the directional mic 51a can precisely receive the occupant's voice at the position of the mouth. The adjustment logic of the direction of the directional microphone 51a is not limited, and any existing adjustment logic can be used.

The speaking voice determination section 12a determines based on the face information acquired by the face information acquiring section 11a and the noise received through the directional mic 51a whether the noise received through the directional mic 51a is a voice raised by an occupant. For such a determination, for example, the determination explained in the determination control section 12 in the first embodiment is used.

The speaker/listener recognition section 12b specifies speakers and listeners based on the face information acquired by the face information acquiring section 11a and the result of determining the speaking voice dess voice section 12a when it is determined by the speaking voice determination section 12a that the noise received by the directional microphone 51a is an occupant's voice. For example, the speaker/listener recognition section 12b determines one occupant, of which the speaking voice determination section 12a has determined a raising voice, from among a plurality of occupants whose face information has been acquired by the face information acquiring section 11a, as a speaker, and designates the other occupants as listeners. Here, the speaker/listener recognition section 12b also determines the position of the speaker's mouth and the position of the ear(s) of the listener(s). The result of determination by the speaker/listener detection section 12b is output to the noise muting section 13a and the noise driving section 13b via the drive control section 13c.

The noise muting section 13a drives based on the result of determination by the speaker/listener detection section 12b to output a voice drowning out the speaker's voice from the speaker group 52a to the listeners. Concretely, the noise muting section 13a controls the speaker group 52a based on the position of the speaker's mouth and the position of the listener's ear(s) determined by the speaker/listener detecting section 12b and the noise received by the directional microphone 51a of the speaker such that a voice drowning out the speaker's voice is output to the position of the ear(s) of the listener(s). The drowning of the speaker's voice may be drowning to an extent that renders the speaker's voice inaudible, but may not be complete drowning.

The noise driving section 13b drives to output the speaker's voice from the speaker group 52a to the listener(s) based on the result of the determination by the speaker/listener detecting section 12b. Concretely, the noise driving section 13b controls the speaker group 52a based on the position of the speaker's mouth and the position of the listener's ear(s) determined by the speaker/listener detecting section 12b and that of the directional microphone 51a received sound of the speaker such that the position of the ear(s) of the listener(s) is outputted with the voice of the speaker.

The drive control section 13c performs drive control that controls the drive of the noise mute section 13a and the drive of the noise drive section 13b. In the present second embodiment, the drive control section 13c performs drive control based on whether or not the speaker makes a hands-free call. The speaker may be the driver or an occupant other than the driver.

For example, when it is determined that the speaker is making a hands-free call, the drive control section 13c performs control for not driving the noise driving section 13b and driving the noise muting section 13a for all handsets. Further, when it is determined, for example, that the speaker is not making a hands-free call, the drive control section 13c performs control for not operating the noise muting section 13a and operating the noise drive section 13b for all listeners.

Whether the speaker is in hands-free calling can be determined, for example, based on hands-free calling status information inputted from a hands-free calling controller (not shown) or based on the face information acquired by the face information acquiring section 11a.

<How it works>

3 14 is a flow chart showing the operation of the sound field control device 1 according to the present second embodiment.

First, in step ST1, the face information acquiring section 11a acquires face information from the image recognition device 53.

In step ST2, the micro control section 14 controls the direction of the directional micro 51a based on the face information acquired by the face information acquiring section 11a.

In step ST3, the speaking voice determination section 12a determines whether the noise received by the directional mic 51a is a voice raised by an occupant based on the face information acquired by the face information acquiring section 11a and the sound received by the directional mic 51a. That is, the speaking voice determination section 12a determines whether an occupant is speaking. When it is determined that an occupant is speaking, processing proceeds to step ST4, and when it is determined that no occupant is speaking, processing returns to step ST1.

In step ST4, the speaker/listener recognition section 12b determines speakers and listeners based on the face information acquired by the face information acquiring section 11a and the determination result of the speaking voice determination section 12a.

In step ST5, the drive control section 13c determines whether or not the speaker is making a hands-free call. When it is determined that the speaker is making a hands-free call, the processing proceeds to step ST6, and when it is determined that the speaker is not making a hands-free call, the processing proceeds to step ST7.

In step ST6, the driving control section 13c performs control driving the noise muting section 13a. Thereby, based on the result of determination by the speaker/listener detection section 12b, the noise muting section 13a drives to output a voice drowning out the speaker's voice from the speaker group 52a to all listeners. Thereafter, the processing returns to step ST1.

In step ST7, the drive control section 13c performs control operating the sound drive section 13b. Thereby, based on the result of the determination by the speaker/listener detection section 12b, the sound driving section 13b drives to output the speaker's voice from the speaker group 52a to all the listeners. Thereafter, the processing returns to step ST1.

<Summary of Second Embodiment>

According to the sound field control device 1 according to the present second embodiment above, whether the speaker is the driver or an occupant other than the driver, the noise muting section 13a is driven based on whether the speaker is making hands-free calls and drive controller controlling the drive of the sound drive section 13b. According to this structure, if any occupant makes a hands-free call, the conversation can be confidential to the other occupants.

<Modification 1>

The drive control section 13c can also perform drive control controlling the drive of the noise mute section 13a and the drive of the noise drive section 13b based on an action other than hands-free calling. This example is explained below.

<Modification 1A>

The drive control section 13c can also perform drive control based on operation by the speaker and the listener. For example, the drive control section 13c may also control not to operate the noise muting section 13a and operate the noise drive section 13b when an unillustrated icon or button is operated by both the speaker and the listener, indicating the intention of agreeing to talk. Further, the drive control section 13c can, for example, also perform control for not operating the noise drive section 13b and for operating the noise muting section 13a when both the speaker and the listener have operated an unillustrated icon or button indicating the intention of rejecting a call .

The drive control can also be performed following a preset rule such as a talk refusal priority rule or a speaker priority rule when the speaker's intention and the listener's intention are different. Here, the conversation refusal priority rule is a rule according to which the noise driving section 13b is not operated and the noise muting section 13a is prioritized operated when an operation indicating the intention of conversation refusal is performed at least by either the speaker or the listener. The speaker priority rule is a rule for driving control of the noise muting section 13a and the noise driving section 13b with prioritizing the speaker's operation over the listener's operation.

From then on, by storing the setting due to the operation from the speaker and the listener, the drive control section 13c can also perform the drive control based on this setting. The operation of the speaker and the listener is not limited to the operation of an icon or button, not shown, so that it is also a z. B. by means of a camera, etc. detected gesture operation of the speaker or listener. The gesture operation may include, for example, as a gesture operation indicating agreement to talk, a gesture operation in which an OK sign is formed with fingers, and a gesture operation such as a gesture operation indicative of rejecting talk involve pushing away with your hands. The gesture operation can also be appropriately changed depending on the area (eg, region, country) where the sound field control device 1 is positioned.

<Modification 1B>

The drive control section 13c can also determine the speaker's facial expression and the listener's facial expression based on the face information acquired by the face information acquiring section 11a, and perform the drive control based on the result of this determination. For example, the driving control section 13c can also determine in the case that a wink is determined in the face of either the speaker or the listener, indicating the intention of agreeing to talk, and a controller for not operating the noise muting section 13a and operating the noise driving section 13b carry out. Further, for example, the driving control section 13c can also determine, for example, in the case that a distorted facial expression is determined in either the speaker or the listener, indicating the intention of refusing to speak, and performing control of not operating the noise driving section 13b and operating the noise muting section 13a .

Further, the drive control section 13c can also determine the speaker's face direction and the listener's face direction based on the face information acquired by the face information acquisition section 11a, and perform drive control based on the result of this determination. For example, the driving control section 13c can also determine in the case that it has been determined that the face of either the speaker or the listener faces the other, indicating the intention of agreeing to talk, and a control for not operating the noise muting section 13a and operating the noise - Carry out drive section 13b. Further, the driving control section 13c can also determine, for example, in the case where it has been determined that the face of either the speaker or the listener faces away from the other, indicating the intention of rejecting the conversation, and control for not operating the noise driving section 13b and for perform operation of the noise muting section 13a.

Further, the drive control section 13c can also determine the speaker's face movement and the listener's face movement based on the face information acquired by the face information acquiring section 11a, and perform the drive control based on the result of this determination. For example, the drive control section 13c can also, in the case where it has been determined that the face of either the speaker or the listener has approached the other or has been moved up and down, determine the indication of the intention of agreeing to talk, and control for Conduct non-operating the noise muting section 13a and operating the noise driving section 13b. Further, the driving control section 13c can also determine, for example, in the case where it has been determined that the face of either the speaker or the listener has moved away from the other or has been shaken sideways, indicating the intention of rejecting the conversation, and controlling not to operate the perform the noise driving section 13b and for operating the noise muting section 13a.

Also, in the present Modification 1B, as in Modification 1A, the drive control can be performed following a preset rule such as a call refusal priority rule or a speaker priority rule when the speaker's intention and the listener's intention are different.

<Modification 1C>

The drive control section 13c can also perform the drive control based on the contents of the speaker's and the listener's speech. For example, the drive control section 13c can also determine, in the case where it has been determined that the speaker has called a listener's name or the listener has responded to the calling of the name, to indicate the intention of agreeing to talk, and control not to operate the noise muting section 13a and to operate the sound driving section 13b. Further, the drive control section 13c can also, for example, in the case where it has been determined that the speaker has not called a listener's name or the listener has not responded to the calling of the name, determines displaying the intention of rejecting a call, and a controller for Conduct non-operating the noise driving section 13b and to operate the noise muting section 13a.

In the present modification 1C, the drive control can also be performed following a preset rule such as a call refusal priority rule or a talker priority rule, just as in the modification 1A when the intention of the speaker and the intention of the hearer are different.

<Modification 2>

In the second embodiment, the drive control section 13c uniformly performs drive control for all handsets when it is determined that the speaker is making hands-free calls, but is not limited to this. For example, the drive control section 13c can also perform drive control per handset. That is, the drive control section 13c can also perform control driving the noise driving section 13b for one listener, and simultaneously perform control driving the noise muting section 13a for another listener. According to such a structure, contradictory drives such as the drive of the noise muting section 13a and the drive of the noise drive section 13b can be performed at the same time, so that a comfortable conversation environment in a vehicle can be realized.

<Modification 3>

For example, the sound field control device 1 can also perform music volume reduction control in the case that while music is being output inside the vehicle, the noise driving section 13b is driven and the speaker's voice is output from the speaker group 52a to a listener carry out. Further, also in the case where the speaker's voice is output by driving the noise driving section 13b, the drive control section 13c can perform noise suppression (e.g. suppression of engine noise of the vehicle or noise caused by driving) by driving the noise muting section 13a. carry out.

<Modification 4>

The noise driving section 13b can also change the frequency characteristic of the voice output from the speaker group 52a to the listener concerned based on characteristics such as the listener's hearing ability or preferences. Also, for example, when the listener is a person who finds it difficult to hear low-frequency sounds, the noise driving section 13b can increase the low-frequency content of the voice output from the speaker array 52a to the listener.

The characteristics of the listener can be acquired from the smartphone or a hearing aid of the listener, but they can also be stored in the sound field control device 1 in advance by an operation of the user.

<Third embodiment>

4 12 is a view for explaining the operation of the sound field control device 1 according to the second embodiment described above, and FIG 5 12 is a view for explaining the operation of the sound field control device 1 according to a third embodiment. The block structure of the sound field control device 1 according to the present third embodiment is the same as the block structure of the sound field control device 1 according to the second embodiment ( 2 ). Therefore, in the following, among the constituent elements according to the third embodiment, the same or similar constituent elements as the constituent elements described above are given the same or similar reference numerals, and constituent elements different therefrom are mainly explained.

ausgedrückt werden.First, by means of 4 the operation of the sound field control device 1 according to the second embodiment will be explained. A noise Vs received by the directional mic 51a has a superimposed waveform of a voice S1 from a speaker 21 that has reached the directional mic 51a and vehicle-outside noise N1 that has reached the directional mic 51a from outside the vehicle. Therefore, the noise Vs can be used as $$\text{vs}=\text{S1}+\text{N1}$$

be expressed.

ausgedrückt werden. When the noise muting section 13a is not driven, a voice h1 heard by a listener 22 has a superimposed waveform of a voice S2 from the speaker 21 that has reached the ear position of the listener 22 and vehicle-outside noise N2 coming from outside the vehicle Position of the ear of the listener 22 have reached. Therefore the voice h1 can be used as $$\text{h1}=\text{S2}+\text{N2}$$

be expressed.

ausgedrückt werden.The sound field control device 1 generates a noise Va based on the voice Vs of the equation (1), which is output from the speaker group 52a at the position of the listener's 22 ear. The sound Va can as $$\text{Va}=\text{f}\left(\text{vs}\right)=\text{f}\left(\text{S1}+\text{N1}\right)\fallingdotseq \text{f}1\left(\text{S1}\right)+\text{f}2\left(\text{N1}\right)$$

be expressed.

$$\text{f}2\left(\text{N1}\right)\fallingdotseq -\text{N1}$$

ergibt.The voice Va drowns out the voice S2 and the vehicle-external noise N1, so that the relationship $$\text{f}1\left(\text{S1}\right)\fallingdotseq -\text{S2}$$

$$\text{f}2\left(\text{N1}\right)\fallingdotseq -\text{N1}$$

results.

Bei der letzten Zeile des Ausdrucks (6) wurden die Ausdrücke (4) und (5) angewendet.Therefore, when the noise muting section 13a is driven, the voice h2 heard by the listener 22 becomes $$\begin{array}{l}\text{h2}=\text{S2}+\text{N2}+\text{Va}\\ =\text{S2}+\text{N2}+\text{f}1\left(\text{S1}\right)+\text{f}2\left(\text{N1}\right)\\ =\left\{\text{S2}+\text{f}1\left(\text{S1}\right)\right\}+\left\{\text{N2}+\text{f}2\left(\text{N1}\right)\right\}\\ \fallingdotseq 0+\left(\text{N2}-\text{N1}\right)\end{array}$$

In the last line of expression (6), expressions (4) and (5) have been applied.

wird.Assuming that the noise outside the vehicle is substantially the same regardless of the position of the occupants, N2 ≒ N1, so the voice h2 of expression (6) holds $$\text{H}2\fallingdotseq 0$$

will.

Therefore, in addition to the voice S2 of the speaker 21, the noise N2 outside the vehicle is also eliminated for the listener 22. However, when the listener 22 is the driver, for example, it is desirable for the driver (listener 22) to hear vehicle-external sounds associated with driving, such as another vehicle's horn, collision sounds, sudden braking sounds, an ambulance siren, etc.

In the present third embodiment, therefore, the noise muting section 13a performs a drive in which, without drowning out noise from outside the vehicle, a voice drowning out a speaker's voice is output from the speaker group 52a to the listener. This allows the listener 22 to hear sounds external to the vehicle under conditions or no conditions.

In the following, using 5 the operation of the sound field control device 1 according to the third embodiment will be explained. In the present third specific embodiment, a microphone 56 is provided in a vehicle for actively receiving noises external to the vehicle. The mic 56 is, for example, an omnidirectional mic that is placed at a location where it can more easily receive vehicle-external noises than the directional mic 51a.

ausgedrückt werden.A noise Vr received by the mic 56 has a superimposed waveform of a voice S3 from the speaker 21 that has reached the mic 56 and in-vehicle noise N3 that has reached the mic 56 from outside the vehicle. Therefore, the voice Vr as $$\text{Vr=S3+N3}$$

be expressed.

ausgedrückt werden.The sound field control device 1 according to the present third embodiment treats the voice Vr and the voice Vs as input sounds and generates a signal representing a voice Vmix from their difference. The voice Vmix can be used as $$\begin{array}{l}\text{Vmix=Vs}-\text{vr}\\ \text{=S1}+\text{N1}-\left(\text{S3}+\text{N3}\right)\end{array}$$

be expressed.

ausgedrückt werden.Usually, the sound field control device 1 performs drive control or sound field control in consideration of an acoustic filter including the voice S3, but for convenience of explanation, S3 is neglected and considered as S3≒0. Furthermore, assuming that the vehicle-external noises are essentially the same regardless of the position of the occupants, N1≒N3 results. Consequently, the voice Vmix of expression (9) can be taken as $$\text{Vmix}\fallingdotseq \text{S1}$$

be expressed.

ausgedrückt werden.When the sound field control device 1 drives the noise muting section 13a using the signal of the voice Vmix, the voice Va output from the speaker group 52a at the position of the ear of the listener 22 can be used as $$\text{Va=f}\left(\text{Vmix}\right)\fallingdotseq \text{f}1\left(\text{S1}\right)\fallingdotseq -\text{S2}$$

be expressed.

Consequently, when the noise muting section 13a is driven, the voice h2 heard by the listener 22 becomes $$\text{H}2=\text{S2+N2+Va}\fallingdotseq \text{N2}$$

Therefore, the listener 22 does not hear the voice S2 from the speaker 21, but hears the vehicle-external noise N2.

<Summary of Third Embodiment>

According to the sound field control device 1 according to the present third embodiment above, a voice drowning out the voice of the speaker 21 is output from the speaker group 52a to the listener 22 without drowning out vehicle-external noise. According to such a configuration, the listener 22 can hear vehicle-external sounds associated with the driver's driving, such as another vehicle's horn, collision sound, sudden stop sound, ambulance siren, and so on.

<Other modifications>

The acquisition section 11, determination control section 12 and sound field control section 13 of FIG 1 , described above are hereinafter referred to as "Acquisition Section 11, etc." The acquisition section 11 etc. is replaced by an in 6 processing circuit 81 shown is realized. That is, the processing circuit 81 includes the acquisition section 11 that acquires face information of the plurality of occupants based on a captured image of a plurality of occupants in a vehicle, a determination control section 12 that, based on the face information acquired by the acquisition section 11 and a sound, received by a microphone inside the vehicle determines speakers and listeners from among the plurality of occupants, and a sound field control section 13 which, based on the result of determination by the determination control section 12, determines a sound field at the position of the plurality controlled by occupants. The processing circuit 81 may use dedicated hardware, but a processor executing programs stored in a memory may also be used. The processor corresponds to, for example, a central processing unit, a processing device, a computing device, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), etc.

When the processing circuit 81 is a special hardware, the processing circuit 81 corresponds to e.g. B. a simple circuit, a complex circuit, a programmed processor, parallel programmed processors, ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array) or a combination thereof. The respective functions of the individual sections of the acquisition section 11, etc. can be realized by circuits into which the processing circuit is divided, but the functions of the individual sections can be realized by a single processing circuit as a whole.

When the processing circuit 81 is a processor, the functions of the acquisition component 11 and so on are realized in combination with software and so on. The software etc. corresponds to, for example, software, firmware, or software and firmware. The software etc. is represented as a program and stored in a memory. As in 7 1, a processor 82 used in the processing circuit 81 reads out and executes programs stored in a memory 83, thereby realizing the functions of the various sections. That is, when implemented by the processing circuit 81, the sound field control device 1 includes the memory 83 in which programs are stored, through which finally a step in which, based on a picked-up image of a plurality of occupants in a vehicle, facial information of the plural occupants are acquired, a step in which, based on the acquired face information and a sound received by a microphone inside the vehicle, speakers and listeners are determined from among the plurality of occupants, and a step in which, based on the As a result of the determination of speaker and listener, a sound field is controlled at the position of the plurality of occupants to be executed. In other words, they can also be said to be programs by which the sequence and procedure of the acquisition section 11, etc. are executed by a computer. The memory 83 can be a non-volatile or volatile semiconductor memory such as e.g. RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), etc., HDD (Hard Disk drive), a magnetic disk, a floppy disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc), its driving device, etc., and further various storage media used in the future.

The structure when the respective functions of the acquisition section 11 etc. are executed by either hardware or software etc. has been explained above. However, it is not limited to this, and it may be a structure in which a part of the acquisition section 11 etc. is realized by dedicated hardware and another part is realized by software and so on. It is possible, for example, that regarding the acquisition section 11 by means of the processing circuit 81, an interface and a receiver as special hardware whose functions are implemented, and apart from this the functions are implemented in that the processing circuit 81 as a processor 82 reads out programs stored in the memory 83 and executes them.

This enables the processing circuit 81 to realize the respective functions described above by hardware, software, etc., or their combination.

Further, the sound field control device 1 explained above can also be applied to a sound field control system constructed as a system in which a vehicle device such as a DMS (Driver Monitoring System) device, etc., a communication terminal including a portable terminal such as a cellular phone , smartphone, tablet, etc., functions of applications installed in at least one of the in-vehicle device and the communication terminal, and a server are appropriately combined. In this case, the respective functions of the sound field control device 1 explained above or the respective constituent parts can be distributed to the respective devices constituting the system or arranged centrally in one device.

8th 12 is a block diagram showing the structure of a server 91 according to the present modification. The server 91 in 8th includes a communication section 91a and a control section 91b, and can perform wireless communication with a vehicle device 93 such as a DMS device of a vehicle 92.

Through wireless communication with the in-vehicle device 93, the communication section 91a as an acquisition section receives face information of a plurality of occupants in the vehicle and sounds in the vehicle acquired by the in-vehicle device 93.

The control section 91b has the same functions as the determination control section 12 and the sound field control section 13 by having an unillustrated processor of the server 91 execute programs stored in an unillustrated memory of the server 91 1 on. That is, the control section 91b determines, based on the face information and sounds received by the communication section 91a, speakers and listeners from among the plurality of occupants in the vehicle, and generates control signals to control sound fields at the position of the plurality based on the result of the determination controlled by occupants. Then, the communication section 91a sends the control signals generated by the control section 91b to the vehicle device 93. According to the server 91 constructed in this way, the same effects as the sound field control device 1 explained in the first embodiment can be obtained.

9 12 is a block diagram showing the construction of a communication terminal 96 according to the present modification. The communication terminal 96 in 9 comprises a communication section 96a similar to the communication section 91a and a control section 96b similar to the control section 91b, and can perform wireless communication with a vehicle device 98 of a vehicle 97. As the communication terminal 96, a mobile terminal such as a mobile phone, a smartphone, a tablet, etc., which the driver of the vehicle 97 carries with him, is used, for example. According to the communication terminal 96 constructed in this way, the same effects as the sound field control device 1 explained in the first embodiment can be obtained.

In the present invention, any combination of the respective embodiments and the respective modifications is possible within the scope of the invention, and the respective embodiments and modifications can also be modified or omitted as appropriate.

The present invention has been explained in detail, and the above explanation is an example in all aspects, and the present invention is not limited thereto. Also, innumerable modified examples not illustrated by way of example are contemplated as conceivable without departing from the scope of the present invention.

Reference List

1

sound field control device

11

acquisition section

12

destination control section

13

Sound field control section

13a

noise mute section

13b

noise drive section

13c

drive control section

51

Micro

52

speaker

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• JP 2009253526 A [0003]

Claims (10)

A sound field control device that controls a sound field at a specific position in a vehicle, the sound field being generated by a speaker in the vehicle, comprising an acquisition section that acquires face information of the plurality of passengers based on a captured image of a plurality of passengers in the vehicle, a determination control section that determines speakers and listeners from among the plurality of occupants based on the face information acquired by the acquisition section and a sound received through a microphone inside the vehicle, and a sound field control section that controls a sound field at the position of the plurality of occupants based on the determination result by the determination control section. Sound field control device according to claim 1 wherein the sound field control section comprises a noise muting section which, based on the result of the determination by the determination control section, performs driving for outputting a noise drowning out the speaker's voice from the speaker to the listener(s), a noise driving section which based on the result of determination by the determination control section, performs driving to output the speaker's voice from the speaker to the listener(s), and a drive control section that performs drive control controlling driving of the noise muting section and driving of the noise driving section. Sound field control device according to claim 2 wherein the noise muting section performs a drive in which, without drowning out noise outside the vehicle, a noise drowning out the speaker's voice is output from the speaker to the listener(s). Sound field control device according to claim 2 wherein the noise driving section changes the frequency characteristic of the voice output from the speaker to the listener concerned based on characteristics of the listener. Sound field control device according to claim 2 , wherein the drive control section determines, based on the facial information acquired by the acquiring section, the speaker's facial expression and the listener's facial expression, the speaker's facial direction and the listener's facial direction, or a speaker's facial movement and a listener's facial movement, and based on the result of this determination performs the drive control. Sound field control device according to claim 2 wherein the drive control section performs drive control based on whether or not the speaker makes a hands-free call. Sound field control device according to claim 2 , wherein the drive control section performs the drive control based on an operation by the speaker and the listener. Sound field control device according to claim 2 wherein the drive control section performs drive control based on the content of the speech of the speaker and the listener. Sound field control device according to claim 2 , wherein the drive control section performs drive control per handset A sound field control method for controlling a sound field at a specific position in the vehicle, the sound field being generated by a speaker in the vehicle by which facial information of the plurality of occupants is acquired from a picked-up image of a plurality of occupants in the vehicle based on the acquired facial information and a sound received through the microphone inside the vehicle, speakers and listeners are determined from among the plurality of occupants, and based on the result of the speaker and listener determination, a sound field is controlled at the position of the plurality of occupants will.

Images