JP6995254B2 - Sound field control device and sound field control method - Google Patents

Description

The present invention relates to a sound field control device and a sound field control method for controlling a sound field at a specific position in a vehicle, which is generated by a speaker in the vehicle.

Conventionally, a system that provides a comfortable conversation environment in a vehicle has been proposed. For example, in a hand-free telephone system equipped with a microphone (microphone) and a speaker, a comfortable call environment is realized by noise canceling technology that removes noise in the vehicle and echo canceling technology that removes echo in the vehicle. There is. In addition, in order to facilitate conversations among occupants in the vehicle, microphones and speakers are placed near each occupant, the voice of one occupant is captured from the microphone, and the voice is output from the speaker near the other occupant. Technology has been proposed.

As a technique for controlling the sound field in the vehicle by using a microphone and a speaker, for example, the techniques of Patent Document 1 and Patent Document 2 have been proposed. Patent Document 1 proposes a technique for controlling a sound field in response to rotation of the head. Patent Document 2 proposes a technique for controlling a sound field according to the position of a seat.

Japanese Unexamined Patent Publication No. 2009-25526 Japanese Unexamined Patent Publication No. 07-184298

In the above-mentioned conventional technology, it is assumed that the sound field is appropriately controlled only with respect to the voice from a specific occupant called the driver. Therefore, there is a problem that the sound field in the vehicle cannot be appropriately controlled with respect to the sound from an arbitrary occupant in the vehicle.

Therefore, the present invention has been made in view of the above-mentioned problems, and provides a technique capable of appropriately controlling the sound field in the vehicle with respect to the sound from an arbitrary occupant in the vehicle. With the goal.

The sound field control device according to the present invention accepts the acquisition unit that acquires the face information of a plurality of occupants from the images taken by a plurality of occupants in the vehicle, the face information acquired by the acquisition unit, and the microphone in the vehicle. A sound field that controls the sound field at the positions of a plurality of occupants based on the judgment control unit that determines a speaker and a listener from among a plurality of occupants based on the sound and the result of the determination by the determination control unit. The sound field control unit includes a control unit, and the sound field control unit includes a mute sound drive unit that drives the listener to output a sound that cancels the voice from the speaker based on the result of the judgment by the judgment control unit, and a judgment control unit. Based on the result of the determination by the unit, the voice drive unit that drives the speaker to output the voice from the speaker to the listener, and the drive that controls the drive of the mute voice drive unit and the drive of the voice drive unit. Including the control unit .

According to the present invention, a speaker and a listener are determined from a plurality of occupants based on the acquired face information and the voice received by the microphone, and based on the result of the determination of the speaker and the listener, the speaker and the listener are determined. Control the sound field at the positions of multiple occupants. According to such a configuration, the sound field in the vehicle can be appropriately controlled with respect to the voice from any occupant in the vehicle.

The objects, features, embodiments and advantages of the present invention will be made clearer by the following detailed description and accompanying drawings.

It is a block diagram which shows the structure of the sound field control apparatus which concerns on Embodiment 1. FIG. It is a block diagram which shows the structure of the sound field control apparatus which concerns on Embodiment 2. FIG. It is a flowchart which shows the operation of the sound field control apparatus which concerns on Embodiment 2. It is a figure for demonstrating operation of the sound field control apparatus which concerns on Embodiment 2. FIG. It is a figure for demonstrating operation of the sound field control apparatus which concerns on Embodiment 3. FIG. It is a block diagram which shows the hardware composition of the sound field control device which concerns on other modification. It is a block diagram which shows the hardware composition of the sound field control device which concerns on other modification. It is a block diagram which shows the structure of the server which concerns on other modification. It is a block diagram which shows the structure of the communication terminal which concerns on other modification.

<Embodiment 1>
FIG. 1 is a block diagram showing a configuration of a sound field control device 1 according to a first embodiment of the present invention. The sound field control device 1 can be applied to, for example, a DMS (Driver Monitoring System) or the like. The sound field control device 1 of FIG. 1 is connected to a microphone (microphone) 51 and a speaker 52 in a vehicle via wireless or wired.

The microphone 51 receives voices emitted from the mouths of a plurality of occupants in the vehicle. For example, a super-directional microphone capable of receiving only voice from each occupant is applied to the microphone 51. The plurality of occupants include, for example, a driver, a passenger in the passenger seat, and a passenger in the rear seat.

The speaker 52 generates a sound field in the vehicle under the control of the sound field control device 1. For example, a super-directional speaker such as a parametric array speaker capable of generating a different sound field at each occupant's position is applied to the speaker 52.

Next, the sound field control device 1 will be described. As will be described below, the sound field control device 1 can control the sound field generated by the speaker 52 at a specific position in the vehicle.

The sound field control device 1 of FIG. 1 includes an acquisition unit 11, a determination control unit 12, and a sound field control unit 13.

The acquisition unit 11 acquires the face information of the plurality of occupants from the images taken by the plurality of occupants in the vehicle. The face information is information including feature points and states of the occupant's face, and is, for example, information on the position of the occupant's mouth, the position of the occupant's ears (for example, both ears), and the movement of the mouth. The acquisition unit 11 uses at least one of a device capable of recognizing an image of an occupant taken by a photographing device such as a camera and an interface thereof.

The determination control unit 12 determines a speaker and a listener from among a plurality of occupants based on the face information acquired by the acquisition unit 11 and the voice received by the microphone 51. For example, when the determination control unit 12 recognizes that the occupant's mouth is open / closed from the time-series face information, and recognizes that the voice waveform received by the microphone 51 is a vowel type or a consonant type. In addition, the occupant is determined to be a speaker, and the other occupants are determined to be listeners. Further, for example, when the determination control unit 12 estimates that the occupant is speaking to the face information in time series using the lip reading function, and detects the sound pressure above a certain level with the microphone 51, the determination control unit 12 determines the occupant. It is determined that the speaker is a speaker, and the other occupants are determined to be listeners.

The sound field control unit 13 controls the speaker 52 based on the result of the determination by the determination control unit 12 to control the sound field at a specific position in the vehicle, and by extension, the sound field at the positions of a plurality of occupants. For example, the sound field control unit 13 transfers the speaker 52 to the listener's ear based on the position of the speaker's mouth determined by the determination control unit 12 and the position of the listener's ear determined by the determination control unit 12. On the other hand, control is performed to selectively output the speaker's voice and the voice that cancels the speaker's voice.

<Summary of Embodiment 1>
According to the sound field control device 1 according to the first embodiment as described above, the speaker and the listener are selected from a plurality of occupants based on the acquired face information and the voice received by the microphone 51. Judgment is made, and the sound field at the positions of a plurality of occupants is controlled based on the judgment results of the speaker and the listener. According to such a configuration, for example, any occupant who emits a voice in the vehicle can be determined as a speaker, and other occupants can be determined as a listener. The sound field at the position of the occupant can be appropriately controlled. This is particularly effective in the recently proposed rideshare.

<Embodiment 2>
FIG. 2 is a block diagram showing the configuration of the sound field control device 1 according to the second embodiment of the present invention. Hereinafter, among the components according to the second embodiment, the same or similar components as those described above will be designated by the same or similar reference numerals, and different components will be mainly described.

The sound field control device 1 of FIG. 2 is communicably connected to the directional microphone 51a, the speaker array 52a, and the image recognition device 53. The directional microphone 51a and the speaker array 52a are included in the concept of the microphone 51 and the speaker 52 in FIG. Specifically, the directional microphone 51a receives voices emitted from the mouths of a plurality of occupants in the vehicle, similarly to the microphone 51 described in the first embodiment. Similar to the speaker 52 described in the first embodiment, the speaker array 52a generates a sound field in the vehicle under the control of the sound field control device 1. The image recognition device 53 generates face information of a plurality of occupants by performing recognition processing on images taken by a plurality of occupants in the vehicle.

Next, the sound field control device 1 will be described. The sound field control device 1 of FIG. 2 includes a face information acquisition unit 11a, an utterance voice determination unit 12a, a speaker / listener recognition unit 12b, a muffling voice drive unit 13a, a voice drive unit 13b, and a drive control unit 13c. , A microphone control unit 14 is provided. The face information acquisition unit 11a is included in the concept of the acquisition unit 11 of FIG. 1, and the utterance voice determination unit 12a and the speaker / listener recognition unit 12b are included in the concept of the determination control unit 12 of FIG. The mute voice drive unit 13a, the voice drive unit 13b, and the drive control unit 13c are included in the concept of the sound field control unit 13 of FIG.

The face information acquisition unit 11a acquires the face information generated by the image recognition device 53. The face information is information including feature points and states of the occupant's face as in the first embodiment.

The microphone control unit 14 has the directivity of the directional microphone 51a based on the face information acquired by the face information acquisition unit 11a so that the directional microphone 51a can accurately receive the voice of the position of the occupant's mouth. To control. The directivity adjustment logic of the directional microphone 51a is not limited, and any existing adjustment logic can be used.

The utterance voice determination unit 12a is a voice emitted by the occupant based on the face information acquired by the face information acquisition unit 11a and the voice received by the directional microphone 51a. Judge whether or not. For such a determination, for example, the determination described by the determination control unit 12 of the first embodiment is used.

When the utterance voice determination unit 12a determines that the voice received by the directional microphone 51a is the voice from the occupant, the speaker / listener recognition unit 12b receives the face information acquired by the face information acquisition unit 11a and the face information. The speaker and the listener are identified based on the result of the determination of the utterance voice determination unit 12a. For example, the speaker / listener recognition unit 12b determines that the speaker is determined to be the occupant who has made a voice by the utterance voice determination unit 12a among the plurality of occupants whose face information has been acquired by the face information acquisition unit 11a. Crew members other than are judged to be listeners. At this time, the speaker-listener recognition unit 12b also determines the position of the speaker's mouth and the position of the listener's ear. The result of the determination by the speaker / listener recognition unit 12b is output to the mute voice drive unit 13a and the voice drive unit 13b via the drive control unit 13c.

The muffling voice driving unit 13a drives the speaker array 52a to output the voice for canceling the voice from the speaker to the listener based on the result of the determination by the speaker listener recognition unit 12b. Specifically, the muffling voice driving unit 13a is based on the position of the speaker's mouth and the position of the listener's ear determined by the speaker-listener recognition unit 12b, and the speaker's voice received by the directional microphone 51a. The speaker array 52a is controlled so that the voice that cancels the voice of the speaker is output to the position of the listener's ear. The cancellation of the speaker's voice may be such that the listener cannot hear the speaker's voice, and may not be a complete cancellation.

The voice driving unit 13b drives the speaker array 52a to output the voice from the speaker to the listener based on the result of the determination by the speaker listener recognition unit 12b. Specifically, the voice driving unit 13b is based on the position of the speaker's mouth and the position of the listener's ear determined by the speaker-listener recognition unit 12b, and the voice of the speaker received by the directional microphone 51a. , The speaker array 52a is controlled so that the speaker's voice is output to the position of the listener's ear.

The drive control unit 13c performs drive control for controlling the drive of the mute voice drive unit 13a and the drive of the voice drive unit 13b. In the second embodiment, the drive control unit 13c performs drive control based on whether or not the speaker is making a hands-free call. The speaker here may be a driver or an occupant other than the driver.

For example, when it is determined that the speaker is making a hands-free call, the drive control unit 13c controls all listeners to drive the mute voice drive unit 13a without driving the voice drive unit 13b. .. Further, for example, when it is determined that the speaker is not making a hands-free call, the drive control unit 13c controls all listeners to drive the voice drive unit 13b without driving the mute voice drive unit 13a. conduct.

Whether or not the speaker is making a hands-free call may be determined based on the status information of the hands-free call input from the hand-free call control device (not shown), or face information acquisition. The determination may be made based on the face information acquired by the unit 11a.

<Operation>
FIG. 3 is a flowchart showing the operation of the sound field control device 1 according to the second embodiment.

First, in step ST1, the face information acquisition unit 11a acquires face information from the image recognition device 53.

In step ST2, the microphone control unit 14 controls the directivity of the directional microphone 51a based on the face information acquired by the face information acquisition unit 11a.

In step ST3, the utterance voice determination unit 12a receives the voice received by the directional microphone 51a based on the face information acquired by the face information acquisition unit 11a and the voice received by the directional microphone 51a by the occupant. It is determined whether or not the voice is emitted. That is, the utterance voice determination unit 12a determines whether or not the occupant is speaking. If it is determined that the occupant is speaking, the process proceeds to step ST4, and if it is determined that the occupant is not speaking, the process returns to step ST1.

In step ST4, the speaker / listener recognition unit 12b determines the speaker and the listener based on the face information acquired by the face information acquisition unit 11a and the determination result of the utterance voice determination unit 12a.

In step ST5, the drive control unit 13c determines whether or not the speaker is making a hands-free call. If it is determined that the speaker is making a hands-free call, the process proceeds to step ST6, and if it is determined that the speaker is not making a hands-free call, the process proceeds to step ST7.

In step ST6, the drive control unit 13c controls to drive the mute voice drive unit 13a. As a result, the muffling voice driving unit 13a drives all listeners to output the voice for canceling the voice from the speaker based on the result of the determination by the speaker listener recognition unit 12b. After that, the process returns to step ST1.

In step ST7, the drive control unit 13c controls to drive the voice drive unit 13b. As a result, the voice driving unit 13b drives the speaker array 52a to output the voice from the speaker to all listeners based on the result of the determination by the speaker listener recognition unit 12b. After that, the process returns to step ST1.

<Summary of Embodiment 2>
According to the sound field control device 1 according to the second embodiment as described above, whether the speaker is making a hands-free call regardless of whether the speaker is a driver or a occupant other than the driver. Based on whether or not, the drive control for controlling the drive of the muffling voice driving unit 13a and the driving of the voice driving unit 13b is performed. According to such a configuration, when any occupant is making a hands-free call, the conversation can be concealed from other occupants.

<Modification 1>
The drive control unit 13c may perform drive control for controlling the drive of the mute voice drive unit 13a and the drive of the voice drive unit 13b based on an operation other than the hands-free call. An example thereof will be described below.

<Modification 1A>
The drive control unit 13c may perform drive control based on operations from the speaker and the listener. For example, when both the speaker and the listener perform an operation indicating the intention of permission to talk to an icon or key (not shown), the drive control unit 13c does not drive the mute voice drive unit 13a and is a voice drive unit. Control to drive 13b may be performed. Further, for example, when both the speaker and the listener perform an operation indicating the intention of refusing the conversation to an icon or key (not shown), the drive control unit 13c does not drive the voice drive unit 13b but drives the mute voice. Control to drive the unit 13a may be performed.

If the intention of the speaker and the intention of the listener are different, the drive control may be performed according to a predetermined rule such as a conversation refusal priority rule or a speaker priority rule. Here, the conversation refusal priority rule means that when at least one of the speaker and the listener performs an operation indicating the intention of refusing the conversation, the muffling voice driving unit 13a is preferentially driven without driving the voice driving unit 13b. It is a rule to do. The speaker priority rule is a rule for driving and controlling the muffling voice driving unit 13a and the voice driving unit 13b by giving priority to the speaker's operation over the listener's operation.

The drive control unit 13c may store the settings by the operation from the speaker and the listener, and thereafter perform the drive control based on the settings. Further, the operations from the speaker and the listener are not limited to the operations for icons and keys (not shown), and may be, for example, gesture operations detected by the speaker and the listener using a camera or the like. The gesture operation may include, for example, a gesture operation for creating an OK mark with a finger as a gesture operation for indicating a conversation permission, or may include a gesture operation for manually dispelling as a gesture operation for indicating a conversation refusal. .. The gesture operation may be appropriately changed depending on the range (for example, region, country) in which the sound field control device 1 is located.

<Modification 1B>
Even if the drive control unit 13c determines the facial expression of the speaker and the facial expression of the listener based on the face information acquired by the face information acquisition unit 11a, and performs drive control based on the result of the determination. good. For example, when the drive control unit 13c determines that one of the faces of the speaker and the listener is winking, the drive control unit 13c determines that the intention to permit conversation is indicated and drives the mute voice drive unit 13a. Control may be performed to drive the voice driving unit 13b without using the method. Further, for example, when the drive control unit 13c determines that one of the faces of the speaker and the listener has a distorted facial expression, the drive control unit 13c determines that the intention to refuse the conversation is indicated, and the voice drive unit 13b You may control to drive the mute voice driving unit 13a without driving.

Alternatively, the drive control unit 13c determines the orientation of the speaker's face and the orientation of the listener's face based on the face information acquired by the face information acquisition unit 11a, and performs drive control based on the result of the determination. You may. For example, when the drive control unit 13c determines that one of the faces of the speaker and the listener is facing the other, the drive control unit 13c determines that the intention to permit conversation is indicated and drives the mute voice drive unit 13a. Control may be performed to drive the voice driving unit 13b without using the method. Further, for example, when the drive control unit 13c determines that one of the faces of the speaker and the listener is turned away from the other, the drive control unit 13c determines that the intention of refusing the conversation is indicated and drives the voice drive unit 13b. You may control to drive the mute voice driving unit 13a without doing so.

Alternatively, the drive control unit 13c determines the movement of the speaker's face and the movement of the listener's face based on the face information acquired by the face information acquisition unit 11a, and performs drive control based on the result of the determination. You may. For example, when the drive control unit 13c determines that one of the faces of the speaker and the listener approaches the other or sways in the vertical direction, the drive control unit 13c determines that the intention to permit conversation is indicated and mute the sound. Control may be performed to drive the voice driving unit 13b without driving the voice driving unit 13a. Further, for example, when the drive control unit 13c determines that one of the faces of the speaker and the listener is separated from the other or sways in the lateral direction, the drive control unit 13c determines that the intention of refusing the conversation is indicated. Control may be performed to drive the mute voice driving unit 13a without driving the voice driving unit 13b.

In the present modification 1B, when the intention of the speaker and the intention of the listener are different, the drive control is performed according to predetermined rules such as the conversation refusal priority rule and the speaker priority rule as in the modification 1A. You may.

<Modification 1C>
The drive control unit 13c may perform drive control based on the contents of voices from the speaker and the listener. For example, when the drive control unit 13c determines that the speaker has called the listener's name or the listener has responded to the call for the name, the drive control unit 13c determines that the intention to permit the conversation is indicated and mute the sound. Control may be performed to drive the voice driving unit 13b without driving the voice driving unit 13a. Further, for example, when the drive control unit 13c determines that the speaker did not call the listener's name or the listener did not respond to the call for the name, the drive control unit 13c determined that the intention to refuse the conversation was indicated. Alternatively, control may be performed to drive the muffling voice driving unit 13a without driving the voice driving unit 13b.

In the present modification 1C, when the intention of the speaker and the intention of the listener are different, the drive control is performed according to predetermined rules such as the conversation refusal priority rule and the speaker priority rule as in the modification 1A. You may.

<Modification 2>
In the second embodiment, the drive control unit 13c uniformly performs drive control to all listeners when it is determined that the speaker is making a hands-free call, but the present invention is not limited to this. .. For example, the drive control unit 13c may perform drive control for each listener. That is, the drive control unit 13c may control to drive the voice drive unit 13b for a certain listener, and at the same time, control to drive the mute voice drive unit 13a for another listener. According to such a configuration, it is possible to simultaneously perform contradictory drives of driving the muffling voice driving unit 13a and driving the voice driving unit 13b, so that a comfortable talking space in the vehicle can be realized.

<Modification 3>
For example, when the voice drive unit 13b is driven to output the voice from the speaker to the listener from the speaker array 52a while the music is being output in the vehicle, the sound field control device 1 lowers the volume of the music. Control may be performed. Further, when the drive control unit 13c drives the voice drive unit 13b to output the voice from the speaker, the drive control unit 13c drives the mute voice drive unit 13a to cancel noise (for example, noise caused by vehicle engine noise or running). Cancellation) may be performed.

<Modification example 4>
The voice driving unit 13b may change the frequency characteristics of the voice output from the speaker array 52a to the listener based on the characteristics such as the hearing ability and the taste of the listener. For example, when the listener is a person who has difficulty in hearing the low frequency sound, the voice driving unit 13b may increase the low frequency component of the voice output to the listener from the speaker array 52a.

The characteristics of the listener may be acquired from the listener's smartphone and hearing aid, or may be stored in advance in the sound field control device 1 by an operation from the user.

<Embodiment 3>
FIG. 4 is a diagram for explaining the operation of the sound field control device 1 according to the second embodiment described above, and FIG. 5 is a diagram for explaining the operation of the sound field control device 1 according to the third embodiment of the present invention. It is a figure to do. The block configuration of the sound field control device 1 according to the third embodiment is the same as the block configuration (FIG. 2) of the sound field control device 1 according to the second embodiment. Therefore, among the components according to the third embodiment, the same or similar components as those described above are designated by the same or similar reference numerals, and different components will be mainly described.

First, the operation of the sound field control device 1 according to the second embodiment will be described with reference to FIG. The voice Vs received by the directional microphone 51a has a waveform in which the voice S1 reaching the directional microphone 51a from the speaker 21 and the outside sound N1 reaching the directional microphone 51a from the outside of the vehicle are superimposed. Therefore, the voice Vs is
Vs = S1 + N1 ... (1)
It can be expressed as.

When the muffling voice driving unit 13a is not driven, the voice h1 that the listener 22 can hear is the voice S2 that reaches the position of the ear of the listener 22 from the speaker 21 and the outside of the vehicle that reaches the position of the ear of the listener 22 from the outside of the vehicle. It has a waveform superimposed on the sound N2. Therefore, the voice h1 is
h1 = S2 + N2 ... (2)
It can be expressed as.

The sound field control device 1 generates a voice Va output from the speaker array 52a to the position of the ear of the listener 22 based on the voice Vs of the above equation (1). This voice Va is
Va = f (Vs) = f (S1 + N1) ≒ f1 (S1) + f2 (N1) ... (3)
It can be expressed as.

Since this voice Va cancels the voice S2 and the outside sound N1,
f1 (S1) ≈ −S2 ・ ・ ・ (4)
f2 (N1) ≒ -N1 ... (5)
The relationship holds.

Therefore, when the muffling voice driving unit 13a is being driven, the voice h2 heard by the listener 22 is
h2 = S2 + N2 + Va
= S2 + N2 + f1 (S1) + f2 (N1)
= {S2 + f1 (S1)} + {N2 + f2 (N1)}
≒ 0 + (N2-N1) ・ ・ ・ (6)
Will be. In the last line of the equation (6), the equations (4) and (5) were applied.

Assuming that the sound outside the vehicle is almost the same regardless of the position of the occupant, N2 ≈ N1. Therefore, the voice h2 in the equation (6) is
h2 ≒ 0 ・ ・ ・ (7)
Will be.

Therefore, for the listener 22, not only the voice S2 from the speaker 21 but also the outside sound N2 is erased. However, for example, when the listener 22 is the driver, the driver (listener 22) can hear the horn, collision sound, sudden braking sound of another vehicle, and the outside sound related to driving such as the siren of the emergency vehicle. Is desirable.

Therefore, in the third embodiment, the muffling voice driving unit 13a drives the listener to output the voice that cancels the voice from the speaker without canceling the voice from the outside of the vehicle from the speaker array 52a. This allows the listener 22 to hear the outside sound conditionally or unconditionally.

Hereinafter, the operation of the sound field control device 1 according to the third embodiment will be described with reference to FIG. In the third embodiment, the vehicle is provided with a microphone 56 for positively receiving the sound outside the vehicle. The microphone 56 is, for example, an omnidirectional microphone, and is provided at a position where it is easier to receive outside sound than the directional microphone 51a.

The voice Vr received by the microphone 56 has a waveform in which the voice S3 reaching the microphone 56 from the speaker 21 and the outside sound N3 reaching the microphone 56 from the outside of the vehicle are superimposed. Therefore, the voice Vr is
Vr = S3 + N3 ... (8)
It can be expressed as.

The sound field control device 1 according to the third embodiment treats the voice Vr and the voice Vs as input voices, and generates a signal indicating the voice Vmix of the difference. This voice Vmix is
Vmix = Vs-Vr
= S1 + N1- (S3 + N3) ... (9)
It can be expressed as.

Normally, the sound field control device 1 performs drive control and thus sound field control in consideration of an acoustic filter including the voice S3, but for the sake of simplicity of explanation, S3 is ignored and S3≈0 is set. .. Further, assuming that the sound outside the vehicle is almost the same regardless of the position of the occupant, N1 ≈ N3. From these, the audio Vmix of equation (9) is
Vmix ≒ S1 ... (10)
It can be expressed as.

When the sound field control device 1 drives the muffling voice driving unit 13a using the voice Vmix signal, the voice Va output from the speaker array 52a to the position of the ear of the listener 22 becomes.
Va = f (Vmix) ≒ f1 (S1) ≒ −S2 ・ ・ ・ (11)
It can be expressed as.

Therefore, when the muffling voice driving unit 13a is being driven, the voice h2 heard by the listener 22 is
h2 = S2 + N2 + Va≈N2 ... (12)
Will be.

Therefore, the listener 22 cannot hear the voice S2 from the speaker 21, but can hear the outside sound N2.

<Summary of Embodiment 3>
According to the sound field control device 1 according to the third embodiment as described above, the speaker array 52a drives the listener 22 to output the sound that cancels the sound from the speaker 21 without canceling the sound outside the vehicle. According to such a configuration, the listener 22 can hear outside vehicle sounds related to the driver's driving, such as horns, collision sounds, sudden braking sounds of other vehicles, and sirens of emergency vehicles.

<Other variants>
The acquisition unit 11, the determination control unit 12, and the sound field control unit 13 of FIG. 1 described above are hereinafter referred to as “acquisition unit 11 and the like”. The acquisition unit 11 and the like are realized by the processing circuit 81 shown in FIG. That is, the processing circuit 81 receives the acquisition unit 11 that acquires the face information of the plurality of occupants from the images taken by the plurality of occupants in the vehicle, the face information acquired by the acquisition unit 11, and the microphone in the vehicle. A sound that controls the sound field at the positions of a plurality of occupants based on the determination control unit 12 that determines a speaker and a listener from among a plurality of occupants based on the voice and the determination result by the determination control unit 12. A field control unit 13 is provided. Dedicated hardware may be applied to the processing circuit 81, or a processor that executes a program stored in the memory may be applied to the processing circuit 81. The processor corresponds to, for example, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), and the like.

When the processing circuit 81 is dedicated hardware, the processing circuit 81 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate). Array), or a combination of these. Each of the functions of each part such as the acquisition unit 11 may be realized by a circuit in which processing circuits are distributed, or the functions of each part may be collectively realized by one processing circuit.

When the processing circuit 81 is a processor, the functions of the acquisition unit 11 and the like are realized by combining with software and the like. The software or the like corresponds to, for example, software, firmware, or software and firmware. Software and the like are described as programs and stored in memory. As shown in FIG. 7, the processor 82 applied to the processing circuit 81 realizes the functions of each part by reading and executing the program stored in the memory 83. That is, when the sound field control device 1 is executed by the processing circuit 81, the step of acquiring the face information of the plurality of occupants from the captured images of the plurality of occupants in the vehicle, the acquired face information, and the vehicle. Based on the voice received by the microphone inside, the step of determining the speaker and the listener from among a plurality of occupants, and the sound field at the positions of the plurality of occupants based on the result of the determination of the speaker and the listener. It includes a step to control and a memory 83 for storing a program that will be executed as a result. In other words, it can be said that this program causes the computer to execute the procedure or method of the acquisition unit 11 or the like. Here, the memory 83 is, for example, non-volatile such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory). Volatile semiconductor memory, HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), its drive device, etc., or any storage medium used in the future. You may.

The configuration in which each function of the acquisition unit 11 and the like is realized by either hardware or software has been described above. However, the present invention is not limited to this, and a configuration may be configured in which a part of the acquisition unit 11 or the like is realized by dedicated hardware and another part is realized by software or the like. For example, the acquisition unit 11 realizes its function by a processing circuit 81 as dedicated hardware, an interface, a receiver, and the like, and in other cases, the processing circuit 81 as a processor 82 reads a program stored in the memory 83. It is possible to realize the function by executing it.

As described above, the processing circuit 81 can realize each of the above-mentioned functions by hardware, software, or a combination thereof.

Further, the sound field control device 1 described above includes at least one of a vehicle device such as a DMS (Driver Monitoring System) device, a communication terminal including a mobile terminal such as a mobile phone, a smartphone and a tablet, and a vehicle device and a communication terminal. It can also be applied to a sound field control system constructed as a system by appropriately combining the functions of the application to be installed and the server. In this case, each function or each component of the sound field control device 1 described above may be distributed and arranged in each device for constructing the system, or may be centrally arranged in any of the devices. May be good.

FIG. 8 is a block diagram showing the configuration of the server 91 according to this modification. The server 91 of FIG. 8 includes a communication unit 91a and a control unit 91b, and can perform wireless communication with a vehicle device 93 such as a DMS device of the vehicle 92.

The communication unit 91a, which is an acquisition unit, receives the face information of a plurality of occupants in the vehicle acquired by the vehicle device 93 and the voice in the vehicle by performing wireless communication with the vehicle device 93.

The control unit 91b has the same functions as the determination control unit 12 and the sound field control unit 13 of FIG. 1 when a processor (not shown) of the server 91 or the like executes a program stored in a memory (not shown) of the server 91. is doing. That is, the control unit 91b determines the speaker and the listener from among the plurality of occupants in the vehicle based on the face information and the voice received by the communication unit 91a, and based on the result of the determination, the plurality of occupants. Generates a control signal to control the sound field at the position. Then, the communication unit 91a transmits the control signal generated by the control unit 91b to the vehicle device 93. According to the server 91 configured in this way, the same effect as that of the sound field control device 1 described in the first embodiment can be obtained.

FIG. 9 is a block diagram showing the configuration of the communication terminal 96 according to this modification. The communication terminal 96 of FIG. 9 includes a communication unit 96a similar to the communication unit 91a and a control unit 96b similar to the control unit 91b, and can perform wireless communication with the vehicle device 98 of the vehicle 97. ing. The communication terminal 96 is, for example, a mobile terminal such as a mobile phone, a smartphone, or a tablet carried by the driver of the vehicle 97. According to the communication terminal 96 configured in this way, the same effect as that of the sound field control device 1 described in the first embodiment can be obtained.

In the present invention, each embodiment and each modification can be freely combined, and each embodiment and each modification can be appropriately modified or omitted within the scope of the invention.

Although the present invention has been described in detail, the above description is exemplary in all embodiments and the present invention is not limited thereto. It is understood that innumerable variations not illustrated can be assumed without departing from the scope of the present invention.

1 Sound field control device, 11 Acquisition unit, 12 Judgment control unit, 13 Sound field control unit, 13a Mute voice drive unit, 13b Voice drive unit, 13c Drive control unit, 51 Microphone, 52 Speaker.

Claims (9)

A sound field control device that controls a sound field at a specific position in the vehicle, which is generated by a speaker in the vehicle.
An acquisition unit that acquires face information of the plurality of occupants from images taken by the plurality of occupants in the vehicle, and an acquisition unit.
A determination control unit that determines a speaker and a listener from among the plurality of occupants based on the face information acquired by the acquisition unit and the voice received by the microphone in the vehicle.
A sound field control unit that controls the sound field at the positions of the plurality of occupants based on the result of the determination by the determination control unit is provided .
The sound field control unit is
Based on the result of the determination by the determination control unit, the mute voice driving unit that drives the speaker to output the voice that cancels the voice from the speaker to the listener.
Based on the result of the determination by the determination control unit, the voice drive unit that drives the speaker to output the voice from the speaker to the listener.
A drive control unit that performs drive control for controlling the drive of the mute voice drive unit and the drive of the voice drive unit.
Including sound field control device. The sound field control device according to claim 1 .
The mute voice drive unit is
A sound field control device that drives a speaker to output a sound that cancels a sound from a speaker without canceling a sound from the outside of the vehicle to the listener. The sound field control device according to claim 1 .
The voice drive unit is
A sound field control device that changes the frequency characteristics of the sound output from the speaker to the listener based on the characteristics of the listener. The sound field control device according to claim 1 .
The drive control unit
Based on the face information acquired by the acquisition unit, the facial expression of the speaker and the facial expression of the listener, the orientation of the face of the speaker and the orientation of the face of the listener, or the orientation of the speaker's face. A sound field control device that determines the movement of the face and the movement of the listener's face, and performs the drive control based on the result of the determination. The sound field control device according to claim 1 .
The drive control unit
A sound field control device that performs the drive control based on whether or not the speaker is making a hands-free call. The sound field control device according to claim 1 .
The drive control unit
A sound field control device that performs the drive control based on operations from the speaker and the listener. The sound field control device according to claim 1 .
The drive control unit
A sound field control device that performs the drive control based on the contents of voices from the speaker and the listener. The sound field control device according to claim 1 .
The drive control unit is a sound field control device that performs the drive control for each listener. A sound field control method for controlling a sound field at a specific position in the vehicle, which is generated by a speaker in the vehicle.
The face information of the plurality of occupants is acquired from the images taken by the plurality of occupants in the vehicle, and the face information of the plurality of occupants is acquired.
Based on the acquired face information and the voice received by the microphone in the vehicle, the speaker and the listener are determined from the plurality of occupants.
Based on the judgment results of the speaker and the listener, the sound field at the positions of the plurality of occupants is controlled .
The control of the sound field at the positions of the plurality of occupants is
Based on the result of the determination, the drive to output the voice for canceling the voice from the speaker from the speaker to the listener, and based on the result of the determination, the voice from the speaker is transmitted from the speaker to the listener. A sound field control method including drive to output and drive control to control.

Images