JP2023044750A - Sound wave output device, sound wave output method, and sound wave output program - Google Patents

Abstract

To make it possible to output, from sound wave output means, a sound wave adjusted to be easily heard by a user.SOLUTION: A sound wave output device is configured to: generate sound wave characteristic information indicating characteristics of a sound wave reflected in the interior of the ears by sound wave output means for converting an acoustic signal into a sound wave and outputting it, among sound waves heard by a user; and control characteristics of the acoustic signal based on a voice of a telephone conversion partner so that a deviation becomes smaller between frequency characteristics indicated by the sound wave characteristic information and frequency characteristics indicated by reference sound characteristic information related to a reference sound having a frequency component that the user can hear, which are characteristics of a sound wave in which the sound wave of the reference sound is reflected in the interior of the ears of the user and is set to be easily heard by the user, before being input to the sound wave output means via amplification means.SELECTED DRAWING: Figure 1

Description

The present invention relates to a sound wave output device, a sound wave output method, and a sound wave output program.

When listening to the speaker's voice without meeting face-to-face, or in an online conference, the volume and frequency of the voice differ for each speaker, so it may be difficult for the listener to hear the speaker's voice. In addition, there are individual differences in the internal functions of listeners' ears and how they perceive sounds, and the frequency components or volumes of sound waves that listeners find easy to hear differ from listener to listener. Therefore, there is a problem that it is difficult for a listener who listens to the speaker's voice without face-to-face to hear the speaker's voice in an easy-to-hear state.

Patent Literature 1 describes a communication device that identifies a speaker and makes it easier to hear the voice uttered by the identified speaker than other speakers. Specifically, the communication device described in Patent Literature 1 changes audio signal processing of voice to audio signal processing preset for each speaker, and outputs voice that has undergone the changed audio signal processing.

Japanese Patent Application Laid-Open No. 2014-233000

In the method described in Patent Document 1, it may be difficult for the listener to hear the voice processed by the voice signal processing set for each speaker. This is because, as described above, there are individual differences in the internal functions of listeners' ears and how they perceive sounds, and the frequency components or volumes of sound waves that listeners find easy to hear differ from listener to listener. Therefore, when it is difficult for the listener to hear the speech, the communication device cannot make it easier to hear the speaker's speech unless the listener changes the setting of the speech signal processing. However, with the method described in Patent Document 1, the communication device cannot output a sound wave that is adjusted to be easily audible for the listener.

An example of an object of the present invention is to provide a sound wave output device, a sound wave output method, and a sound wave output program that enable sound waves adjusted to be easily audible for the user to be output from sound wave output means.

In one aspect of the present invention, the sound wave output device stores sound wave characteristic information indicating the characteristics of the sound wave reflected inside the ear, among the sound waves heard by the user by the sound wave output means that converts the acoustic signal into a sound wave and outputs the sound wave. the frequency characteristics indicated by the sound wave characteristic information; and the frequency characteristics indicated by the reference sound characteristic information relating to the reference sound having frequency components that can be heard by the user, wherein the sound wave of the reference sound is the frequency characteristic indicated by the sound wave characteristic information. Sound waves are output through amplification means by controlling the characteristics of acoustic signals based on the voice of the other party so as to reduce the difference between the characteristics of sound waves that are reflected inside the ear and set to be easily audible to the user. and control means for inputting to the means.

In another aspect of the present invention, a sound wave output method includes a sound wave having characteristics of a sound wave reflected inside an ear, among sound waves heard by a user by a sound wave output means for converting an acoustic signal into a sound wave and outputting the sound wave. The characteristic information is generated, and the frequency characteristic indicated by the sound wave characteristic information and the frequency characteristic indicated by the reference sound characteristic information related to the reference sound having frequency components that the user can hear, wherein the sound wave of the reference sound is the frequency characteristic indicated by the reference sound characteristic information. Sound waves are output through amplification means by controlling the characteristics of acoustic signals based on the voice of the other party so as to reduce the difference between the characteristics of sound waves that are reflected inside the ear and set to be easily audible to the user. Input to means.

In another aspect of the present invention, the sound wave output program provides the computer with characteristics of sound waves reflected inside the ear among sound waves heard by the user by sound wave output means for converting acoustic signals into sound waves and outputting them. , the frequency characteristics indicated by the sound wave characteristic information, and the frequency characteristics indicated by the reference sound characteristic information regarding the reference sound having frequency components that can be heard by the user, the reference sound The sound waves are reflected inside the user's ears, and the characteristics of the acoustic signal based on the voice of the other party are controlled and amplified so that the deviation between the characteristics of the sound waves set to be easy for the user to hear and the characteristics of the sound waves is reduced. and a control function of inputting to the sound wave output means via the means.

According to the sound wave output device, the sound wave output method, and the sound wave output program of the present invention, it is possible to output sound waves adjusted to be easily audible to the user from the sound wave output means.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the structural example of the sound wave output device of 1st embodiment in this invention. It is a flow chart which shows an example of operation of a sound wave output device of a first embodiment in the present invention. It is a block diagram which shows the structural example of the sound wave output system of 2nd embodiment in this invention. FIG. 7 is a sequence diagram showing an operation example of the sound wave output system of the second embodiment of the present invention; FIG. 7 is a sequence diagram showing an operation example of the sound wave output system of the second embodiment of the present invention; 9 is a flow chart showing an operation example of an external device according to the second embodiment of the present invention; It is a flowchart which shows the operation example of the sound wave output device of 2nd embodiment in this invention. It is a flowchart which shows the operation example of the sound wave output device of 2nd embodiment in this invention. FIG. 11 is a block diagram showing a configuration example of a sound wave output system according to a third embodiment of the present invention; FIG. 11 is a sequence diagram showing an operation example of the sound wave output system of the third embodiment of the present invention; It is a flowchart which shows the operation example of the sound wave output device of 3rd embodiment in this invention. It is a figure which shows the hardware structural example of each embodiment in this invention.

[First embodiment]
A first embodiment of the present invention will be described.

FIG. 1 is a block diagram showing a configuration example of a sound wave output device 1 of this embodiment.

A sound wave output device 1 of this embodiment includes a generator 11 and a controller 12 .

For example, the sound wave output device 1 includes a sound wave output means (not shown) that converts an acoustic signal into a sound wave and outputs it, and a sound wave input means (not shown) that receives the sound wave reflected from the inside of the user's ear. and an earphone (not shown). Alternatively, the sound wave output device 1 is connected to a headphone having a sound wave output means for converting an acoustic signal into a sound wave and outputting it, and a sound wave input means for receiving the sound wave reflected from the inside of the user's ear. The sound wave output means is, for example, a speaker. The sound wave output means converts the acoustic signal into a sound wave and outputs the sound wave. The acoustic signal input to the sound wave output means is, for example, a voice signal representing the voice of a speaker (speaker) other than the user. The speaker's voice can be live voice during a call or conference, voice recorded in memory, or voice signals received over a network. The sound wave input means is, for example, a microphone.

The generation unit 11 generates sound wave characteristic information indicating characteristics of sound waves reflected inside the ear of the user, among the sound waves heard by the user by sound wave output means for converting an acoustic signal into sound waves and outputting the sound waves. Specifically, the generation unit 11 generates the sound wave characteristic information from the output of the microphone to which the sound wave reflected inside the ear is input.

The control unit 12 controls the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information regarding the reference sound having a frequency component that can be heard by the user so as to reduce the divergence between them. Control the characteristics of the audio based audio signal. The frequency characteristic indicated by the reference sound characteristic information regarding the reference sound is the characteristic of the sound wave that is set such that the sound wave of the reference sound is reflected inside the user's ear and is easily audible for the user. The control unit 12 inputs the acoustic signal whose characteristics are controlled to the sound wave output means via the amplification means (not shown).

In this way, the sound wave output device 1 generates sound wave characteristic information of sound waves reflected inside the ear among the sound waves heard by the user by the sound wave output means that converts an acoustic signal into a sound wave and outputs the sound wave. The sound wave output device 1 has the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information. The characteristic of the acoustic signal based on the voice of the other party is controlled so that the difference between the characteristic of the sound wave and the characteristic of the sound wave obtained is small. The sound wave output device 1 inputs an acoustic signal whose characteristics are controlled to the sound wave output means through the amplifier means. As a result, the sound wave output means can output, to the user, a sound wave of the sound signal whose characteristic is adjusted according to the reference sound characteristic information.

Next, an operation example of the sound wave output device 1 of this embodiment will be described with reference to FIG. FIG. 2 is a flowchart showing an operation example of the sound wave output device 1. FIG.

The generation unit 11 generates sound wave characteristic information indicating characteristics of sound waves reflected inside the ear, among the sound waves heard by the user by sound wave output means that converts sound signals into sound waves and outputs the sound waves (step S101).

The control unit 12 controls the characteristics of the acoustic signal based on the voice of the other party and inputs it to the sound wave output means via the amplification means (step S102). In step S102, the control unit 12 controls the characteristics of the acoustic signal so as to reduce the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information.

In this way, the sound wave output device 1 outputs sound waves whose characteristics are controlled so that the difference between the characteristics of the sound waves that are easy for the user to hear and the characteristics of the sound waves reflected inside the ears of the user is small. Input to means. Therefore, the sound wave output device 1 can cause the sound wave output means to output sound waves that are adjusted to be easily audible to the user.

[Second embodiment]
Next, the sound wave output device 3 according to the second embodiment of the present invention will be specifically described.

FIG. 3 is a block diagram showing a configuration example of a sound wave output system according to a second embodiment of the present invention. In a second embodiment, the sonic power system comprises a sonic power device 3 . In the second embodiment, the generator 31 corresponds to, for example, the generator 11 of the first embodiment. In the second embodiment, the control section 32, the determination section 36, and the reference sound characteristic information storage section 37 correspond to, for example, the control section 12 of the first embodiment. The second embodiment differs from the sound wave output device 1 of the first embodiment in that the sound wave output device 3 further includes a sound wave output section 33 , a reception section 34 and a sound wave input section 35 .

The sound wave output system of this embodiment is a conference communication system. Before starting the conference, the sound wave output device 3 performs initial setting for generating reference sound characteristic information indicating the characteristic of the sound wave reflected inside the ear of the user, which the user considers easy to hear. The sound wave output device 3 is connected to the external device 2 by wireless communication or wired communication, as shown in FIG. 3, when performing the initial setting described later. The external device 2 may not be connected to the sound wave output device 3 after the initial setting of the sound wave output device 3 is completed. That is, the external device 2 may be separated from the sound wave output device 3 .

Also, the sound wave output device 3 is connected to a sound wave input device (sonic wave input device 4-1 in this example) that receives voice input from a user (user A1) through wired communication or wireless communication. Further, as shown in FIG. 3, the sound wave output device 3 is a sound wave input device (in this example, a sound wave input device 4-2, . , 4-n (where n is an integer equal to or greater than 2)) by wire or wireless communication. The sonic input devices (sonic wave input devices 4-2 to 4-n in this example) that receive voice input from other users (users A2 to An) output sound waves after completing the initial settings described above. It may be connected with the device 3 .

The external device 2 initializes the sound wave output device 3 . The external device 2 is, for example, a smart phone. Alternatively, the external device 2 may be a computer. The external device 2 is preinstalled with application software that implements the functions of an input/output unit 21, a transmission unit 22, and a determination unit 23, which will be described later. The details of the initial settings using the external device 2 will be described later.

The sound wave output device 3 is, for example, headphones. Alternatively, the sound wave output device 3 may be an earphone. Thus, in this embodiment, the sound wave output device 3 is built into the headphone or earphone.

The sound wave output device 3 is used, for example, for an online conference or a conference by calling two or more people. For example, each of the participants in the online conference uses a sound wave output device that is the same as the sound wave output device 3 of user A1 and a sound wave input device that receives voice input. In this example, the user A1 of the sound wave output device 3 among the participants of the online conference inputs voice using the sound wave input device 4-1. Further, each of the other users (user A2 to user An) inputs voice using any one of the sound wave input devices 4-2 to 4-n. The receiving unit 34 of the sound wave output device 3 receives a sound signal corresponding to a sound from a sound wave input device (each of the sound wave input devices 4-2 to 4-n in this example) that receives voice input from another user. receive a signal.

Note that the receiving unit 34 may receive an audio signal corresponding to the voice from the sonic input device (sonic wave input device 4-1 in this example) that receives the voice input of the user A1. The receiving unit 34 may transmit the voice signal of the user A1 to the sound wave output devices of other users (from the user A2 to the user An). In this embodiment, the sound wave input device 4-1 transmits audio signals to sound wave output devices of other users (users A2 to An).

The sound wave output device 3 of this embodiment performs the initial setting described above. In the initial setting, the control unit 32 causes the sound wave output unit 33 to output the reference sound based on the output request notification requesting the output of the reference sound for the initial setting from the external device 2 . The reference sound is a sound wave that is used by the user to determine whether or not the sound is easy to hear. Also, the reference sound is a plurality of types of sound waves having different frequencies and volumes. For each of the reference sounds, the sound wave input unit 35 receives the sound wave output from the sound wave output unit 33 to the inside of the ear and reflected inside the ear, and outputs an input acoustic signal representing the input sound wave to the generation unit 31. do. In the initial setting, the receiving unit 34 receives evaluation information indicating whether or not each of the reference sounds is easy to hear from the external device 2 . The evaluation information includes information indicating the result of determination by the user as to whether or not the reference sound is easy to hear. The generating unit 31 (to be described later) causes the reference sound characteristic information storage unit 37 to store the evaluation information and the reference sound characteristic information. The reference sound characteristic information may include evaluation information. The reference sound characteristic information includes output sound information indicating the frequency characteristics of the sound signal when each of the reference sounds was output, and reflected sound wave characteristic information indicating the frequency characteristics of the input sound signal corresponding to each of the reference sounds. included. In the initial setting, the sound wave output device 3 causes the reference sound characteristic information storage unit 37 to store reference sound characteristic information including frequency information indicating the frequency characteristics of the sound waves reflected inside the ear of the user. When used in an online conference, the sound wave output device 3 controls the characteristics of the sound signal.

Further, when the initial setting is completed and the online conference is started, the reception unit 34 of the sound wave output device 3 receives the voice signal, which is an acoustic signal based on the voice of the other user A2 to the other user An. Signals are received from sonic input devices 4-2 to 4-n. The acoustic signal is input to the sound wave output section 33 via the control section 32 . Immediately after the start of the online conference, the control unit 32 still controls the characteristics of the acoustic signal so that the difference between the characteristics of the sound waves that the user A1 finds easy to hear and the characteristics of the sound waves reflected inside the ears of the user is small. not The sound wave output unit 33 converts the sound signal input from the control unit 32 into a sound wave and outputs the sound wave to the ear of the user A1. The sound wave input unit 35 receives the sound wave of the other party, which is output from the sound wave output unit 33 to the inside of the ear and reflected inside the ear, and outputs the sound wave to the generation unit 31 . The generation unit 31 generates sound wave characteristic information indicating the characteristics of the sound wave reflected inside the ear of the sound wave output by the sound wave output unit 33 to the inside of the ear of the user A1. Next, the sound wave output device 3 uses the determination section 36, the reference sound characteristic information storage section 37, and the control section 32 to perform the following processing. The process is based on the difference between the frequency characteristic indicated by the sound wave characteristic information and the characteristic of the sound wave reflected inside the user's ear, which is the frequency characteristic indicated by the reference sound characteristic information and which is easy for the user A1 to hear. This is the process of controlling the characteristics of the acoustic signal so that the Details of the processing will be described later. The control unit 32 inputs the acoustic signal whose characteristics are controlled to the sound wave output unit 33 via an amplifier (not shown).

[Configuration of sound wave input device]
The sound wave input device 4-1 includes a sound wave input section 41-1 and a transmission section 42-1. The sound wave input device 4-2 includes a sound wave input section 41-2 and a transmission section 42-2. The sound wave input device 4-n includes a sound wave input section 41-n and a transmission section 42-n. Thus, each of the sound wave input devices 4-1 to 4-n includes a sound wave input section and a transmission section.

The sonic input devices 4-1 to 4-n are smart phones, for example. Alternatively, the sonic input device may be built into the external device. For example, the sonic input device 4-1 may be incorporated in the external device 2 used by the user A1. Specifically, the external device 2 further includes the function of a sound wave input unit 41-1, which will be described later, and the transmission unit 22, which will be described later, realizes the function of the transmission unit 42-1 of the sound wave input device 4-1, which will be described later. may

Each of the sound wave input devices 4-1 to 4-n receives voice input from participants (users A1 to An) of the online conference and outputs voice signals corresponding to the voices.

Here, the configurations of the sound wave input devices 4-1 to 4-n of the present embodiment will be described in detail with the sound wave input device 4-1 as a representative.

The sound wave input unit 41-1 receives the voice input of the user A1 and generates a voice signal corresponding to the voice. The sound wave input unit 41-1 outputs the generated audio signal to the transmission unit 42-1.

The voice signal of the user A1 is input from the sound wave input section 41-1 to the transmission section 42-1. The transmission unit 42-1 transmits the input audio signal to a sound wave output device (in this example, another sound wave output device other than the sound wave output device 3) used by other users participating in the online conference (Fig. not shown)). The sound wave input device 4-1 that receives the voice input of the user A1 does not output voice signals to the sound wave output device 3 used by the user A1.

[Configuration of external device 2]
The configuration of the external device 2 of this embodiment will be described in detail with reference to FIG. The external device 2 includes an input/output unit 21 , a transmission unit 22 and a determination unit 23 .

The input/output unit 21 receives an input of an instruction to start initial setting in accordance with a user's operation. The input/output unit 21 outputs an instruction to start the initialization to the transmission unit 22 .

An instruction to start initialization is input from the input/output unit 21 to the transmission unit 22 . When the instruction to start the initialization is input, the transmission unit 22 transmits an output request notification requesting the output of the sound wave of the reference sound of the initial value to the sound wave output device 3 for the initialization. The reference sounds are multiple types of sound waves having different frequencies and volumes.

Frequency information about the sound wave of the reference sound and volume information of the reference sound are input from the transmission unit 22 to the determination unit 23 . Frequency information is information indicating one or more frequencies. The frequency information and the volume information are characteristic information of the reference sound to which the evaluation information is transmitted, which is input by the transmission unit 22 when the evaluation information (to be described later) is transmitted to the sound wave output device 3 . When frequency information and volume information are input, the determination unit 23 determines whether or not evaluation information has been generated for all reference sounds. When determining that the evaluation information has been generated for all the reference sounds, the determination unit 23 outputs an initial setting end notification to the transmission unit 22 . Further, if there is a reference sound for which evaluation information is not generated among all the sound waves of the reference sounds, the determination unit 23 associates the frequency information and volume information of the reference sound for which evaluation information is not generated, and outputs them to the transmission unit 22. .

Frequency information and volume information are input from the determination unit 23 to the transmission unit 22 . The transmission unit 22 transmits an output request notification for initial setting, including the input frequency information and volume information, to the sound wave output device 3 . As a result, the external device 2 causes the sound wave output unit 33 of the sound wave output device 3 to output the sound wave of the reference sound.

In addition, the determination unit 23 inputs an initial setting end notification to the transmission unit 22 . The transmission unit 22 transmits a notification of completion of initialization to the sound wave output device 3 .

Note that the input/output unit 21 may receive an input of the reference sound characteristic information according to the user's operation when an instruction to start the initial setting is input. When the reference sound characteristic information is input to the input/output unit 21 , the transmission unit 22 may transmit the reference sound characteristic information to the sound wave output device 3 .

By transmitting the output request notification, the sound wave output unit 33 of the sound wave output device 3 outputs the sound wave of the reference sound into the ear of the user A1. The input/output unit 21 of the external device 2 receives, for each sound wave of a plurality of reference sounds, an operation input regarding an evaluation indicating whether or not the reference sound is easy to hear in accordance with a user's operation, and responds to the operation input. Generate evaluation information for The evaluation information may include the following information. The evaluation information may include information indicating whether the pitch of the reference sound output from the sound wave output unit 33 is easy for the user to hear. Further, when the evaluation information is information indicating that it is difficult to hear, the input/output unit 21 may include information indicating that the volume of the sound wave is loud or information indicating that the volume is low in the evaluation information. The input/output unit 21 outputs the generated evaluation information to the transmission unit 22 .

The transmitter 22 transmits the generated evaluation information to the sound wave output device 3 . Further, when the evaluation information is transmitted to the sound wave output device 3, the transmission unit 22 associates the frequency information and the volume information included in the output request notification transmitted immediately before the evaluation information is generated, and transmits the information to the determination unit 23. Output.

[Structure of sound wave output device 3]
Next, the configuration of the sound wave output device 3 of this embodiment will be described in detail with reference to FIG. The sound wave output device 3 includes a generator 31 and a controller 32 .

The sound wave output section 33 is connected to the control section 32 . The receiver 34 is connected to the generator 31 and the controller 32 . The sound wave input section 35 is connected to the generation section 31 . The determination unit 36 is connected to the generation unit 31 , the control unit 32 and the reference sound characteristic information storage unit 37 . The reference sound characteristic information storage unit 37 is connected to the generation unit 31 and the determination unit 36 .

The sound wave output device 3 may be realized as two or more devices. For example, among the components of the sound wave output device 3, the sound wave output unit 33 and the sound wave input unit 35 may be earphones or headphones, and the other components may be implemented as an information processing device. Further, in the earphone or headphone, the left ear sound wave input section 351 and the left ear sound wave output section 331 are integrally mounted, and the right ear sound wave input section 352 and the right ear sound wave output section 332 are integrally mounted. good.

First, each configuration of the sound wave output device 3 of the present embodiment will be described in detail for initial setting.

The receiving unit 34 receives an output request notification for initial setting, including frequency information and volume information. The receiving unit 34 associates the frequency information with the volume information and outputs them to the control unit 32 .

The control unit 32 inputs an acoustic signal for outputting a sound wave of a reference sound having the frequency indicated by the input frequency information and the volume indicated by the input volume information to the sound wave output unit 33 via the amplifying means. .

The sound wave output section 33 includes a left ear sound wave output section 331 and a right ear sound wave output section 332 .

An acoustic signal for outputting a reference sound wave is input from the control unit 32 to the left ear sound wave output unit 331 . The left ear sound wave output unit 331 converts an acoustic signal into a sound wave and outputs the sound wave to the left ear of the user. For example, the left ear sound wave output unit 331 is a speaker for the left ear that converts electrical signals into sound waves.

An acoustic signal for outputting the sound wave of the reference sound is input from the control unit 32 to the right ear sound wave output unit 332 . The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave to the inside of the user's right ear. For example, the right ear sound wave output unit 332 is a right ear speaker.

The sound wave input section 35 includes a left ear sound wave input section 351 and a right ear sound wave input section 352 .

The left ear sound wave input unit 351 inputs sound waves reflected inside the user's left ear, and outputs an input acoustic signal representing the input sound waves to the generation unit 31 . For example, the input acoustic signal is a signal representing the waveform of sound waves reflected inside the ear. For example, the left ear sound wave input unit 351 is a left ear microphone. Specifically, the left ear sound wave input unit 351 is a bone conduction microphone for the left ear.

The right ear sound wave input unit 352 inputs sound waves reflected inside the user's right ear, and outputs an input acoustic signal representing the input sound waves to the generation unit 31 . For example, the right ear sound wave input unit 352 is a microphone for the right ear. Specifically, the right ear sound wave input unit 352 is a bone conduction microphone for the right ear.

In the initial setting, an input acoustic signal related to the acoustic signal of the reference sound, which is generated by the control unit 32 based on the frequency information and volume information transmitted from the transmission unit 22, is input from the sound wave input unit 35 to the generation unit 31. . The generator 31 analyzes the strength of the signal of each frequency component forming the input acoustic signal. The generation unit 31 performs frequency decomposition of the input acoustic signal to generate reflected sound wave characteristic information indicating frequency characteristics including information indicating frequency distribution or information indicating frequency components according to the analysis result. The reflected sound wave characteristic information is frequency information indicating the frequency characteristic of the sound wave in which the reference sound is reflected inside the user's ear. The generator 31 outputs sound wave characteristic information including frequency information. For example, the sound wave characteristic information includes frequency information indicating at least one frequency. The generator 31 also detects the volume at that frequency based on the amplitude intensity of the input acoustic signal. Therefore, the reflected sound wave characteristic information of the present embodiment includes sound volume information indicating the sound volume of the sound wave indicated by the input acoustic signal.

Note that the left ear sound wave input section 351 and the right ear sound wave input section 352 may generate the reflected sound wave characteristic information. When the left ear sound wave input unit 351 and the right ear sound wave input unit 352 generate the reflected sound wave characteristic information, the generator 31 does not generate the reflected sound wave characteristic information, but the left ear sound wave input unit 351 and the right ear sound wave input unit 351 generate the reflected sound wave characteristic information. Sound wave characteristic information may be acquired from each of the input units 352 .

Also, the receiving unit 34 receives the evaluation information. The receiving unit 34 associates the evaluation information with the frequency information and the volume information included in the output request notification received before receiving the evaluation information, and outputs them to the generating unit 31 .

The frequency information, volume information, and evaluation information are input from the receiver 34 to the generator 31 . The frequency information and volume information input from the receiving unit 34 are output acoustic information indicating frequency characteristics of acoustic signals when each of the reference sounds is output.

Further, the receiving unit 34 receives a notification of completion of initialization from the external device 2 . Upon receiving the initialization end notification, the reception unit 34 outputs the initialization end notification to the generation unit 31 .

When the initial setting completion notification is input, the generation unit 31 performs the following operations. The generation unit 31 generates output acoustic information indicating the frequency characteristics of the acoustic signal when each of the reference sounds is output, and reflected sound wave characteristic information indicating the frequency characteristics of the input acoustic signal corresponding to each of the reference sounds. It is stored in the reference sound characteristic information storage unit 37 as characteristic information. In this embodiment, the generator 31 associates the reference sound characteristic information with the evaluation information and stores them in the reference sound characteristic information storage unit 37 .

The generating unit 31 stores the evaluation information and the reference sound characteristic information in association with each other in the reference sound characteristic information storage unit 37 . The reference sound characteristic information storage unit 37 stores at least reference sound characteristic information associated with evaluation information indicating ease of hearing. Specifically, the reference sound characteristic information storage unit 37 of the present embodiment stores reference sound characteristic information associated with evaluation information indicating ease of hearing and reference sound characteristic information associated with evaluation information indicating difficulty of hearing. are stored. The reference sound characteristic information includes output sound information indicating frequency characteristics of sound signals when each of the reference sounds is output. Further, the reference sound characteristic information includes reflected sound wave characteristic information indicating the frequency characteristic of the input acoustic signal corresponding to each reference sound.

Note that the reference sound characteristic information for the left ear and the reference sound characteristic information for the right ear may be stored in the reference sound characteristic information storage unit 37 . When the reference sound characteristic information for the left ear and the reference sound characteristic information for the right ear are stored in the reference sound characteristic information storage unit 37, the control unit 32 may operate as follows. The control unit 32 controls the acoustic signal characteristics of the left ear sound wave output unit 331 based on the reference sound characteristic information for the left ear, and controls the acoustic signal characteristics of the right ear sound wave output unit 332 based on the reference sound characteristic information for the right ear. You may control based on sound characteristic information. In this way, the sound wave output device 3 may individually control the left ear sound wave output section 331 and the right ear sound wave output section 332 when performing initial settings and when used in an online conference.

Next, each configuration of the sound wave output device 3 of this embodiment will be described in detail for use in an online conference.

The receiving unit 34 receives audio signals from sound wave input devices (sound wave input devices 4-2 to 4-n in this example). The receiver 34 outputs the audio signal to the controller 32 .

An audio signal is input from the receiver 34 to the controller 32 . In addition, a notification requesting control of the characteristics of the acoustic signal is input from the determination unit 36 to the control unit 32 . When the notification requesting control of the characteristics of the acoustic signal is not input, the control unit 32 inputs an acoustic signal for outputting a sound wave based on the audio signal to the sound wave output unit 33 via the amplifying means. When the notification requesting control of the characteristics of the acoustic signal is input, the control unit 32 controls the characteristics of the acoustic signal to be output with respect to the audio signal from the receiving unit 34 . Specifically, the control unit 32 converts the audio signal into a signal or data in the frequency domain via a Fourier transform circuit. The control unit 32 determines the difference between the frequency characteristics of the signal or data in the frequency domain and the characteristics of the sound wave reflected inside the ear of the user, which is the frequency characteristic indicated by the reference sound characteristic information and which is easy for the user to hear. change the characteristics of the acoustic signal based on the caller's voice so that the For example, changing the volume, which is the frequency and its magnitude. After that, the control unit 32 restores the time-axis audio signal by the inverse Fourier transform circuit for the frequency domain signal or data whose characteristics are controlled by the change. Then, the control unit 32 inputs an acoustic signal, which is an audio signal whose characteristics are controlled, to the sound wave output unit 33 via the amplifying means.

For example, when sound waves with frequency characteristics that are difficult for the user to hear are being output from the sound wave output unit 33 , a notification requesting control of the characteristics of the sound signal is input from the determination unit 36 . The control unit 32 adjusts the reception unit 34 so that the difference between the frequency characteristics indicated by the reference sound characteristic information and the characteristics of the sound waves that are easy for the user to hear and the characteristics of the sound waves reflected inside the ears of the user is small. Controls the characteristics of the audio signal input from the . Specifically, the control unit 32 controls the acoustic signal so that the deviation of the frequency characteristics of the sound wave characteristic information from the frequency characteristics of the reference sound characteristic information becomes small. For example, as the frequency characteristics of the sound wave characteristic information, if the strength of the signal of a certain frequency component is weaker than the signal strength of the frequency component of the sound wave reflected inside the user's ear that the user can easily hear. , the control unit 32 controls the characteristics of the acoustic signal as follows. The control unit 32 controls the characteristics of the acoustic signal so as to increase the strength of the signal of a certain frequency component.

The sound wave output unit 33 converts the acoustic signal input from the control unit 32 into a sound wave and outputs the sound wave.

The sound wave input unit 35 receives a sound wave reflected inside the user's ear and outputs an input acoustic signal representing the input sound wave to the generation unit 31 .

The generation unit 31 generates sound wave characteristic information according to the analysis result of the signal strength of each frequency component forming the input acoustic signal input from the sound wave input unit 35 . The generation unit 31 outputs the generated sound wave characteristic information to the determination unit 36 .

The determination unit 36 determines whether sound waves that are easy for the user to hear are being output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 based on the input sound wave characteristic information. The determining unit 36 determines the frequency characteristic indicated by the reflected sound wave characteristic information included in the reference sound characteristic information stored in the reference sound characteristic information storage unit 37, which is easy for the user to hear, and the frequency characteristic indicated by the sound wave characteristic information. If there is a deviation of more than the threshold of , the determination is made as follows. The determination unit 36 determines that the sound wave output unit 33 outputs sound waves that are difficult for the user to hear. When determining that the sound wave output unit 33 outputs sound waves that are difficult for the user to hear, the determination unit 36 performs the following operation. The determining unit 36 determines that the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear are output associated with the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 in association with the evaluation information indicating that the sound is easy to hear. Identify acoustic information. The determination unit 36 outputs to the control unit 32 a notification requesting control of the characteristics of the acoustic signal. The notification requesting control of the characteristics of the acoustic signal includes the identified output acoustic information.

In addition, the determination unit 36 determines the frequency characteristics indicated by the reflected sound wave characteristic information included in the reference sound characteristic information stored in the reference sound characteristic information storage unit 37, which is easy for the user to hear, and the frequency characteristics indicated by the sound wave characteristic information. is less than a predetermined threshold, the following determination is made. The determination unit 36 determines that the sound wave output unit 33 outputs sound waves that are easy for the user to hear. When it is determined that sound waves that are easy for the user to hear are being output by the sound wave output unit 33, the process of outputting a notification requesting control of the characteristics of the acoustic signal to the control unit 32 is not performed.

As described above, when a notification requesting control of characteristics of an acoustic signal is input, the control unit 32 controls the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information so as to reduce the divergence. Then, the characteristics of the acoustic signal are controlled and input to the sound wave output section 33 via the amplifying means. Specifically, the control unit 32 controls the characteristics of the acoustic signal so as to reduce the deviation from the frequency characteristics indicated by the output acoustic information included in the notification requesting the control of the characteristics of the acoustic signal.

Note that the sound wave output device 3 sends a notification requesting control of characteristics of an acoustic signal including two or more pieces of output acoustic information for the sound wave characteristic information of the left ear sound wave output unit 331 and the right ear sound wave output unit 332. 32 may be output. Based on the frequency characteristics of each of the plurality of output acoustic information, the control unit 32 controls the acoustic signal so that the frequency characteristics of the acoustic signal correspond to the frequency characteristics of the sound wave characteristic information and the frequency characteristics of the reference sound characteristic information. Signals may be controlled.

As described above, the sound wave output device 3 outputs the sound wave characteristic information indicating the characteristics of the sound wave reflected inside the ear among the sound waves heard by the user by the sound wave output unit 33 which converts the acoustic signal into a sound wave and outputs the sound wave. Generate. The sound wave output device 3 has the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information. The characteristic of the acoustic signal based on the voice of the other party is controlled so that the difference between the characteristic of the sound wave and the characteristic of the sound wave obtained is small. The sound wave output device 3 also inputs the acoustic signal whose characteristics are controlled to the sound wave output unit 33 via the amplifying means. As a result, the sound wave output unit 33 can output, to the user, a sound wave based on the sound signal whose characteristic of the sound signal is adjusted according to the reference sound characteristic information. As a result, the sound wave output device 3 can cause the sound wave output unit 33 to output sound waves that are adjusted to be easily audible to the user.

[motion]
Next, an operation example of the sound wave output system of this embodiment will be described with reference to FIGS. 4 to 8. FIG.

4 and 5 are sequence diagrams showing an operation example of the sound wave output system. FIG. 6 is a flow chart showing an operation example of the external device 2 . 7 and 8 are flow charts showing an operation example of the sound wave output device 3. FIG.

First, with reference to FIG. 4, the operation of the external device 2 for initializing the sound wave output device 3 will be described together with the operation of the sound wave output device 3. FIG.

The input/output unit 21 of the external device 2 receives an instruction to start initial setting according to the user's operation (step S201).

An instruction to start initialization is input from the input/output unit 21 to the transmission unit 22 . The transmitting unit 22 sends an output request requesting output of a reference sound for initial setting, which includes frequency information indicating an initial frequency and volume information indicating an initial volume, among sound waves having different characteristics to be evaluated. A notification is sent to the sound wave output device 3 (step S202).

The receiving unit 34 of the sound wave output device 3 receives an output request notification for initial setting, including frequency information and volume information. The receiving unit 34 associates the frequency information with the volume information and outputs them to the control unit 32 .

Frequency information and volume information are input from the receiving unit 34 to the control unit 32 . The control unit 32 outputs acoustic signals for outputting sound waves having the frequency indicated by the input frequency information and the volume indicated by the input volume information to the left ear sound wave output unit 331 and the right ear sound wave output unit 332. to enter.

An acoustic signal for outputting a reference sound wave is input from the control unit 32 to the left ear sound wave output unit 331 . Further, an acoustic signal for outputting the sound wave of the reference sound is input from the control unit 32 to the right ear sound wave output unit 332 . The left ear sound wave output unit 331 converts an acoustic signal into a sound wave and outputs the sound wave to the left ear of the user. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave to the inside of the user's right ear (step S203). As a result, the user can listen to the sound wave of the reference sound based on the output request notification.

The left ear sound wave input unit 351 receives a sound wave reflected inside the left ear and outputs an input acoustic signal representing the received sound wave to the generation unit 31 . Further, the right ear sound wave input unit 352 receives the sound wave reflected inside the right ear, and outputs an input acoustic signal representing the input sound wave to the generation unit 31 (step S204).

An input acoustic signal is input from the left ear sound wave input unit 351 to the generation unit 31 . An input acoustic signal is input from the right ear sound wave input unit 352 to the generation unit 31 . The generation unit 31 analyzes the signal strength of each frequency component forming the input acoustic signal (step S205). The generation unit 31 generates reflected sound wave characteristic information indicating frequency characteristics according to the analysis result.

The input/output unit 21 of the external device 2 receives an operation input regarding an evaluation indicating whether or not it is easy to hear according to the user's operation, and generates evaluation information corresponding to the operation input (step S206). The input/output unit 21 outputs the generated evaluation information to the transmission unit 22 .

The transmitter 22 transmits the evaluation information generated in step S206 to the sound wave output device 3 (step S207). The transmitting unit 22 associates the frequency information and the volume information included in the output request notification transmitted immediately before the evaluation information is generated, and outputs them to the determining unit 23 .

The operations of steps S206 and S207 are performed in parallel with the operations of steps S204 and S205. The order in which the operations from step S204 to step S207 are performed is not limited to the order shown in FIG.

The receiver 34 of the sound wave output device 3 receives the evaluation information. The receiving unit 34 associates the evaluation information with the frequency information and the volume information included in the output request notification received before receiving the evaluation information, and outputs them to the generating unit 31 .

The frequency information, volume information, and evaluation information are input from the receiver 34 to the generator 31 . The frequency information and volume information input from the receiving unit 34 are output acoustic information indicating frequency characteristics of acoustic signals when each of the reference sounds is output.

The determination unit 23 of the external device 2 determines whether evaluation information has been generated by the input/output unit 21 for all reference sounds (step S208). When the evaluation information has been generated for all the reference sounds, the determination unit 23 outputs an initial setting end notification to the transmission unit 22 . When the initialization end notification is input, the transmission unit 22 transmits the initialization end notification to the sound wave output device 3 .

The receiving unit 34 of the sound wave output device 3 receives the initialization end notification from the external device 2 . Upon receiving the initialization end notification, the reception unit 34 outputs the initialization end notification to the generation unit 31 .

When the initial setting completion notification is input, the generation unit 31 associates the evaluation information with the reference sound characteristic information for each reference sound and stores them in the reference sound characteristic information storage unit 37 . The reference sound characteristic information includes output sound information indicating the frequency characteristics of the sound signal when each of the reference sounds was output, and reflected sound wave characteristic information indicating the frequency characteristics of the input sound signal corresponding to each of the reference sounds. included.

In step S208, if there is a reference sound for which evaluation information is not generated among all the reference sounds, the determining unit 23 of the external device 2 associates the frequency information and volume information of the reference sound for which evaluation information is not generated. Output to the transmission unit 22 . When the frequency information and volume information are input, the transmission unit 22 of the external device 2 performs the operation of step S202. That is, the transmitting unit 22 transmits to the sound wave output device 3 a reference sound output request notification for initial setting, which includes the input frequency information and volume information. Then, the external device 2 performs the operations of steps S206, S207, and S208. Further, the sound wave output device 3 performs the operations from step S203 to step S205 until receiving the notification of completion of the initial setting.

Next, referring to FIG. 5, for the audio signal received from the sound wave input device (sound wave input devices 4-2 to 4-n in this example), the characteristic of the sound signal is controlled and the sound wave is output to the sound wave output unit 33. The operation input by the device 3 will be described.

The sound wave output device 3 of the present embodiment generates sound wave characteristic information of the sound wave reflected inside the ear among the previously output sound waves during the online conference. Then, the sound wave output device 3 controls the characteristics of the acoustic signal to be output to the sound wave output section 33 next.

Note that in FIG. 5, before the sound wave output device 3 receives the sound signal transmitted from the sound wave input devices 4-2 to 4-n in step S302 described later, the notification requesting control of the characteristics of the sound signal is sent as a sound wave. A case where no data is input to the control unit 32 of the output device 3 will be described as an example.

Each of the sound wave input devices 4-2 to 4-n receives voice input from another user (step S301). Each of the sound wave input devices 4-2 to 4-n that receives the voice input generates a voice signal corresponding to the voice. Each of the sound wave input devices 4-2 to 4-n that generated the audio signal is a sound wave output device (in this example, the sound wave output device 3 is (step S302).

The receiving unit 34 of the sound wave output device 3 receives audio signals from the sound wave input devices (the sound wave input devices 4-2 to 4-n in this example). The receiver 34 outputs the audio signal to the controller 32 .

An audio signal is input from the receiver 34 to the controller 32 . The control unit 32 inputs an acoustic signal for outputting a sound wave based on the audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplifying means.

The left ear sound wave output unit 331 converts an acoustic signal into a sound wave and outputs the sound wave to the left ear of the user. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave to the inside of the user's right ear (step S303).

The left ear sound wave input unit 351 inputs sound waves reflected inside the user's left ear, and outputs an input acoustic signal representing the input sound waves to the generation unit 31 . The right ear sound wave input unit 352 receives the sound wave reflected inside the user's right ear, and outputs an input acoustic signal representing the input sound wave to the generation unit 31 (step S304).

Input acoustic signals are input to the generation unit 31 from the left ear sound wave input unit 351 and the right ear sound wave input unit 352 . The generator 31 analyzes the strength of the signal of each frequency component forming the input acoustic signal (step S305). The generation unit 31 generates sound wave characteristic information according to the analysis result. The generation unit 31 determines the sound wave characteristic information generated from the input sound signal input from the left ear sound wave input unit 351 and the sound wave characteristic information generated from the input sound signal input from the right ear sound wave input unit 352 to the determination unit 36 . output to

Based on the input sound wave characteristic information, the determination unit 36 determines whether sound waves that are easy for the user to hear are being output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 (step S306). ). Specifically, the determination unit 36 determines the frequency characteristic indicated by the reflected sound wave characteristic information included in the reference sound characteristic information that is easy for the user to hear, stored in the reference sound characteristic information storage unit 37, and the frequency characteristic indicated by the sound wave characteristic information. It is determined whether or not the frequency characteristic diverges by a predetermined threshold or more. If the difference is equal to or greater than a predetermined threshold, the sound wave corresponding to the input sound wave characteristic information is a sound wave that is difficult for the user to hear. If the difference is less than the predetermined threshold, the sound wave is easy for the user to hear.

If the determination unit 36 determines in step S306 that the sound wave output unit 33 is outputting a sound wave that is difficult for the user to hear based on the input sound wave characteristic information, the determination unit 36 performs the following operation. The determining unit 36 determines that the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear are output associated with the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 in association with the evaluation information indicating that the sound is easy to hear. Identify acoustic information. Alternatively, the determination unit 36 calculates the similarity between the frequency characteristics indicated by the input sound wave characteristic information and the frequency characteristics indicated by the reflected sound wave characteristic information associated with the evaluation information indicating ease of hearing, and the similarity is set to a threshold value. Higher reflected sound wave characteristic information may be specified. Further, the determination unit 36 may specify output sound information stored in the reference sound property information storage unit 37 in association with the specified reflected sound wave property information.

The determination unit 36 outputs to the control unit 32 a notification requesting control of the characteristics of the acoustic signal, including the specified output acoustic information.

Further, if the determination unit 36 determines in step S306 that sound waves that are easy for the user to hear are being output by the sound wave output unit 33 based on the input sound wave characteristic information, the determination unit 36 does not specify the output sound information. . Further, the determination unit 36 does not perform processing for outputting a notification requesting control of the characteristics of the acoustic signal to the control unit 32 .

An audio signal is input from the receiver 34 to the controller 32 . If a notification requesting control of the characteristics of the acoustic signal is not input, the control unit 32 generates an acoustic signal for outputting sound waves based on the audio signal. On the other hand, when a notification requesting control of the characteristics of the acoustic signal is input from the determination unit 36 , the control unit 32 controls the characteristics of the acoustic signal to be output with respect to the audio signal from the reception unit 34 . Specifically, the control unit 32 controls the characteristics of the acoustic signal input to the sound wave output unit 33 so as to reduce the divergence from the output acoustic information included in the notification requesting control of the characteristics of the acoustic signal. As a result, the control unit 32 generates an acoustic signal for causing the sound wave output unit 33 to output a sound wave based on the audio signal in which the characteristics of the acoustic signal are controlled (step S307). The control unit 32 inputs the acoustic signal generated in step S307 to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplifying means. In this way, the control unit 32 allows the reflected sound wave characteristic information, which is the frequency characteristic indicated by the reference sound characteristic information, to be easily audible by the user, and the characteristics of the sound wave reflected inside the ear of the user. The characteristics of the acoustic signal are controlled so that the divergence becomes small.

The output acoustic information included in the notification requesting the control of the characteristics of the acoustic signal indicates the frequency characteristics of the acoustic signal when the reference sound that is easily audible to the user is output. When controlling the characteristics of the acoustic signal so as to reduce the divergence from the frequency characteristics of the output acoustic information, the frequency characteristics of the sound wave characteristic information in the input acoustic signal of the acoustic signal whose characteristics are controlled are the frequency characteristics of the reflected sound wave characteristic information. is smaller than the frequency characteristic of the sound wave characteristic information of the acoustic signal whose characteristic is not controlled.

Acoustic signals are input from the control unit 32 to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 . The left ear sound wave output unit 331 and the right ear sound wave output unit 332 convert the acoustic signal into sound waves and output them (step S308).

Note that when the receiving unit 34 receives an audio signal after step S303, the acoustic signal generated in step S307 is an acoustic signal for outputting a sound wave based on the audio signal received after step S303.

Moreover, when the sound wave based on the audio|voice signal transmitted in step S302 is output at the time of the start of step S307, the control part 32 may perform the following operation|movement in step S307. The control unit 32 may generate an acoustic signal for outputting a sound wave to be output from among the sound waves based on the audio signal transmitted in step S302.

Next, an operation example of the external device 2 will be described with reference to FIG. Further, the operation of FIG. 6 details the operations of steps S201, S202, S206, S207 and S208 of FIG.

The input/output unit 21 receives an instruction to start initial setting according to the user's operation (step S401). The input/output unit 21 outputs an instruction to start the initialization to the transmission unit 22 .

An instruction to start initialization is input from the input/output unit 21 to the transmission unit 22 . When an instruction to start initial setting is input, the transmission unit 22 performs the following operations. The transmitting unit 22 transmits to the sound wave output device 3 an output request notification requesting output of a reference sound for initial setting, which includes frequency information and volume information (step S402). For example, the output request notification that is first output at the time of initial setting includes frequency information indicating the initial frequency and volume information indicating the initial volume of the reference sound.

The input/output unit 21 receives an operation input regarding an evaluation indicating whether or not the sound is easy to hear in accordance with the user's operation. If an operation input regarding the evaluation indicating easy-to-listen is received (step S<b>403 , YES), the input/output unit 21 generates evaluation information indicating easy-to-listen, and outputs the generated evaluation information to the transmission unit 22 . Further, when an operation input regarding an evaluation indicating difficulty in hearing is received (step S403, NO), the input/output unit 21 generates evaluation information indicating difficulty in hearing, and outputs the generated evaluation information to the transmission unit 22. do.

When the evaluation information indicating that the sound is easy to hear is input, the transmitter 22 transmits the evaluation information indicating that the sound is easy to hear to the sound wave output device 3 (step S404). When the evaluation information indicating that it is difficult to hear is input, the transmission unit 22 does not perform the operation of step S404. The transmitter 22 transmits the evaluation information indicating the difficulty of hearing to the sound wave output device 3 (step S405).

The transmission unit 22 that has performed the operation of step S404 or step S405 associates the frequency information and volume information included in the output request notification transmitted in step S402 and outputs them to the determination unit 23 .

Frequency information and volume information are input from the transmission unit 22 to the determination unit 23 . The determination unit 23 determines whether or not evaluation information has been generated for reference sounds of all volumes to be evaluated in reference sounds of frequencies indicated in the input frequency information (step S406). Here, the volume to be evaluated is the volume preset as the volume to be evaluated for the reference sound of a predetermined frequency. For example, the external device 2 causes the sound wave output device 3 to output a reference sound of a predetermined frequency with different volumes at predetermined intervals from the minimum volume for evaluation to the maximum volume for evaluation. Also, the external device may set the volume of the reference sound to be evaluated for each frequency based on the A characteristic, which is the frequency weighting characteristic.

If there is a reference sound with a volume for which evaluation information is not generated among the reference sounds with frequencies indicated in the input frequency information (step S406, NO), the determination unit 23 performs the following operations. The determination unit 23 associates the frequency information and volume information of the reference sound to be evaluated next, and outputs them to the transmission unit 22 (step S407). Specifically, in step S<b>407 , the determination unit 23 associates the frequency information input from the transmission unit 22 with the volume information indicating the volume for which the evaluation information is not generated, and outputs them to the transmission unit 22 .

Frequency information and volume information are input from the determination unit 23 to the transmission unit 22 . In step S402, the transmission unit 22 transmits to the sound wave output device 3 an output request notification including the frequency information and volume information output in step S407. The external device 2 performs the operations from step S402 to step S407 until evaluation information is generated for the reference sounds of all volumes to be evaluated in the reference sounds of the frequencies indicated by the predetermined frequency information.

When the evaluation information is generated for the reference sounds with the frequencies indicated in the input frequency information and the reference sounds with all volumes to be evaluated (step S406, YES), the determination unit 23 performs the following operation. The determination unit 23 checks whether evaluation information has been generated for the reference sounds of all frequencies to be evaluated (step S408). Here, the frequency for evaluation is a frequency preset as a frequency for evaluation. For example, the external device 2 causes the sound wave output device 3 to output reference sounds of different frequencies at predetermined intervals from the minimum frequency for evaluation to the maximum frequency for evaluation.

If there is a reference sound with a frequency for which evaluation information has not been generated (step S408, NO), the determination unit 23 performs the following operation. The determination unit 23 associates the frequency information of the sound wave to be evaluated next with the volume information, and outputs them to the transmission unit 22 (step S409). Specifically, in step S409, the determination unit 23 associates the frequency information indicating the frequency to be evaluated next for which no evaluation information has been generated with the volume information indicating the volume to be evaluated, and transmits the information to the transmission unit 22. Output (step S409). In step S402, the transmission unit 22 transmits to the sound wave output device 3 an output request notification including the frequency information and volume information output in step S409. The external device 2 performs operations from step S402 to step S409 until evaluation information is generated for reference sounds of all frequencies to be evaluated.

If the evaluation information has been generated for the reference sounds of all frequencies to be evaluated (step S408, YES), the determination unit 23 inputs an initialization end notification to the transmission unit 22. FIG.

The determination unit 23 inputs the initial setting end notification to the transmission unit 22 . The transmission unit 22 transmits a notification of completion of initialization to the sound wave output device 3 (step S410). Also, since the evaluation information has been generated for all the reference sounds, the external device 2 terminates the operation of FIG.

Next, an operation example in which the sound wave output device 3 performs initialization under the control of the external device 2 will be described with reference to FIG. Also, the operation in FIG. 7 details the operation from step S203 to step S205 in FIG.

The receiving unit 34 receives an output request notification for initial setting including frequency information and volume information (step S501). The receiving unit 34 associates the frequency information with the volume information and outputs them to the control unit 32 .

Frequency information and volume information are input from the receiving unit 34 to the control unit 32 . The control unit 32 transmits an acoustic signal for outputting a sound wave having the frequency indicated by the input frequency information and the volume indicated by the input volume information to the left ear sound wave output unit 331 and the right ear sound wave output unit 331 via the amplifying means. It is input to the output unit 332 .

An acoustic signal is input from the control unit 32 to the left ear sound wave output unit 331 . An acoustic signal is input from the control unit 32 to the right ear sound wave output unit 332 . The left ear sound wave output unit 331 converts an acoustic signal into a sound wave and outputs the sound wave to the left ear of the user. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave to the inside of the user's right ear (step S502).

In step S502, the left ear sound wave output unit 331 may output sound waves at a timing different from that of the right ear sound wave output unit 332, or may output sound waves at the same timing as the right ear sound wave output unit 332. . When the left ear sound wave output section 331 outputs sound waves at different timings than the right ear sound wave output section 332, the sound wave output device 3 receives the evaluation information for each of the left and right ears, and receives the reference sound for each of the left and right ears. The characteristic information may be stored in the reference sound characteristic information storage section 37 .

The left ear sound wave input unit 351 receives a sound wave reflected inside the left ear and outputs an input acoustic signal representing the received sound wave to the generation unit 31 . The right ear sound wave input unit 352 inputs the sound wave reflected inside the right ear, and outputs an input acoustic signal representing the input sound wave to the generation unit 31 (step S503). The right ear sound wave input section 352 and the left ear sound wave input section 351 output input acoustic signals to the generation section 31 at the same timing. Note that the right ear sound wave input section 352 may output the input acoustic signal to the generation section 31 at a timing different from that of the left ear sound wave input section 351 .

An input acoustic signal is input from the left ear sound wave input unit 351 to the generation unit 31 . An input acoustic signal is input from the right ear sound wave input unit 352 to the generation unit 31 . The generation unit 31 analyzes the signal strength of each frequency component that constitutes the frequency of the input acoustic signal from the left ear sound wave input unit 351 and the right ear sound wave input unit 352 . The generation unit 31 generates reflected sound wave characteristic information according to the analysis result (step S504).

The receiving unit 34 also receives the evaluation information (step S505). The receiving unit 34 associates the evaluation information with the frequency information and the volume information included in the output request notification received before receiving the evaluation information, and outputs them to the generating unit 31 . Note that the receiving unit 34 receives the evaluation information at arbitrary timing. The frequency information and the volume information included in the output request notification output to the generation unit 31 are output acoustic information.

The evaluation information and the output acoustic information are input from the receiving unit 34 to the generating unit 31 . The generation unit 31 does not perform the operations after step S507 until the notification of completion of the initial setting is input (step S506, NO).

Note that each time the receiving unit 34 receives the output request notification (step S501), the left ear sound wave output unit 331 and the right ear sound wave output unit 332 perform the operation of step S502. Also, the left ear sound wave input unit 351 and the right ear sound wave input unit 352 perform the operation of step S503. The generation unit 31 performs the operation of step S504. The receiving unit 34 performs the operation of step S505.

The receiving unit 34 receives the initial setting end notification from the external device 2 . Upon receiving the initialization end notification, the reception unit 34 outputs the initialization end notification to the generation unit 31 .

The generation unit 31 receives the initialization end notification from the reception unit 34 (step S506, YES). When the initial setting completion notification is input from the receiving unit 34, the following operations are performed. For each reference sound, the generation unit 31 associates the evaluation information with the reference sound characteristic information and stores them in the reference sound characteristic information storage unit 37 (step S507). The reference sound characteristic information includes output sound information indicating the frequency characteristics of the sound signal when each of the reference sounds was output, and reflected sound wave characteristic information indicating the frequency characteristics of the input sound signal corresponding to each of the reference sounds. included.

Note that the generation unit 31 may associate the evaluation information with the reference sound characteristic information and store them in the reference sound characteristic information storage unit 37 each time the output sound information and the evaluation information are input.

Next, referring to FIG. 8, the sound wave output device 3 inputs the sound signal to the sound wave output unit 33 with respect to the audio signal received from the sound wave input device (sound wave input devices 4-2 to 4-n in this example). An operation example will be described. Also, the operation in FIG. 8 details the operation from step S303 to step S308 in FIG.

Note that FIG. 8 assumes that a notification requesting control of the characteristics of the acoustic signal has not been input to the control unit 32 of the sound wave output device 3 before the sound wave output device 3 receives the audio signal in step S601, which will be described later. An example will be explained.

The receiving unit 34 receives an audio signal from a sound wave input device (sound wave input devices 4-2 to 4-n in this example) (step S601). The receiver 34 outputs the audio signal to the controller 32 .

An audio signal is input from the receiver 34 to the controller 32 . The control unit 32 inputs an acoustic signal for outputting a sound wave based on the audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplifying means (step S602).

The left ear sound wave output unit 331 converts an acoustic signal into a sound wave and outputs the sound wave to the left ear of the user. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave to the inside of the user's right ear (step S603).

The left ear sound wave input unit 351 receives a sound wave reflected inside the left ear and outputs an input acoustic signal representing the received sound wave to the generation unit 31 . The right ear sound wave input unit 352 receives the sound wave reflected inside the right ear, and outputs an input acoustic signal representing the input sound wave to the generation unit 31 (step S604).

Input acoustic signals are input to the generation unit 31 from the left ear sound wave input unit 351 and the right ear sound wave input unit 352 . The generator 31 analyzes the strength of the signal of each frequency component forming the input acoustic signal (step S605). The generation unit 31 generates sound wave characteristic information according to the analysis result. The generation unit 31 determines the sound wave characteristic information generated from the input sound signal input from the left ear sound wave input unit 351 and the sound wave characteristic information generated from the input sound signal input from the right ear sound wave input unit 352 to the determination unit 36 . output to

Based on the input sound wave characteristic information, the determination unit 36 determines whether sound waves that are easy for the user to hear are being output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 (step S606). ).

The determination unit 36 determines as follows in step S606. The determining unit 36 determines whether the frequency characteristic indicated by the reflected sound wave characteristic information included in the reference sound characteristic information stored in the reference sound characteristic information storage unit 37, which is easy for the user to hear, and the frequency characteristic indicated by the sound wave characteristic information. It is determined whether or not the difference is greater than or equal to a predetermined threshold. If the difference is greater than or equal to a predetermined threshold, the determination unit 36 determines that the sound wave output unit 33 outputs sound waves that are difficult for the user to hear. If it is less than the predetermined threshold, the determination unit 36 determines that the sound wave output unit 33 outputs sound waves that are easy for the user to hear.

If the determination unit 36 determines that the sound wave output unit 33 is outputting sound waves that are difficult for the user to hear (step S606, NO), the determination unit 36 performs the following operation. The determining unit 36 determines that the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear are output associated with the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 in association with the evaluation information indicating that the sound is easy to hear. Identify acoustic information. For example, the determination unit 36 determines the degree of similarity between the frequency characteristics indicated by the reflected sound wave property information and the frequency characteristics indicated by the input sound wave property information, among the reflected sound wave property information associated with the evaluation information indicating that the sound wave property is easy to hear. Then, the reflected sound wave characteristic information whose similarity is higher than the threshold is specified.

The determination unit 36 outputs to the control unit 32 a notification requesting control of the characteristics of the acoustic signal, including the specified output acoustic information (step S607).

If the determination unit 36 determines that sound waves that are easy for the user to hear are being output from the sound wave output unit 33 based on the input sound wave characteristic information (step S606, YES), the determination unit 36 does not perform the operation of step S607.

If the notification requesting control of the characteristics of the acoustic signal is not input, the control unit 32 generates an acoustic signal for outputting sound waves based on the audio signal in step S602. Also, when a notification requesting control of the characteristics of the acoustic signal is input, the control unit 32 performs the following operation in step S602. The control unit 32 controls characteristics of an acoustic signal to be output with respect to the audio signal from the receiving unit 34 . Specifically, the control unit 32 controls the characteristics of the acoustic signal so as to reduce the divergence from the output acoustic information included in the notification requesting the control of the characteristics of the acoustic signal. As a result, the control unit 32 generates an acoustic signal for causing the sound wave output unit 33 to output a sound wave based on the audio signal in which the characteristic of the acoustic signal is controlled.

Note that the determination unit 36 may determine in step S606 as follows. The determination unit 36 determines whether the frequency characteristic indicated by the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 in association with the evaluation information indicating difficulty in hearing and the frequency characteristic indicated by the sound wave characteristic information are equal to or greater than a predetermined threshold. It may be determined whether or not there is a divergence. If the difference is greater than or equal to a predetermined threshold, the determination unit 36 determines that the sound wave output unit 33 is outputting sound waves that are easy for the user to hear. If the deviation is less than the predetermined threshold, the determination unit 36 determines that the sound wave output unit 33 outputs sound waves that are difficult for the user to hear. In addition to these methods, the determination unit 36 can use any method to determine whether or not the sound wave output unit 33 is outputting sound waves that are easy for the user to hear.

As described above, the sound wave output device 3 of the present embodiment converts an acoustic signal into a sound wave and outputs the sound wave output by the sound wave output unit 33. Of the sound waves heard by the user, the characteristic of the sound wave reflected inside the ear is Generates sound wave characteristic information indicating The sound wave output device 3 has the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information. The characteristic of the acoustic signal based on the voice of the other party is controlled so that the difference between the characteristic of the sound wave and the characteristic of the sound wave obtained is small. The sound wave output device 3 also inputs the acoustic signal whose characteristics are controlled to the sound wave output unit 33 via the amplifying means. As a result, the sound wave output unit 33 can output, to the user, a sound wave based on the sound signal whose characteristic of the sound signal is adjusted according to the reference sound characteristic information. As a result, the sound wave output device 3 can cause the sound wave output unit 33 to output sound waves that are adjusted to be easily audible to the user.

In addition, the sound wave output device 3 of the present embodiment controls the characteristics of the acoustic signal so as to reduce the divergence from the output acoustic information included in the notification requesting the control of the characteristics of the acoustic signal. Generates an acoustic signal for outputting The output acoustic information included in the notification requesting control of the characteristics of the acoustic signal indicates the frequency characteristics of the acoustic signal when each of the reference sounds that are easy for the user to hear is output. The sound wave characteristic information of the input acoustic signal whose characteristic is controlled has a smaller divergence from the reflected sound wave characteristic information of the reference sound characteristic information than the sound wave characteristic information of the input acoustic signal of the acoustic signal whose characteristic is not controlled. Become. By operating in this manner, the control unit 32 can cause the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output sound waves that are easy for the user to hear.

In addition, since the sound wave output device 3 of the present embodiment continuously controls the characteristics of the acoustic signal input to the sound wave output unit 33, the sound wave output device 3 emits sound waves that are easy for the user to hear during a call or during an online meeting. can be automatically output in

Further, the sound wave output device 3 of the present embodiment controls the characteristics of the acoustic signal for the left ear sound wave output unit 331 based on the reference sound characteristic information for the left ear, and controls the characteristics of the acoustic signal for the right ear sound wave output unit 332. is controlled based on the reference sound characteristic information for the right ear. As a result, even if the user's left and right ears have different frequency characteristics of sound waves that are easily audible, the sound wave output device 3 produces sound waves that are adjusted to be easily audible according to the audibility of each of the left and right ears. can be output from the sound wave output unit 33 .

[Third Embodiment]
Next, the sound wave output device 5 according to the third embodiment of the present invention will be specifically described.

In the third embodiment, the generator 51 corresponds to, for example, the generator 11 of the first embodiment. In the third embodiment, the control section 52, the determination section 53, the reference sound characteristic information storage section 37, and the speaker information storage section 54 correspond to, for example, the control section 12 of the first embodiment. The third embodiment differs from the sound wave output device 1 of the first embodiment in that the sound wave output device 3 further includes a sound wave output section 33 , a reception section 34 and a sound wave input section 35 .

Further, the third embodiment is different from the second embodiment in that the left ear sound wave output unit 331 and the right ear sound wave output unit 332 of the sound wave output device 5 output sound waves based on voice signals at different timings for each speaker. It is different from the sound wave output device 3 of the embodiment. In this embodiment, the sound wave output device 5 controls the timing of outputting the sound wave based on the audio signal from the left ear sound wave output section 331 and the right ear sound wave output section 332 as characteristics of the sound signal for each speaker. In this embodiment, differences from the second embodiment will be mainly described, but various examples described in the second embodiment can be applied.

A configuration example of the sound wave output system of the present embodiment will be described with reference to FIG. FIG. 9 is a block diagram showing a configuration example of the sound wave output system of this embodiment.

The sound wave output system of this embodiment includes a sound wave output device 5 .

Each configuration of the external device 2 in this embodiment is the same as each configuration of the external device 2 in the second embodiment shown in FIG. omitted.

Since each of the configurations of the sound wave input devices 4-1 to 4-n in this embodiment is the same as each of the configurations of the sound wave input devices 4-1 to 4-n in the second embodiment shown in FIG. Corresponding elements are denoted by the same reference numerals as in FIG. 3, and description thereof is omitted.

Next, the configuration of the sound wave output device 5 of this embodiment will be described in detail with reference to FIG.

The sound wave output device 5 includes a generator 51 and a controller 52 .

The sound wave output section 33 is connected to the control section 52 . The receiver 34 is connected to the generator 51 and the controller 52 . The sound wave input section 35 is connected to the generation section 51 . The determination unit 53 is connected to the generation unit 51 , the control unit 52 , the reference sound characteristic information storage unit 37 and the speaker information storage unit 54 . The reference sound characteristic information storage unit 37 is connected to the generation unit 51 and the determination unit 53 . Speaker information storage unit 54 is connected to determination unit 53 .

The sound wave output unit 33, the reception unit 34, the sound wave input unit 35, and the reference sound characteristic information storage unit 37 in this embodiment are the same as those of the sound wave output device 3 in the second embodiment shown in FIG. Therefore, the same reference numerals as in FIG. 3 are given to the corresponding elements, and the common description is omitted.

Further, the processing for initializing each component of the sound wave output device 5 under the control of the external device 2 is the same as the processing for each component of the sound wave output device 3 of the second embodiment. Therefore, the description of the initial setting process performed by the sound wave output device 5 under the control of the external device 2 will be omitted.

Hereinafter, each configuration of the sound wave output device 5 when used in an online conference will be described in detail.

The receiving unit 34 receives an audio signal, which is a signal corresponding to audio, from the sonic input device (sonic wave input devices 4-2 to 4-n in this example). The receiver 34 outputs the audio signal to the controller 52 .

The sound wave output section 33 includes a left ear sound wave output section 331 and a right ear sound wave output section 332 .

An acoustic signal is input to the left ear sound wave output unit 331 from the control unit 52 to be described later. The left ear sound wave output unit 331 converts an acoustic signal into a sound wave and outputs the sound wave to the left ear of the user.

An acoustic signal is input from the control unit 52 to the right ear sound wave output unit 332 . The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave to the left ear of the user. Each of right ear sound wave output section 332 and left ear sound wave output section 331 outputs sound waves at predetermined timings based on the acoustic signals. Further, the predetermined timing is the timing at which sound is output from the right ear sound wave output unit 332 and the timing at which sound is output from the left ear sound wave output unit 331, which is set for each speaker using the sound wave output system. is the timing indicated by the time difference information indicating . It should be noted that the sound wave output device 5 causes the right ear sound wave output unit 332 and the left ear sound wave output unit 331 to output sound waves at different volumes, as in the timing, so that each speaker can hear the voice from a specific direction. may output sound waves.

The generation unit 51 in this embodiment is the same as the generation unit 31 of the sound wave output device 3 in the second embodiment shown in FIG. 3, so description thereof will be omitted.

Sound wave characteristic information generated from the input acoustic signal of the sound wave input unit 35 is input from the generation unit 51 to the determination unit 53 . Based on the input sound wave characteristic information, the determination unit 53 determines whether or not sound waves that are easy for the user to hear are being output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 . The process of determining whether or not sound waves that are easy for the user to hear are being output by the left ear sound wave output unit 331 and the right ear sound wave output unit 332 are the same as those in the second embodiment, so description thereof is omitted. do.

If the determination unit 53 determines that the sound wave output unit 33 is outputting a sound wave that is difficult for the user to hear based on the input sound wave characteristic information, the determination unit 53 specifies the output sound information. The operation of specifying the output acoustic information is the same as in the second embodiment, so the explanation is omitted.

Further, if the determination unit 53 determines that the sound wave output unit 33 is outputting sound waves that are easy for the user to hear based on the input sound wave characteristic information, the determination unit 53 does not perform the process of specifying the output sound information.

The determination unit 53 identifies the speaker of the voice based on the input sound wave characteristic information and the speaker information stored in the speaker information storage unit 54 . The speaker information indicates the frequency characteristics of each voice of the speaker. Speaker information storage unit 54 stores speaker information, the timing at which sound is output from right ear sound wave output unit 332 and the timing at which sound is output from left ear sound wave output unit 331, which are set for each speaker. are stored in association with the indicated time difference information. The speaker information stored in the speaker information storage unit 54 is sound wave characteristic information generated by the generation unit 51 . The determination unit 53 may calculate the degree of similarity between the information indicating the frequency distribution included in the sound wave characteristic information and the information indicating the frequency distribution included in the speaker information. The determination unit 53 identifies speaker information whose similarity is higher than a threshold, and identifies time difference information to be stored in the speaker information storage unit 54 in association with the identified speaker information.

Also, the determination unit 53 may use machine learning to identify the speaker. For example, the determination unit 53 uses learning data that includes input acoustic signals of voices of a plurality of people and information indicating which person's voices are the input acoustic signals of voices of a plurality of people. Based on the learning data, the determination unit 53 generates an extraction model for extracting a feature quantity unique to each speaker from the input acoustic signal, and extracts information indicating the feature quantity from the input acoustic signal using the extraction model. good too. The determination unit 53 may calculate the degree of similarity between the feature amount extracted from the input acoustic signal and the feature amount stored in the speaker information storage unit 54 as speaker information. The determination unit 53 identifies speaker information whose similarity is higher than a threshold, and identifies time difference information to be stored in the speaker information storage unit 54 in association with the identified speaker information. If the time difference information cannot be identified, information indicating the feature amount extracted from the input acoustic signal is stored in the speaker information storage unit 54 as speaker information.

When the time difference information cannot be specified, the determination unit 53 performs the following operations. The determination unit 53 assigns time difference information different from the time difference information stored in the speaker information storage unit 54 to the speaker whose time difference information cannot be specified. For example, the determination unit 53 assigns unassigned time difference information from a plurality of preset time difference information in the order of utterance by the speaker. In this way, the determination unit 53 determines the timing at which the sound is output from the left ear sound wave output unit 331 and the right ear sound wave output unit 332 for each speaker who speaks in the online conference so as not to overlap with other speakers. to be specified. As a result, the control unit 52 can output sound waves so that the voice of the user who newly participates in the online conference can be heard from a direction different from that of other users. The determination unit 53 causes the speaker information storage unit 54 to store the assigned time difference information and the sound wave characteristic information as the speaker information.

When the output acoustic information is specified, the determination unit 53 outputs to the control unit 52 a notification requesting control of the characteristics of the acoustic signal including the specified output acoustic information and the time difference information. When the output acoustic information is not specified, the determination unit 53 outputs to the control unit 52 a notification requesting control of the sound wave output including the time difference information.

Speaker information and time difference information are associated and stored in the speaker information storage unit 54 . The speaker information stored in the speaker information storage unit 54 is sound wave characteristic information generated by the generation unit 51, which indicates the frequency characteristics of the voice of each speaker. Also, the speaker information may be information indicating a feature amount extracted from the input acoustic signal. In addition, the speaker information storage unit 54 stores time difference information that differs for each speaker in association with the speaker information. The speaker information and the time difference information may be stored in advance in the speaker information storage unit 54 by the user.

An audio signal is input from the receiver 34 to the controller 52 . A notification requesting control of the characteristics of the acoustic signal is input from the determination unit 53 to the control unit 52 . Alternatively, the control unit 52 receives a notification requesting control of the output of sound waves from the determination unit 53 . When a notification requesting control of sound wave output is input from the determination unit 53, the control unit 52 controls the characteristics of the acoustic signal to be output with respect to the audio signal from the reception unit 34 as follows. The control unit 52 outputs an acoustic signal for causing the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output a sound wave based on the audio signal at the timing indicated by the time difference information included in the notification requesting control of the sound wave output. to generate The control unit 52 inputs the generated acoustic signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplifying means.

When a notification requesting control of the characteristics of the acoustic signal is input, the control unit 52 controls the characteristics of the acoustic signal to be output with respect to the audio signal from the receiving unit 34 as follows. The control unit 52 adjusts the frequency characteristics of the other party so that the difference between the frequency characteristics indicated by the reference sound characteristic information and the characteristics of the sound waves that are easy for the user to hear and the characteristics of the sound waves reflected inside the ears of the user is small. Controls the characteristics of speech-based acoustic signals. In addition to controlling the characteristics of the acoustic signal, the control unit 52 generates an acoustic signal for causing the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output sound waves based on the audio signal at the timing indicated by the time difference information. do. The control unit 52 inputs an acoustic signal for outputting a sound wave based on the audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplifying means. In this manner, the control unit 52 controls the acoustic signals so that sound waves are output from the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the specified time difference information.

Note that when neither the notification requesting control of the characteristics of the acoustic signal nor the notification requesting control of the output of the sound wave is input, the control unit 52 may perform the following operation. The control unit 52 supplies an acoustic signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 so that the left ear sound wave output unit 331 and the right ear sound wave output unit 332 simultaneously output sound waves based on the audio signal. may be entered. If neither the notification requesting the control of the characteristics of the acoustic signal nor the notification requesting the control of the sound wave output is input, it is assumed, for example, at the start of an online conference.

In this way, the sound wave output device 5 generates sound wave characteristic information indicating the characteristics of the sound wave reflected inside the ear among the sound waves heard by the user by the sound wave output unit 33 that converts an acoustic signal into a sound wave and outputs the sound wave. do. The sound wave output device 5 has the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information. The characteristics of the acoustic signal based on the voice of the other party are controlled so that the divergence from the characteristics of the sound waves obtained is small. The sound wave output device 5 inputs the acoustic signal whose characteristics are controlled to the sound wave output section 33 via the amplifying means. As a result, the sound wave output unit 33 can output, to the user, a sound wave based on the sound signal whose characteristic of the sound signal is adjusted according to the reference sound characteristic information. As a result, the sound wave output device 5 can cause the sound wave output unit 33 to output sound waves that are adjusted to be easily audible to the user.

Next, an operation example of the sound wave output system of this embodiment will be described with reference to FIGS. 10 to 11. FIG.

FIG. 10 is a sequence diagram showing an operation example of the sound wave output system. FIG. 11 is a flow chart showing an operation example of the sound wave output device 5 .

First, referring to FIG. 10, for audio signals received from sound wave input devices (sound wave input devices 4-2 to 4-n in this example), sound wave output device 5 controls characteristics of the sound signal to output sound waves. The operation of inputting to the unit 33 will be described.

The sound wave output device 5 of the present embodiment generates sound wave characteristic information of the sound wave reflected inside the ear among the previously output sound waves during the online conference. Then, the sound wave output device 5 controls at least the output timing of the left ear sound wave output section 331 and the right ear sound wave output section 332 as the characteristic of the acoustic signal of the sound wave output next by the sound wave output section 33 .

In FIG. 10, an example will be described in which a notification requesting control of the output of sound waves is input to the control unit 52 of the sound wave output device 5 before step S703, which will be described later.

Each of the sound wave input devices 4-2 to 4-n receives voice input from another user (step S701). Each of the sound wave input devices 4-2 to 4-n that receives the voice input generates a voice signal corresponding to the voice. Each of the sound wave input devices 4-2 to 4-n that generated the audio signal is a sound wave output device (in this example, the sound wave output device 5 is (step S702).

The receiver 34 of the sound wave output device 5 receives audio signals from the sound wave input devices (the sound wave input devices 4-2 to 4-n in this example). The receiver 34 outputs the audio signal to the controller 52 .

An audio signal is input from the receiver 34 to the controller 52 . In addition, a notification requesting control of the sound wave output is input to the control unit 52 . The control unit 52 outputs an acoustic signal for causing the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output a sound wave based on the audio signal at the timing indicated by the time difference information included in the notification requesting control of the sound wave output. to generate The control unit 52 inputs the generated acoustic signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplifying means.

The left ear sound wave output unit 331 converts an acoustic signal into a sound wave and outputs the sound wave to the left ear of the user. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave to the inside of the user's right ear (step S703). In step S703, the left ear sound wave output unit 331 and the right ear sound wave output unit 332 output sound waves at predetermined timing based on the acoustic signal. Note that the sound wave output in step S703 may be a sound wave whose characteristics are controlled by the sound wave output device 5 .

The left ear sound wave input unit 351 inputs sound waves reflected inside the user's left ear, and outputs an input acoustic signal representing the input sound waves to the generation unit 51 . The right ear sound wave input unit 352 receives the sound wave reflected inside the user's right ear, and outputs an input acoustic signal representing the input sound wave to the generation unit 51 (step S704).

Input acoustic signals are input to the generation unit 51 from the left ear sound wave input unit 351 and the right ear sound wave input unit 352 . The generation unit 51 analyzes the signal strength of each frequency component that constitutes the input acoustic signal (step S705). The generation unit 51 generates sound wave characteristic information according to the analysis result. The generation unit 51 determines the sound wave characteristic information generated from the input sound signal input from the left ear sound wave input unit 351 and the sound wave characteristic information generated from the input sound signal input from the right ear sound wave input unit 352 to the determination unit 53 . output to

Based on the input sound wave characteristic information, the determination unit 53 determines whether sound waves that are easy for the user to hear are being output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 (step S706). ). Specifically, the determination unit 53 determines that the frequency characteristic indicated by the reflected sound wave characteristic information included in the reference sound characteristic information that is easy for the user to hear, stored in the reference sound characteristic information storage unit 37, and the sound wave characteristic information. It is determined whether or not there is a deviation from the indicated frequency characteristic by a predetermined threshold or more. If the difference is equal to or greater than a predetermined threshold, the sound wave corresponding to the input sound wave characteristic information is a sound wave that is difficult for the user to hear. If the difference is less than the predetermined threshold, the sound wave is easy for the user to hear.

If the determining unit 53 determines in step S706 that the sound wave output unit 33 is outputting a sound wave that is difficult for the user to hear based on the input sound wave characteristic information, the determining unit 53 performs the following operation. For the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear, the determining unit 53 outputs the output associated with the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 in association with the evaluation information indicating that the sound is easy to hear. Identify acoustic information.

If the determining unit 53 determines in step S706 that sound waves that are easy for the user to hear are being output from the sound wave output unit 33 based on the input sound wave characteristic information, the determining unit 53 does not specify the output sound information.

The determination unit 53 identifies the speaker of the voice based on the input sound wave characteristic information and the speaker information stored in the speaker information storage unit 54 . The determination unit 53 identifies time difference information to be stored in the speaker information storage unit 54 in association with the identified speaker information.

When the time difference information cannot be specified, the determination unit 53 performs the following operations. The determination unit 53 assigns time difference information different from the time difference information stored in the speaker information storage unit 54 . The determination unit 53 causes the speaker information storage unit 54 to store the time difference information and the frequency information included in the sound wave characteristic information as the speaker information. Thus, the determination unit 53 identifies or assigns the time difference information (step S707).

When the output acoustic information is specified, the determination unit 53 outputs to the control unit 52 a notification requesting control of the characteristics of the acoustic signal including the specified output acoustic information and the time difference information. When the output acoustic information is not specified, the determination unit 53 outputs to the control unit 52 a notification requesting control of the sound wave output including the time difference information.

An audio signal is input from the receiver 34 to the controller 52 . A notification requesting control of the characteristics of the acoustic signal is input from the determination unit 53 to the control unit 52 . Alternatively, the control unit 52 receives a notification requesting control of the output of sound waves from the determination unit 53 . When a notification requesting control of sound wave output is input from the determination unit 53 , the control unit 52 performs the following operations on the audio signal to be output from the reception unit 34 . The control unit 52 outputs an acoustic signal for causing the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output a sound wave based on the audio signal at the timing indicated by the time difference information included in the notification requesting control of the sound wave output. to generate The control unit 52 inputs the generated acoustic signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplifying means. Also, when a notification requesting control of the characteristics of the acoustic signal is input, the control unit 52 performs the following operation on the acoustic signal to be output with respect to the audio signal from the receiving unit 34 . The control unit 52 adjusts the frequency characteristics of the other party so that the difference between the frequency characteristics indicated by the reference sound characteristic information and the characteristics of the sound waves that are easy for the user to hear and the characteristics of the sound waves reflected inside the ears of the user is small. Controls the characteristics of speech-based acoustic signals. In addition to controlling the characteristics of the acoustic signal, the control unit 52 generates an acoustic signal for causing the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output sound waves based on the audio signal at the timing indicated by the time difference information. do. Thus, the control unit 52 generates an acoustic signal for outputting a sound wave based on the audio signal (step S708). The control unit 52 inputs the acoustic signal generated in step S708 to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplifying means.

Acoustic signals are input from the control unit 52 to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 . The right ear sound wave output unit 332 and the left ear sound wave output unit 331 convert the acoustic signal into sound waves and output the sound waves into the ears of the user (step S709).

Note that the determination unit 53 may perform the operations of steps S706 and S707 in parallel. Alternatively, the determination unit 53 may perform the operation of step S706 after performing the operation of step S707.

Next, referring to FIG. 11, the sound wave output device 5 inputs the sound signal to the sound wave output unit 33 with respect to the sound signal received from the sound wave input device (sound wave input devices 4-2 to 4-n in this example). An operation example will be described. Also, the operation in FIG. 11 details the operation from step S703 to step S709 in FIG. In FIG. 11, an example will be described in which a notification requesting control of the output of sound waves is input to the control unit 52 of the sound wave output device 5 before step S802, which will be described later.

The receiving unit 34 receives an audio signal from a sound wave input device (sound wave input devices 4-2 to 4-n in this example) (step S801). The receiver 34 outputs the audio signal to the controller 52 .

An audio signal is input from the receiver 34 to the controller 52 . In addition, a notification requesting control of the sound wave output is input to the control unit 52 . The control unit 52 causes the left ear sound wave output unit 331 and the right ear sound wave output unit 331 to output acoustic signals for causing the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output sound waves based on the audio signal at predetermined timings. 332 through the amplifying means (step S802). The predetermined timing is the timing indicated by the time difference information.

It should be noted that, in step S802, when a notification requesting control of the characteristics of the acoustic signal is input, the control unit 52 performs the following operation on the acoustic signal to be output with respect to the audio signal from the receiving unit 34. The control unit 52 adjusts the frequency characteristics of the other party so that the difference between the frequency characteristics indicated by the reference sound characteristic information and the characteristics of the sound waves that are easy for the user to hear and the characteristics of the sound waves reflected inside the ears of the user is small. Controls the characteristics of speech-based acoustic signals. In addition to controlling the characteristics of the acoustic signal, the control unit 52 generates an acoustic signal for causing the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output sound waves based on the audio signal at the timing indicated by the time difference information. do. The control unit 52 inputs an acoustic signal for outputting a sound wave based on the audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via the amplifying means.

The left ear sound wave output unit 331 converts an acoustic signal into a sound wave and outputs the sound wave to the left ear of the user. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave to the inside of the user's right ear (step S803).

The left ear sound wave input unit 351 receives a sound wave reflected inside the left ear and outputs an input acoustic signal representing the received sound wave to the generation unit 51 . The right ear sound wave input unit 352 receives the sound wave reflected inside the right ear and outputs an input acoustic signal representing the input sound wave to the generation unit 51 (step S804).

Input acoustic signals are input to the generation unit 51 from the left ear sound wave input unit 351 and the right ear sound wave input unit 352 . The generator 51 analyzes the strength of the signal of each frequency component forming the input acoustic signal (step S805). The generation unit 51 generates sound wave characteristic information according to the analysis result. The generation unit 51 determines the sound wave characteristic information generated from the input sound signal input from the left ear sound wave input unit 351 and the sound wave characteristic information generated from the input sound signal input from the right ear sound wave input unit 352 to the determination unit 53 . output to

Based on the input sound wave characteristic information, the determination unit 53 determines whether sound waves that are easy for the user to hear are being output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 (step S806). ). Since the specific processing in step S806 is the same as the processing in step S706 of FIG. 10, the description is omitted.

If the determination unit 53 determines that the sound wave output unit 33 outputs a sound wave that is difficult for the user to hear based on the input sound wave characteristic information (step S806, NO), the determination unit 53 performs the following operation. For the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear, the determining unit 53 outputs the output associated with the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 in association with the evaluation information indicating that the sound is easy to hear. Acoustic information is specified (step S807).

If the determination unit 53 determines that the sound wave output unit 33 is outputting sound waves that are easy for the user to hear based on the input sound wave characteristic information (step S806, YES), the operation of step S807 is not performed.

The determination unit 53 identifies the speaker of the voice based on the input sound wave characteristic information and the speaker information stored in the speaker information storage unit 54 . If the speaker has been identified (step S808, YES), the determination unit 53 performs the following operation. The determination unit 53 identifies the time difference information stored in the speaker information storage unit 54 in association with the speaker information of the identified speaker (step S809).

If the speaker cannot be specified (step S808, NO), the determination unit 53 does not perform the operation of step S809. The determination unit 53 assigns time difference information different from the time difference information stored in the speaker information storage unit 54 to a speaker whose time difference information cannot be identified because the speaker cannot be identified. The determination unit 53 causes the speaker information storage unit 54 to store the assigned time difference information and sound wave characteristic information as speaker information (step S810).

After performing the operation of step S809 or step S810, the determination unit 53 performs the following operation. When the output acoustic information is specified, that is, the determination unit 53 that has performed the operation of step S807 outputs a notification requesting control of the characteristics of the acoustic signal to the control unit 52 . The notification requesting control of the characteristics of the acoustic signal includes the specified output acoustic information and the time difference information. When the output acoustic information is not specified, that is, the determination unit 53 which does not perform the operation of step S807 performs the following operation. The determination unit 53 outputs to the control unit 52 a notification requesting control of the output of the sound wave including the time difference information. Thus, the determination unit 53 outputs a notification requesting control of the characteristics of the acoustic signal or a notification requesting control of the output of the sound wave to the control unit 52 (step S811).

Note that the determination unit 53 may perform the operations from step S806 to step S807 and the operations from step S808 to step S810 in parallel. Alternatively, the determination unit 53 may perform the operations of steps S806 to S807 after performing the operations of steps S808 to S810.

As described above, the sound wave output device 5 of the present embodiment converts an acoustic signal into a sound wave and outputs it by the sound wave output unit 33. Of the sound waves heard by the user, the characteristic of the sound wave reflected inside the ear is Generates sound wave characteristic information indicating The sound wave output device 5 has the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information. The characteristic of the acoustic signal based on the voice of the other party is controlled so that the difference between the characteristic of the sound wave and the characteristic of the sound wave obtained is small. The sound wave output device 5 inputs the acoustic signal whose characteristics are controlled to the sound wave output section 33 via the amplifying means. As a result, the sound wave output unit 33 can output, to the user, a sound wave based on the sound signal whose characteristic of the sound signal is adjusted according to the reference sound characteristic information. As a result, the sound wave output device 5 can cause the sound wave output unit 33 to output sound waves that are adjusted to be easily audible to the user.

Further, the sound wave output device 5 of the present embodiment causes the left ear sound wave output section 331 and the right ear sound wave output section 332 to output sound waves at the timing indicated by the time difference information. Specifically, the sound wave output device 5 causes the left ear sound wave output section 331 and the right ear sound wave output section 332 to output sound waves at different timings for each speaker. As a result, the sound waves are output so that the voice can be heard from a specific direction for each speaker who is speaking, so that the user of the sound wave output device 5 can easily determine who is speaking among the speakers. can do.

In addition, since the sound wave output device 5 of the present embodiment continuously controls the characteristics of the acoustic signal output from the sound wave output unit 33, the sound wave output device 5 emits sound waves that are easy for the user to hear during a call or online. Can be automatically output during the meeting.

[Modification of the third embodiment]
The control unit of the sound wave output device of the modified example of the third embodiment controls the characteristics of the acoustic signal in the third embodiment, and controls the consonant sounds for each of the left ear sound wave output unit and the right ear sound wave output unit. Control is performed so that the corresponding sound wave volume is output within a predetermined threshold. The method of controlling the characteristics of the acoustic signal in the third embodiment is the same as in the third embodiment, so the explanation is omitted. Further, the sound wave output device 3 according to the second embodiment, in addition to controlling the characteristics of the acoustic signal according to the second embodiment, generates sound waves corresponding to consonants for each of the left ear sound wave output section and the right ear sound wave output section. may be controlled so that the volume of is output within a predetermined threshold.

The generator of this modified example analyzes the strength of the signal of each frequency component that constitutes the inputted input acoustic signal. The generation unit outputs sound wave characteristic information, which further includes sound volume information indicating the sound volume of the sound wave corresponding to the consonant, to the determination unit according to the analysis result.

Sound wave characteristic information further including sound wave volume information corresponding to a consonant is input to the determination unit of this modification. The determination unit outputs a notification requesting control of the volume of consonants to the control unit.

Note that the determination unit may determine whether or not the volume of the sound wave corresponding to the consonant is within a predetermined threshold based on the volume information of the sound wave corresponding to the consonant. If it is within the predetermined threshold, the determination unit may operate in the same manner as the sound wave output device 5 of the third embodiment. In addition, if it is not within the predetermined threshold, the determination unit may output a notification requesting control of the volume of the consonant to the control unit.

A notification requesting control of the volume of consonants is input from the determination unit to the control unit of this modification. When a notification requesting control of the volume of consonants is input, the control unit controls the characteristics of the audio signal to be output from the audio signal from the receiving unit as follows. The controller generates an acoustic signal such that the volume of the sound wave corresponding to the consonant is within a predetermined threshold. The control unit inputs an acoustic signal for outputting a sound wave based on the audio signal to the left ear sound wave output unit and the right ear sound wave output unit.

The sound wave output unit outputs a sound wave based on the acoustic signal to the inside of the ear of the user.

The control unit of the sound wave output device of this modified example generates an acoustic signal so that the volume of the sound wave corresponding to the consonant is within a predetermined threshold. As a result, the control unit of the sound wave output device according to the present modification can cause the sound wave output unit to output sound waves adjusted so that the volume of the consonants is approximately the same. Since the control section controls the characteristics of the acoustic signal input to the sound wave output section so that the volume of the sound wave corresponding to the consonants is eliminated, the user can clearly hear the conversation.

[Hardware configuration example]
A configuration example of hardware resources for realizing each of the sound wave output devices (1, 3, 5) in each embodiment of the present invention described above using one information processing device (computer) will be described. Note that the sound wave output device may be physically or functionally realized using at least two information processing devices. Also, the sound wave output device may be implemented as a dedicated device. Also, only a part of the functions of the sound wave output device may be realized using the information processing device.

FIG. 12 is a diagram schematically showing a hardware configuration example of an information processing device capable of realizing the sound wave output device of each embodiment of the present invention. The information processing device 6 includes a communication interface 61 , an input/output interface 62 , an arithmetic device 63 , a storage device 64 , a nonvolatile storage device 65 and a drive device 66 .

For example, the generation unit 11 of the sound wave output device 1 of FIG. Also, the control unit 12 of the sound wave output device 1 of FIG.

The communication interface 61 is communication means for the sound wave output device of each embodiment to communicate with an external device by wire and/or wirelessly. When the sound wave output device is implemented using at least two information processing devices, these devices may be connected via the communication interface 61 so as to be able to communicate with each other.

The input/output interface 62 is a man-machine interface such as a keyboard as an input device and a display as an output device.

The arithmetic unit 63 is realized by a general-purpose CPU (Central Processing Unit), an arithmetic processing unit such as a microprocessor, and a plurality of electric circuits. The computing device 63 can, for example, read various programs stored in the nonvolatile storage device 65 to the storage device 64 and execute processing according to the read programs.

The storage device 64 is a memory device such as a RAM (Random Access Memory) that can be referred to by the computing device 63, and stores programs, various data, and the like. Storage device 64 may be a volatile memory device.

The non-volatile storage device 65 is a non-volatile storage device such as ROM (Read Only Memory), flash memory, etc., and is capable of storing various programs and data.

The drive device 66 is, for example, a device that processes data reading and writing with respect to a recording medium 67, which will be described later.

The recording medium 67 is, for example, an optical disk, a magneto-optical disk, a semiconductor flash memory, or any other recording medium capable of recording data.

In each embodiment of the present invention, for example, the information processing device 6 illustrated in FIG. 12 may constitute the sound wave output device. Each embodiment of the present invention may be realized by supplying a program capable of realizing the functions described in each of the above embodiments to the sound wave output device.

In this case, the embodiment can be realized by having the arithmetic device 63 execute the program supplied to the sound wave output device. It is also possible to configure the information processing device 6 to perform not all the functions of the sound wave output device, but some of the functions.

Further, the sound wave output device is configured so that the program is recorded in the recording medium 67 and stored in the non-volatile storage device 65 as appropriate at the stage of shipment or operation of the sound wave output device. good too. In this case, as the method of supplying the program, a method of installing the program in the sound wave output device using an appropriate jig at the manufacturing stage before shipment or the operation stage may be adopted. Moreover, as a method of supplying the program, a general procedure such as a method of downloading from the outside via a communication line such as the Internet may be adopted.

The embodiments described above are preferred embodiments of the present invention, and various modifications can be made without departing from the gist of the present invention.

1, 3, 5 sound wave output device 11, 31, 51 generation unit 12, 32, 52 control unit 33 sound wave output unit 331 left ear sound wave output unit 332 right ear sound wave output unit 34 reception unit 35 sound wave input unit 351 left ear sound wave input Section 352 Right ear sound wave input section 36, 53 Judgment section 37 Reference sound characteristic information storage section 54 Speaker information storage section 2 External device 21 Input/output section 22 Transmission section 23 Judgment section 4-1 to 4-n Sound wave input device 41- 1 to 41-n sound wave input section 42-1 to 42-n transmission section 6 information processing device 61 communication interface 62 input/output interface 63 arithmetic device 64 storage device 65 nonvolatile storage device 66 drive device 67 recording medium

Claims (8)

generating means for generating sound wave characteristic information indicating the characteristics of sound waves reflected inside the ear among the sound waves heard by the user by the sound wave output means for converting an acoustic signal into sound waves and outputting the sound waves;
A frequency characteristic indicated by the sound wave characteristic information and a frequency characteristic indicated by the reference sound characteristic information regarding a reference sound having a frequency component that can be heard by the user, wherein the sound wave of the reference sound is the ear of the user. The characteristic of the acoustic signal based on the voice of the other party is controlled so that the difference between the characteristic of the sound wave that is reflected inside and is set to be easy for the user to hear and the characteristic of the sound wave is amplified through an amplifying means. a control means for inputting to the output means;
A sound wave output device. further comprising sound wave input means for inputting the sound wave of the other party, which is output from the sound wave output means to the inside of the ear and reflected inside the ear, and for outputting an input acoustic signal representing the input sound wave;
The reference sound characteristic information is frequency information indicating the result of determination by the user as to whether or not the reference sound is easy to hear, and the frequency characteristics of the sound wave reflected inside the ear of the user. including
2. The sound wave output device according to claim 1, wherein said generating means generates said sound wave characteristic information according to an analysis result of signal strength of each frequency component constituting said input sound signal. 3. The sound wave output device according to claim 2, wherein the control means controls the acoustic signal so that the frequency characteristic of the sound wave characteristic information does not deviate from the frequency characteristic of the reference sound characteristic information. the reference sound is a plurality of types of sound waves having different frequencies and volumes output from the sound wave output means;
the sound wave input means outputs the input sound signal for each of the reference sounds;
further comprising a receiving unit that receives evaluation information indicating whether or not each of the reference sounds is easy to hear from an external device;
The generating means includes output acoustic information indicating frequency characteristics of the acoustic signal when each of the reference sounds is output, and reflected sound characteristic information indicating frequency characteristics of the input acoustic signal corresponding to each of the reference sounds. is stored in the reference sound characteristic information storage means as the reference sound characteristic information,
The control means controls the frequency characteristic indicated by the reflected sound wave characteristic information included in the reference sound characteristic information stored in the reference sound characteristic information storage means, the frequency characteristic indicated by the sound wave characteristic information. 3. The characteristic of the acoustic signal is controlled so as to reduce the divergence from the frequency characteristic indicated by the output acoustic information associated with the reflected sound characteristic information when there is a divergence between the The sound wave output device according to claim 3. The generation means generates the sound wave characteristic information further including sound volume information indicating the sound volume corresponding to the consonant according to the analysis result of the signal strength of each frequency component constituting the input sound signal,
5. The controller according to any one of claims 2 to 4, wherein the control means controls the characteristics of the acoustic signal so that the volume of the sound wave corresponding to the consonant is within a predetermined threshold. The sound wave output device according to the item. The sound wave output means includes right ear sound wave output means for inputting sound into the user's right ear and left ear sound wave output means for inputting sound into the user's left ear,
speaker information indicating the frequency characteristics of the voice of each speaker; speaker information storage means for storing time difference information indicating the timing of
A speaker of the voice is specified based on the speaker information stored in the speaker information storage means, and the right ear sound wave output means and the left ear sound wave output are performed according to the specified speaker and the time difference information. Further comprising a determination means for specifying the timing at which the sound is output from the means,
The control means controls the acoustic signal so that sound waves are output from the left ear sound wave output means and the right ear sound wave output means at the timing indicated by the specified time difference information. The sound wave output device according to any one of claims 1 to 5. generating sound wave characteristic information indicating the characteristics of sound waves reflected inside the ear among the sound waves audible to the user by sound wave output means for converting acoustic signals into sound waves and outputting them;
A frequency characteristic indicated by the sound wave characteristic information and a frequency characteristic indicated by the reference sound characteristic information regarding a reference sound having a frequency component that can be heard by the user, wherein the sound wave of the reference sound is the ear of the user. The characteristic of the acoustic signal based on the voice of the other party is controlled so that the difference between the characteristic of the sound wave that is reflected inside and is set to be easy for the user to hear and the characteristic of the sound wave is amplified through an amplifying means. Sound wave output method to input to the output means. to the computer,
a generation function for generating sound wave characteristic information indicating the characteristics of sound waves reflected inside the ear among the sound waves heard by the user by sound wave output means for converting acoustic signals into sound waves and outputting them;
A frequency characteristic indicated by the sound wave characteristic information and a frequency characteristic indicated by the reference sound characteristic information regarding a reference sound having a frequency component that can be heard by the user, wherein the sound wave of the reference sound is the ear of the user. The characteristic of the acoustic signal based on the voice of the other party is controlled so that the difference between the characteristic of the sound wave that is reflected inside and is set to be easy for the user to hear and the characteristic of the sound wave is amplified through an amplifying means. a control function input to the output means;
A sound wave output program that realizes

Images