Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10K—SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
  • G10K15/00—Acoustics not otherwise provided for
  • G10K15/04—Sound-producing devices
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
  • H04R1/00—Details of transducers, loudspeakers or microphones
  • H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones

Definitions

• the present disclosure relates to a signal processing device, an earphone with a microphone, a signal processing method, and a program.
• US Pat. No. 5,001,002 discloses adjusting the volume by increasing or decreasing the amplitude of a sound wave or soundtrack to create a rhythmic stimulus corresponding to a stimulation frequency that induces brain wave entrainment. There is.
• Multi-sensory Gamma Stimulation Ameliorates Alzheimer’s-Associated Pathology and Improves Cognition Cell 2019 Apr 4;177(2):256-271.e22. doi: 10.1016/j.cell.2019.02.014.
• amplitude-modulated acoustic signals In order to improve cognitive function, it is desirable for users to have more opportunities to listen to sounds based on amplitude-modulated acoustic signals.
• listening to an amplitude-modulated soundtrack may be a burden. For example, if a user wants to listen to the surrounding sounds and a soundtrack is played, the user will not be able to concentrate on the surrounding sounds. Also, for example, the user may not be in the mood to listen to a soundtrack.
• a signal processing device includes an acquisition unit that acquires an input acoustic signal based on sound collected by a microphone placed near a user's ear, and an amplitude modulation method for the input acoustic signal acquired by the acquisition unit.
• a generating means generates an output acoustic signal having a volume intensity corresponding to the frequency of the gamma wave by performing modulation processing including and output means for outputting to a speaker.
• FIG. 2 is an explanatory diagram of one aspect of the present embodiment. It is a flowchart of acoustic signal processing of this embodiment.
• 3 is a flowchart illustrating an example of signal processing for an external sound signal.
• 5 is a diagram illustrating a waveform of an acoustic signal handled in the signal processing of FIG. 4.
• FIG. 3 is a flowchart illustrating an example of signal processing for a reproduced sound signal.
• 3 is a flowchart illustrating an example of signal processing for an external sound signal.
• 8 is a diagram illustrating a waveform of an acoustic signal handled in the signal processing of FIG. 7.
• FIG. 3 is a flowchart illustrating an example of signal processing for an external sound signal.
• 10 is a diagram illustrating a waveform of an acoustic signal handled in the signal processing of FIG. 9.
• FIG. 9 is a diagram illustrating a waveform of an acoustic signal handled in the signal processing of FIG. 9. FIG.
• FIG. 1 is a block diagram showing the configuration of an earphone with a microphone according to this embodiment.
• the earphone 1 with a microphone of this embodiment may correspond to, for example, headphones, earphones, or a hearing aid.
• the earphone 1 with a microphone constitutes a cognitive function improvement system.
• the earphone with microphone 1 includes a signal processing device 10, a microphone 21, and a speaker 22.
• the signal processing device 10 performs acoustic signal processing on various input acoustic signals including acoustic signals that can be obtained from the microphone 21.
• the acoustic signal processing by the signal processing device 10 includes at least modulation processing of the acoustic signal (details will be described later).
• the acoustic signal processing by the signal processing device 10 may include conversion processing (for example, separation, extraction, or synthesis) of acoustic signals.
• the audio signal processing by the signal processing device 10 may further include, for example, an audio signal amplification process similar to that of an AV amplifier.
• the signal processing device 10 sends an output acoustic signal generated by acoustic signal processing to the speaker 22.
• the signal processing device 10 is an example of an information processing device.
• the signal processing device 10 may be a controller built into the earphone 1 with a microphone, or may be a general-purpose information processing device (for example, a smartphone, or a PC (tablet PC, etc.) that has the function of controlling the microphone 21 and the speaker 22 (including the same hereinafter)).
• a general-purpose information processing device for example, a smartphone, or a PC (tablet PC, etc.
• the signal processing device 10 includes a storage device 11, a processor 12, an input/output interface 13, and a communication interface 14.
• the storage device 11 is configured to store programs and data.
• the storage device 11 is, for example, a combination of ROM (Read Only Memory), RAM (Random Access Memory), and storage (for example, flash memory or hard disk).
• the programs include, for example, the following programs.
• ⁇ OS (Operating System) program ⁇ Application program that executes information processing (e.g. content playback application)
• the data includes, for example, the following data. ⁇ Databases referenced in information processing ⁇ Data obtained by executing information processing (that is, execution results of information processing)
• the processor 12 is a computer that implements the functions of the signal processing device 10 by activating a program stored in the storage device 11.
• the processor 12 is, for example, at least one of the following. ⁇ CPU (Central Processing Unit) ⁇ GPU (Graphic Processing Unit) ⁇ ASIC (Application Specific Integrated Circuit) ⁇ FPGA (Field Programmable Array)
• the input/output interface 13 acquires information (for example, a user's instruction) from an input device connected to the signal processing device 10 and transmits information (for example, an acoustic signal, a control signal) to an output device connected to the signal processing device 10. , or an image signal).
• information for example, a user's instruction
• information for example, an acoustic signal, a control signal
• An interface capable of transmitting an audio signal is, for example, SPDIF (Sony Philips Digital Interface), HDMI (registered trademark) (High-Definition Multimedia Interface), a pin connector (RCA pin), or an audio interface for headphones.
• the interface may be a wireless interface using Bluetooth (registered trademark) or the like.
• the acoustic signal in this embodiment includes either or both of an analog signal and a digital signal.
• the communication interface 14 is configured to control communication between the signal processing device 10 and an external device (for example, a sound source device not shown).
• the microphone 21 collects surrounding sounds.
• the microphone 21 outputs an electrical signal (hereinafter referred to as an "external sound signal") according to the received sound.
• the earphone 1 with a microphone may be configured to be wearable on the user's head, and in this case, the microphone 21 may be configured to be located near the user's ear when the earphone 1 with a microphone is worn.
• Microphone 21 is typically a headphone or earphone microphone with active noise canceling, or a hearing aid microphone.
• the external sound signal includes a signal component due to the person's speech and a component due to other environmental sounds.
• Other environmental sounds include, for example, noises such as vehicle running sounds, construction noises, and sounds emitted by equipment and equipment.
• noise may differ depending on the user's intention in using the microphone-equipped earphone 1. For example, if a user wants to suppress environmental sounds but still be aware when others are speaking to them, human voices may be excluded from the noise. On the other hand, if the user wants to suppress not only environmental sounds but also other people's conversation sounds, or if the user wants to concentrate on listening to audio content, human voices may also be included in the noise.
• the speaker 22 generates sound according to the output acoustic signal obtained from the signal processing device 10.
• the earphone 1 with a microphone may be configured to be able to be worn on the user's head, and in this case, the speaker 22 is located near the user's ear when the earphone 1 with a microphone is worn, and is configured to emit sound toward the ear. may be configured.
• Speaker 22 is typically a headphone, earphone, or hearing aid speaker.
• the earphone 1 with a microphone is configured to be able to be worn on the user's head, and the speaker 22 is configured to be located near the user's ear when the earphone 1 with a microphone is worn, and to emit sound toward the user's ear.
• the speaker 22 may be a loudspeaker (which may include a powered speaker) or a smart speaker.
• the signal processing device 10, the microphone 21, and the speaker 22 may be partially or entirely integrated, or may be configured as mutually independent devices.
• the earphone with microphone 1 is worn on the head of the user US1.
• the microphone 21-1 and the speaker 22-1 are located near the left ear of the user US1 (for example, outside or inside the ear pad that covers the left ear) when the earphone 1 with microphone is worn.
• the microphone 21-2 and the speaker 22-2 are located near the right ear of the user US1 (eg, outside or inside the ear pad that covers the right ear) when the earphone 1 with microphone is worn.
• the microphones 21-1 and 21-2 collect ambient sounds, generate external sound signals, and output the external sound signals to the signal processing device 10.
• the signal processing device 10 acquires an external sound signal based on sound collected by the microphones 21-1 and 21-2, and performs modulation processing including amplitude modulation on the external sound signal. Thereby, the signal processing device 10 generates an acoustic signal having a volume intensity corresponding to the frequency of the gamma wave.
• the signal processing device 10 outputs the generated modulated acoustic signal to the speakers 22-1 and 22-2.
• the speakers 22-1 and 22-2 emit sound according to the modulated acoustic signal toward the ears of the user US1.
• the modulated acoustic signal has a change in amplitude corresponding to the frequency of gamma waves
• gamma waves are induced in the brain of the user US1 (electroencephalogram). is synchronized to the gamma frequency).
• the effect of improving cognitive function for example, treating or preventing dementia
• the earphone 1 with a microphone of this embodiment generates a modulated acoustic signal based on the sound picked up by the microphones 21-1 and 21-2, the user US1 wears the earphone 1 with a microphone.
• FIG. 3 is a flowchart of acoustic signal processing according to this embodiment.
• FIG. 4 is a flowchart showing an example of signal processing for an external sound signal.
• FIG. 5 is a diagram illustrating the waveform of an acoustic signal handled in the signal processing of FIG. 4.
• FIG. 6 is a flowchart showing an example of signal processing for a reproduced sound signal.
• the acoustic signal processing in FIG. 3 is realized by the processor 12 of the signal processing device 10 reading and executing a program stored in the storage device 11.
• the acoustic signal processing in FIG. 3 starts when any of the following start conditions is met. -
• the acoustic signal processing shown in FIG. 3 is called by another process or an external instruction.
• the user performed an operation to call up the acoustic signal processing shown in FIG. -
• the signal processing device 10 enters a predetermined state (for example, the power is turned on).
• the specified date and time has arrived.
• a predetermined amount of time has passed since a predetermined event (for example, activation of the signal processing device 10 or previous execution of the acoustic signal processing in FIG. 3).
• the signal processing device 10 acquires an input acoustic signal (S110). Specifically, the signal processing device 10 acquires an input acoustic signal.
• the input acoustic signal can include at least one of the following.
• ⁇ External sound signal ⁇ Acoustic signal for playing audio content hereinafter referred to as "playback sound signal"
• the reproduced sound signal corresponds to at least one of the following audio contents, for example.
• ⁇ Music content e.g., singing, playing, or a combination thereof (i.e., a song); may include audio content that accompanies video content.
• ⁇ Audio content for example, audio of recitation, narration, announcement, broadcast play, solo performance, conversation, soliloquy, or a combination thereof.
• ⁇ Other audio content e.g. electronic, environmental, or mechanical sounds
• singing or audio content is not limited to sounds emitted by human vocal organs, but may include sounds generated by speech synthesis technology.
• the signal processing device 10 acquires an external sound signal from the microphone 21.
• the signal processing device 10 acquires a reproduced sound signal from a sound source device (not shown).
• the signal processing device 10 acquires a reproduced audio signal by reproducing an audio content file received from a content distribution device (not shown).
• the signal processing device 10 acquires a reproduced sound signal by reproducing an audio content file stored in a storage medium (for example, a disk medium).
• the fifth example of acquiring the input acoustic signal (S110) is a combination of at least two of the first to fourth examples.
• the signal processing device 10 may be configured to obtain an input acoustic signal depending on the operating mode.
• the operation mode may be determined based on a user's input operation or an external instruction, or may be determined by an algorithm.
• the signal processing device 10 may further perform A/D conversion of the input acoustic signal.
• the signal processing device 10 After step S110, the signal processing device 10 generates an output acoustic signal (S111). Specifically, the signal processing device 10 generates an output acoustic signal by performing amplitude modulation on at least a portion of the input acoustic signal acquired in step S110.
• the signal processing device 10 As a first example of generating an output acoustic signal (S111), the signal processing device 10 generates an intermediate acoustic signal including a plurality of acoustic signals having different characteristics based on the input acoustic signal.
• the features may be determined based on input operations by the user or instructions from the outside, or may be determined by an algorithm.
• the signal processing device 10 may determine the characteristics for generating the intermediate acoustic signal based on the result of analyzing the input acoustic signal.
• the feature may be, for example, at least one of the following: ⁇ Sound characteristics (especially qualitative characteristics) ⁇ Frequency characteristics ⁇ Time characteristics ⁇ Amplitude characteristics (e.g. volume) ⁇ Output characteristics
• the signal processing device 10 selects one or more acoustic signals to which amplitude modulation is applied (hereinafter referred to as "target signal") from a plurality of acoustic signals included in the intermediate acoustic signal. Which acoustic signal to select as the target signal may be determined based on an input operation by the user or an instruction from the outside, or may be determined by an algorithm. For example, the signal processing device 10 may determine the target signal based on the characteristics of the acoustic signal (balance between voice and music, change in volume, type of music, timbre, or other characteristics).
• the signal processing device 10 can select a target signal so that the effect of improving cognitive function by modulation is higher, or select a target signal so that the sense of discomfort given to the user is lowered.
• the signal processing device 10 may treat all of the plurality of acoustic signals included in the intermediate acoustic signal as target signals.
• the signal processing device 10 performs amplitude modulation on the selected target signal.
• the signal processing device 10 performs amplitude modulation on the target signal using a modulation function having a frequency corresponding to gamma waves (for example, a frequency of 35 Hz or more and 45 Hz or less).
• A(t) is a modulation function with periodicity of 35 Hz or more and 45 Hz or less
• X(t) is a function representing the waveform of an acoustic signal before modulation
• the signal processing device 10 may determine the modulation function used for amplitude modulation of the target signal or the modulation degree of the amplitude modulation.
• the modulation function or modulation degree may be determined based on an input operation by a user or an instruction from the outside, or may be determined by an algorithm.
• the modulation function or modulation depth may be determined by an algorithm based on characteristics of the acoustic signal.
• the signal processing device 10 may determine a common modulation function or modulation degree for a plurality of target signals, or may determine a modulation function or a modulation degree individually for a plurality of target signals. .
• the signal processing device 10 generates an output sound based on the modulated target signal and the sound signal that is not selected as the target signal among the plurality of sound signals included in the intermediate sound signal (hereinafter referred to as "non-target signal”). Generate a signal. That is, the signal processing device 10 converts the non-target signal and the modulated target signal into an output acoustic signal. Specifically, the signal processing device 10 combines two or more acoustic signals of the non-target signal and the modulated target signal, or extracts or separates an acoustic signal from at least one of the non-target signal and the modulated target signal. I do things.
• the method of synthesizing the acoustic signals is not limited, but may include, for example, signal summation processing, HRTF (Head Related Transfer Function) convolution processing, transfer function convolution processing that provides position information of the sound source, or any of these convolution processing. Processing for summing later may be included. Further, the signal processing device 10 may further perform at least one of amplification, volume adjustment, or D/A conversion of the output acoustic signal. On the other hand, if the non-target signal and the modulated target signal match the output format of the speaker 22 (for example, the non-target signal and the modulated target signal are multi-channel audio associated with each speaker constituting the surround system as the speaker 22). (corresponding to a signal), no such conversion is necessary. In this case, the non-target signal and the modulated target signal are treated as output acoustic signals.
• HRTF Head Related Transfer Function
• the input acoustic signal includes a plurality of acoustic signals with different characteristics.
• the signal processing device 10 selects one or more acoustic signals to which amplitude modulation is applied from a plurality of acoustic signals included in the input acoustic signal.
• the second example of generating the output acoustic signal (S111) can be understood by appropriately replacing the "intermediate acoustic signal" with the "input acoustic signal" in the description of the first example.
• the signal processing device 10 performs amplitude modulation on the input acoustic signal.
• the amplitude modulation for the input acoustic signal is similar to the amplitude modulation for the target signal described in the first example of output acoustic signal generation (S111).
• a change in amplitude corresponding to the frequency of the gamma wave is added to the input acoustic signal.
• the signal processing device 10 may determine the modulation function used for amplitude modulation of the input acoustic signal or the modulation degree of the amplitude modulation.
• the modulation function or modulation degree may be determined based on an input operation by a user or an instruction from the outside, or may be determined by an algorithm.
• the signal processing device 10 generates an output acoustic signal based on the modulated input acoustic signal. That is, the signal processing device 10 converts the modulated input audio signal into an output audio signal. Specifically, when the modulated input acoustic signal consists of a plurality of acoustic signals, the signal processing device 10 combines two or more of the plurality of acoustic signals, extracts an acoustic signal from the modulated input acoustic signal, or to separate. The details of the acoustic signal synthesis method are as described in the first example of output acoustic signal generation (S111).
• the modulated input audio signal matches the output format of the speaker 22 (for example, if the modulated input audio signal corresponds to a multi-channel audio signal associated with each speaker constituting the surround system as the speaker 22) , no such conversion is necessary. In this case, the modulated input audio signal is treated as the output audio signal.
• the signal processing device 10 can be generated by individually processing the signals to generate a modulated target signal or a non-target signal, and then combining these signals, for example.
• the signal processing device 10 may perform signal processing of the external sound signal as shown in FIG. 4 in S111.
• the signal processing device 10 when the signal processing device 10 is set to an operation mode in which modulation of external sound is effective, it is possible to acquire an input acoustic signal including an external sound signal in step S110 and execute the signal processing shown in FIG. 4. .
• the signal processing device 10 acquires an external sound signal (S201). Specifically, the signal processing device 10 acquires an extraneous sound signal included in the input acoustic signal acquired in step S110.
• the signal processing device 10 executes noise signal extraction (S202). Specifically, the signal processing device 10 extracts a signal component corresponding to noise (hereinafter referred to as a "noise signal") from the external sound signal acquired in step S201.
• Signal processing device 10 may extract the noise signal using any technique. As an example, the signal processing device 10 may extract a frequency component excluding the human voice frequency from the external sound signal as a noise signal, or a signal corresponding to a sound in a specific direction of arrival or a signal corresponding to a specific arrival direction.
• a signal corresponding to a sound other than a report may be extracted as a noise signal, or an extraneous sound signal may be treated as a noise signal as it is. Note that frequency components outside the human audible range may be excluded from the noise signal.
• the signal processing device 10 After step S202, the signal processing device 10 generates an antiphase sound signal (S203). Specifically, the signal processing device 10 generates an antiphase sound signal by inverting the phase of the noise signal extracted in step S202.
• the signal processing device 10 performs modulation of the antiphase sound signal (S204). Specifically, the signal processing device 10 performs amplitude modulation on the anti-phase sound signal (or the intermediate sound signal based on this) generated in step S203. As an example, the signal processing device 10 performs amplitude modulation on the antiphase sound signal using a modulation function having a frequency corresponding to gamma waves (for example, a frequency of 35 Hz or more and 45 Hz or less). Note that the processes in S203 and S204 may be performed in the reverse order. That is, the signal processing device 10 may perform amplitude modulation on the noise signal and then invert the phase.
• the modulated acoustic signal A12 generated in this way has an opposite phase to the noise signal A11 picked up by the microphone 21 near the user's ear, and has a frequency of gamma waves. It has a corresponding volume intensity. Therefore, in the acoustic signal A13 obtained by superimposing the noise signal A11 and the modulated acoustic signal A12, the components originating from the noise signal A11 are ideally reduced by about half on average, and the volume corresponding to the frequency of the gamma wave is reduced. It has different strengths and weaknesses.
• the signal processing device 10 acquires an input acoustic signal including an external sound signal in step S110, and acquires an input acoustic signal including an external sound signal in step S111.
• steps S201 to S203 can be executed.
• the speaker 22 emits a sound (hereinafter referred to as "anti-phase sound") corresponding to the anti-phase sound signal (which is not amplitude modulated), so the user hears a synthesized sound of the noise and the anti-phase sound. .
• the out-of-phase sound cancels out the noise completely, so the volume of the synthesized sound is zero.
• the signal processing device 10 may perform signal processing of the reproduced sound signal in S111 as shown in FIG. 6, for example.
• the signal processing device 10 when the signal processing device 10 is set to an operation mode for reproducing audio content, it is possible to acquire an input audio signal including a reproduced sound signal in step S110 and execute the signal processing shown in FIG. 6.
• the signal processing device 10 acquires a reproduced sound signal (S301). Specifically, the signal processing device 10 acquires a reproduced sound signal included in the input audio signal acquired in step S110.
• the signal processing device 10 modulates the reproduced sound signal (S302). Specifically, the signal processing device 10 determines that modulation of the reproduced sound signal is necessary when the operation mode is set to enable modulation of the audio content and the reproduced sound signal is not modulated. . The signal processing device 10 may determine whether the reproduced sound signal is modulated, for example, based on meta information corresponding to the audio content. The meta-information may be distributed together with or separately from the audio content file, or stored on a storage medium along with the audio content file, for example.
• the signal processing device 10 performs amplitude modulation on the reproduced sound signal acquired in step S301.
• the signal processing device 10 performs amplitude modulation on a reproduced sound signal (or an intermediate sound signal based thereon) using a modulation function having a frequency corresponding to gamma waves (for example, a frequency of 35 Hz or more and 45 Hz or less). I do.
• the signal processing device 10 determines that modulation of the reproduced sound signal is not necessary if the operation mode is set to disable modulation of the audio content or if the reproduced sound signal has already been modulated. In this case, the signal processing device 10 skips modulation of the reproduced sound signal (S302).
• the signal processing device 10 transmits an output acoustic signal (S112). Specifically, the signal processing device 10 sends the output acoustic signal generated in step S111 to the speaker 22.
• the speaker 22 generates sound according to the output acoustic signal.
• the signal processing device 10 ends the acoustic signal processing of FIG. 3 at step S112.
• the signal processing device 10 may repeatedly perform the audio signal processing shown in FIG. 3 every predetermined reproduction section (for example, every 100 ms) of the input audio signal.
• the signal processing device 10 may continuously perform modulation processing on the input acoustic signal, such as modulation by analog signal processing, and output a modulated acoustic signal.
• the acoustic signal processing shown in FIG. You may terminate it accordingly.
• the signal processing device 10 of this embodiment acquires an input acoustic signal (external sound signal) based on the sound picked up by the microphone 21, and performs amplitude modulation on the input acoustic signal.
• an output acoustic signal having a volume intensity corresponding to the frequency of the gamma wave is generated, and the output acoustic signal is sent to the speaker 22.
• external sounds can be used as a sound base for improving cognitive function, so the user can improve cognitive function by simply wearing and operating the earphones 1 with a microphone, without having to play back audio content. You can listen to sounds for improvement. In other words, it is possible to promote listening to sounds based on amplitude-modulated acoustic signals while reducing the burden felt by the user.
• the modulation process may include a process of inverting the phase of the acoustic signal.
• an external sound signal for example, a noise signal
• the present embodiment is not limited to the feedforward type noise canceling as described above, but can also be applied to other types of noise canceling such as a feedback type.
• the modulation process by the signal processing device 10 includes dynamically changing parameters in order to invert the phase of only the component corresponding to the external sound from an acoustic signal containing the external sound and the sound of the reproduced audio content. It may also include processing for changing.
• the signal processing device 10 may control modulation processing based on at least one of the volume and characteristics of the input acoustic signal. Thereby, modulation processing suitable for the volume or characteristics of the input acoustic signal can be performed.
• the output acoustic signal may have a change in amplitude corresponding to a frequency of 35 Hz or more and 45 Hz or less. Thereby, gamma waves can be induced in the user's brain, and the effect of improving cognitive function can be obtained.
• Modification 1 is obtained by separating the audio signal from the external sound signal when a signal component corresponding to a human voice included in the external sound signal (hereinafter referred to as "audio signal") satisfies a predetermined condition.
• audio signal a signal component corresponding to a human voice included in the external sound signal
• residual signal an acoustic signal
• FIG. 7 is a flowchart showing an example of signal processing for an external sound signal.
• FIG. 8 is a diagram illustrating the waveform of an acoustic signal handled in the signal processing of FIG. 7.
• the acoustic signal processing of Modification 1 is realized by the processor 12 of the signal processing device 10 reading and executing a program stored in the storage device 11.
• the acoustic signal processing of Modification 1 may be started in response to establishment of any of the same start conditions as in this embodiment.
• the signal processing device 10 acquires an input acoustic signal (S110) similarly to this embodiment.
• step S110 the signal processing device 10 executes generation of an output acoustic signal (S111) similarly to this embodiment.
• the signal processing device 10 may perform signal processing of the external sound signal as shown in FIG. 7 in S111.
• the signal processing device 10 acquires an input acoustic signal including an external sound signal in step S110, and performs the signal processing of FIG. is possible.
• the signal processing device 10 acquires an external sound signal (S401). Specifically, the signal processing device 10 acquires an extraneous sound signal included in the input acoustic signal acquired in step S110.
• the signal processing device 10 performs audio signal separation (S402). Specifically, the signal processing device 10 separates a signal component corresponding to a human voice (that is, an audio signal) from the external sound signal acquired in step S401.
• Signal processing device 10 may extract the audio signal using any technique. As an example, the signal processing device 10 may separate the frequency component of a human voice from the external sound signal as an audio signal.
• the signal processing device 10 determines, as a predetermined condition, whether or not the audio signal includes a component corresponding to a voice directed toward the user (for example, a call from a surrounding person, an announcement for guidance, etc.). Determine whether As an example, the signal processing device 10 determines that the audio signal includes a voice addressed to the user when the level of the audio signal is equal to or higher than a threshold value.
• the signal processing device 10 performs audio signal enhancement (S403). Specifically, the signal processing device 10 performs signal processing (for example, amplification) to enhance the audio signal separated in step S402. Note that if the signal processing device 10 is not set to an operation mode in which human voice enhancement is effective, the signal processing device 10 can skip the audio signal enhancement (S403).
• the signal processing device 10 modulates the residual signal (S404). Specifically, the signal processing device 10 performs amplitude modulation on the residual acoustic signal (residual signal) obtained by separating the audio signal from the external sound signal in step S402. However, frequency components outside the human audible range may be excluded from the residual signal. As an example, as shown in FIG. 8, the signal processing device 10 performs amplitude modulation on the residual signal A21 using a modulation function having a frequency corresponding to gamma waves (for example, a frequency of 35 Hz or more and 45 Hz or less). Then, a modulated acoustic signal A22 is generated. In addition, when the earphone 1 with a microphone corresponds to a hearing aid, the signal processing device 10 may amplify and amplitude modulate the residual signal by changing the amplification gain of the hearing aid at 40 Hz.
• the signal processing device 10 may amplify and amplitude modulate the residual signal by changing the amplification gain of the hearing aid
• the modulated acoustic signal generated in this way has a volume intensity corresponding to the frequency of the gamma wave. Therefore, according to the signal processing shown in FIG. 7, it is possible to make the user listen to sounds for improving cognitive function by using sounds other than human voices among external sounds.
• the signal processing device 10 may generate an acoustic signal having an opposite phase to the residual signal and modulate the acoustic signal.
• the signal processing device 10 After step S404, the signal processing device 10 performs audio signal synthesis (S405). Specifically, the signal processing device 10 synthesizes the emphasized audio signal generated in step S403 and the modulated residual signal generated in step S404.
• the signal processing device 10 ends the acoustic signal processing of the first modification at step S112.
• the signal processing device 10 may repeatedly perform the audio signal processing of Modification 1 every predetermined reproduction section (for example, every 100 ms) of the input audio signal.
• the signal processing device 10 may continuously perform modulation processing on the input acoustic signal, such as modulation by analog signal processing, and output a modulated acoustic signal.
• the acoustic signal processing of Modification 1 may be terminated according to the same termination conditions as in this embodiment.
• the signal processing device 10 of Modification 1 modulates the residual signal when the audio signal included in the external sound signal satisfies a predetermined condition, and when it does not, modulates the residual signal. modulates the external sound signal.
• external sounds can be used as a sound base for improving cognitive function, so the user can improve cognitive function by simply wearing the earphones with microphone 1 and operating them, without having to play back audio content. You can listen to sounds for improvement.
• the voice signal is excluded from the modulation processing, so the modulation processing for the user's hearing of these voices is The negative effects of implementation can be suppressed.
• the modulation process may include a process of amplifying an acoustic signal (for example, an audio signal). This makes it easier for the user to hear people's voices.
• an acoustic signal for example, an audio signal
• the signal processing device 10 may separate the external sound signal into an audio signal and a residual signal, amplitude-modulate the residual signal, and synthesize the modulated residual signal and the audio signal.
• the parts of the foreign sound that do not correspond to the human voice can be used as a sound base for improving cognitive function. can do.
• Modification 2 is an example in which, when the external sound signal does not satisfy a predetermined condition, another modulated acoustic signal is output instead of modulating the external sound signal.
• FIG. 9 is a flowchart showing an example of signal processing for an external sound signal.
• FIG. 10 is a diagram illustrating the waveform of an acoustic signal handled in the signal processing of FIG. 9.
• the acoustic signal processing of the second modification is realized by the processor 12 of the signal processing device 10 reading and executing a program stored in the storage device 11.
• the acoustic signal processing of Modification 2 may be started in response to establishment of any of the same start conditions as in this embodiment.
• step S110 the signal processing device 10 executes generation of an output acoustic signal (S111) similarly to this embodiment.
• the signal processing device 10 may perform signal processing of the external sound signal as shown in FIG. 9 in S111.
• the signal processing device 10 when the signal processing device 10 is set to an operation mode in which modulation of external sound is effective, it is possible to acquire an input acoustic signal including an external sound signal in step S110 and execute the signal processing shown in FIG. 9. .
• the signal processing device 10 determines, for example, whether the volume of the external sound is sufficiently loud. As an example, the signal processing device 10 determines that the volume of the external sound is sufficiently large when the volume of the external sound signal exceeds a predetermined value.
• the signal processing device 10 modulates the external sound signal (S502). Specifically, the signal processing device 10 performs amplitude modulation on the external sound signal (or the intermediate sound signal based on the external sound signal) acquired in step S501. As an example, the signal processing device 10 performs amplitude modulation on the external sound signal using a modulation function having a frequency corresponding to gamma waves (for example, a frequency of 35 Hz or more and 45 Hz or less). Note that when the microphone-equipped earphone 1 corresponds to a hearing aid, the signal processing device 10 may amplify and amplitude modulate the external sound signal by changing the amplification gain of the hearing aid at 40 Hz.
• the signal processing device 10 may perform at least one of the following signal processing instead of modulating the external sound signal (S502). ⁇ Extraction of noise signal (S202), generation of anti-phase sound signal (S203), and modulation of anti-phase sound signal (S204) - Separation of audio signals (S402) and enhancement of audio signals (S403) ⁇ Separation of audio signals (S402), modulation of residual signals (S404), and synthesis of audio signals (S405)
• the signal processing device 10 acquires a modulated reproduced sound signal from a sound source device (not shown).
• the signal processing device 10 acquires a modulated reproduced audio signal by reproducing an audio content file received from a content distribution device (not shown).
• the signal processing device 10 acquires a modulated reproduced audio signal by reproducing an audio content file stored in a storage medium (for example, a disk medium). .
• the signal processing device 10 acquires a reproduced sound signal by reproducing an audio content file stored in a storage medium (for example, a disk medium), and acquires the reproduced sound signal. By modulating the sound signal, a modulated playback sound signal is obtained.
• the seventh example of obtaining the modulated acoustic signal (S503) is a combination of at least two of the first to sixth examples.
• step S111 the signal processing device 10 sends out an output acoustic signal (S112) similarly to this embodiment.
• the signal processing device 10 ends the acoustic signal processing of the second modification at step S112.
• the signal processing device 10 may repeatedly perform the acoustic signal processing of Modification 2 every predetermined reproduction section (for example, every 100 ms) of the input acoustic signal.
• the signal processing device 10 may continuously perform modulation processing on the input acoustic signal, such as modulation by analog signal processing, and output a modulated acoustic signal.
• the acoustic signal processing of Modification 2 may be terminated according to the same termination conditions as in this embodiment.
• the storage device 11 may be connected to the signal processing device 10 via the network NW.
• each step of audio signal processing is executed in a specific order, but the execution order of each step is not limited to the example described as long as there is no dependency relationship.
• each step of the acoustic signal processing described above may be shared between the signal processing device 10 and an external device (for example, a server (not shown) or a host device such as a smartphone).
• the signal processing device 10 may be connected to an arbitrary number of speakers 22, and in this case, the signal processing device 10 may be configured to be able to select which speaker 22 the output acoustic signal is sent to.
• the signal processing device 10 may be connected to an arbitrary number of microphones 21, and in this case, the signal processing device 10 may be configured to be able to select which microphone 21 to acquire the external sound signal from.
• An algorithm for determining the method of generating the output acoustic signal (e.g. modulation method, combination ratio of two or more acoustic signals, method of determining the acoustic signal to which amplitude modulation is applied, etc.) or a default method of generating the output acoustic signal may be changed by updating the firmware of the signal processing device 10.
• the firmware is updated, for example, by the signal processing device 10 communicating with an external device (for example, a server, not shown).
• the update contents of the firmware of the signal processing device 10 may be notified to the user in advance, and the signal processing device 10 may provide an operation mode in which the user can listen to and compare the sounds before and after the update.
• the signal processing device 10 generates an output audio signal based on a modulated audio signal based on an external sound signal and an unmodulated reproduced sound signal (for example, an audio signal based on audio data such as audio content). Can be generated.
• the signal processing device 10 may control the modulation process for the acoustic signal based on the external sound signal based on at least one of the volume and the characteristics of the reproduced sound signal. For example, when the volume of the reproduced sound signal is larger than the threshold value, the signal processing device 10 outputs a modulated anti-phase signal obtained by modulating a signal with the opposite phase of the external sound signal to the speaker 22, so that the volume of the reproduced sound signal increases.
• a signal with the opposite phase of the external sound signal may be output to the speaker 22.
• the signal processing device 10 outputs a signal with the opposite phase of the external sound signal to the speaker 22 when the volume of the reproduced sound signal is higher than the threshold, and when the volume of the reproduced sound signal is lower than the threshold, , a modulated anti-phase signal obtained by modulating a signal with an anti-phase of the external sound signal may be output to the speaker 22.
• modulation processing on the acoustic signal based on the external sound signal within a range that does not hinder the user's appreciation of the acoustic content, and to allow the user to listen to sounds for improving cognitive function along with the acoustic content.
• the signal processing device 10 generates a modulated acoustic signal based on an external sound signal (hereinafter referred to as a "first modulated acoustic signal”) and a modulated acoustic signal based on a reproduced sound signal (hereinafter referred to as a "second modulated acoustic signal").
• the output acoustic signal can be generated based on the modulated acoustic signal.
• the signal processing device 10 may synchronize the first modulated audio signal and the second modulated audio signal with respect to the volume strength.
• the signal processing device 10 adds the signals after controlling the phases of the first modulated acoustic signal and the second modulated acoustic signal so that the amplitude peaks of these signals overlap, and outputs the combined signal to the speaker 22. You can also output to As a result, it is possible to prevent the intensity of the volume corresponding to the frequency of the gamma wave from being distorted due to the superposition of the first modulated acoustic signal and the second modulated acoustic signal, thereby further enhancing the effect of improving cognitive function. Can be done.
• the signal processing device 10 may control the modulation process according to information (for example, a measurement result of the user's hearing ability) obtained based on a measurement of the user's hearing ability (for example, performed at the time of hearing aid fitting). For example, the signal processing device 10 may control at least one of the following. ⁇ Modulation degree and modulation function for each frequency ⁇ Amplification gain of the acoustic signal in audio signal enhancement (S403) ⁇ In the acoustic signal processing of modification 2 (FIG. 9), it is determined whether the volume of the external sound is sufficiently large. By controlling the modulation process in this way, it is possible to make the user listen to a sound that is suitable for his or her hearing.
• the earphone 1 with a microphone described in this embodiment and each modification can also be applied to a cochlear implant.
• a similar effect can be achieved by supplying an electrical signal for a cochlear implant generated based on the modulated acoustic signal to an electrode implanted in the user's cochlea. can get.
• the acoustic signal picked up by the microphone may be amplitude-modulated and then converted into an electric signal for the cochlear implant and supplied to the electrode, or the sound picked up by the microphone may be The signal may be converted into an electrical signal for a cochlear implant and then modulated (eg, amplitude modulated, pulse density modulated or pulse width modulated) and supplied to the electrodes.
• modulated eg, amplitude modulated, pulse density modulated or pulse width modulated
• the earphone 1 with a microphone described in this embodiment and each modification can also be applied to a bone conduction earphone, a cartilage conduction earphone, an air conduction ultrasound speaker (parametric speaker), a bone conduction ultrasound speaker, or a skin vibration speaker. It is.
• the carrier wave When applied to an ultrasonic speaker, the carrier wave may be modulated based on the sound source signal and then amplitude-modulated at a specific frequency, or the carrier wave may be amplitude-modulated at a specific frequency and then modulated based on the sound source signal.
• Earphone with microphone 10 Signal processing device 11: Storage device 12: Processor 13: Input/output interface 14: Communication interface 21: Microphone 22: Speaker

Landscapes

• Physics & Mathematics (AREA)
• Engineering & Computer Science (AREA)
• Acoustics & Sound (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Headphones And Earphones (AREA)
• Circuit For Audible Band Transducer (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (1)

Family

ID=89383040

Family Applications (1)

Country Status (2)

Citations (6)

• 2023
  • 2023-06-26 WO PCT/JP2023/023524 patent/WO2024004925A1/ja not_active Ceased
  • 2023-06-26 JP JP2024530822A patent/JPWO2024004925A1/ja active Pending

Patent Citations (6)

Also Published As

Similar Documents

Legal Events