JP2022054318A - Sound processor, control method, and program - Google Patents

Abstract

To effectively reduce noise.SOLUTION: A sound processor comprises: a first microphone for acquiring environmental sounds; a second microphone for acquiring sounds from a noise source; first transform means for performing Fourier transform on sound signals from the first microphone to generate first sound signals; second transform means for performing Fourier transform on sound signals from the second microphone to generate second sound signals; detection means for detecting a type of the noise generated from the noise source; generation means for generating noise data using the second sound signals and a parameter related to the noise of the noise source, in accordance with the type of noise detected by the detection means; subtraction means for subtracting the noise data from the first sound signals; and third transform means for performing inverse Fourier transform on sound signals from the subtraction means.SELECTED DRAWING: Figure 3

Description

The present invention relates to a voice processing device capable of reducing noise contained in voice data.

When recording moving image data, a digital camera, which is an example of an audio processing device, can also record ambient audio. In addition, the digital camera has an autofocus function that focuses on the subject during recording of moving image data by driving an optical lens. In addition, the digital camera has a function of driving an optical lens to perform zooming while recording a moving image.

As described above, when the optical lens is driven during the recording of the moving image, the driving sound of the optical lens may be included as noise in the sound recorded together with the moving image. Therefore, conventionally, when a digital camera collects a sliding sound or the like generated when an optical lens is driven as noise, the noise can be reduced and the surrounding sound can be recorded. Patent Document 1 discloses a digital camera that reduces noise by a spectral subtraction method.

Japanese Unexamined Patent Publication No. 2011-205527

However, in Patent Document 1, since the digital camera creates a noise pattern from the noise collected by the microphone that records the surrounding sound, an accurate noise pattern is acquired from the sliding noise generated in the housing of the optical lens. It may not be possible. In this case, the digital camera may not be able to effectively reduce the noise contained in the collected sound.

Therefore, an object of the present invention is to effectively reduce noise.

The voice processing device of the present invention Fourier-converts a first microphone for acquiring environmental sound, a second microphone for acquiring sound from a noise source, and a voice signal from the first microphone. Generated from the noise source, the first conversion means for generating the first audio signal, the second conversion means for generating the second audio signal by Fourier transforming the audio signal from the second microphone, and the noise source. Noise data is generated using the detection means for detecting the type of noise generated, the second audio signal, and the parameters related to the noise of the noise source, depending on the type of noise detected by the detection means. It is characterized by having a generation means, a subtraction means for subtracting the noise data from the first voice signal, and a third conversion means for inverse Fourier transforming the voice signal from the subtraction means.

The voice processing apparatus of the present invention can effectively reduce noise.

It is a perspective view of the image pickup apparatus in 1st Example. It is a block diagram which shows the structure of the image pickup apparatus in 1st Example. It is a block diagram which shows the structure of the audio input part of the image pickup apparatus in the 1st Example. It is a figure which shows the arrangement of the microphone in the audio input part of the image pickup apparatus in the 1st Example. It is a figure which shows the noise parameter in 1st Example. It is a figure which shows the frequency spectrum of voice, and the frequency spectrum of a noise parameter in the case where the driving sound is generated in the situation which can be considered that there is no environmental sound in 1st Example. It is a figure which shows the frequency spectrum of the voice in the case where the driving sound is generated in the situation which there is an environmental sound in the 1st Example. It is a block diagram which shows the structure of the noise parameter selection part in 1st Example. It is a timing chart related to the voice noise reduction processing in the 1st Example. It is a block diagram which shows the structure of the audio input part of the image pickup apparatus in the 2nd Example.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[First Example]
<External view of the image pickup device 100>
1A and 1B show an example of an external view of an image pickup apparatus 100 as an example of an audio processing apparatus to which the present invention can be applied. FIG. 1A is an example of a front perspective view of the image pickup apparatus 100. FIG. 1B is an example of a rear perspective view of the image pickup apparatus 100. In FIG. 1, an optical lens (not shown) is attached to the lens mount 301.

The display unit 107 displays image data, character information, and the like. The display unit 107 is provided on the back surface of the image pickup apparatus 100. The display unit 43 outside the viewfinder is a display unit provided on the upper surface of the image pickup apparatus 100. The out-of-finder display unit 43 displays the set values of the image pickup apparatus 100 such as the shutter speed and the aperture value. The eyepiece finder 16 is a peep-type finder. The user can confirm the focus and composition of the optical image of the subject by observing the focusing screen in the eyepiece finder 16.

The release switch 61 is an operating member for the user to give a shooting instruction. The mode changeover switch 60 is an operation member for the user to switch between various modes. The main electronic dial 71 is a rotation operation member. The user can change the setting values of the image pickup apparatus 100 such as the shutter speed and the aperture value by turning the main electronic dial 71. The release switch 61, the mode changeover switch 60, and the main electronic dial 71 are included in the operation unit 112.

The power switch 72 is an operating member that switches the power of the image pickup apparatus 100 on and off. The sub electronic dial 73 is a rotation operation member. The user can move the selection frame displayed on the display unit 107 by the sub-electronic dial 73, advance the image in the reproduction mode, and the like. The cross key 74 is a cross key (four-way key) capable of pushing up, down, left, and right portions, respectively. The image pickup apparatus 100 executes processing according to the pressed portion (direction) of the cross key 74. The power switch 72, the sub electronic dial 73, and the cross key 74 are included in the operation unit 112.

The SET button 75 is a push button. The SET button 75 is mainly used for the user to determine a selection item displayed on the display unit 107. The LV button 76 is a button used to switch the live view (hereinafter referred to as LV) on and off. The LV button 76 is used to instruct to start and stop moving image recording (recording) in the moving image recording mode. The magnifying button 77 is a push button for turning on and off the magnifying mode and changing the magnifying ratio during the magnifying mode in the live view display of the shooting mode. The SET button 75, the LV button 76, and the enlargement button 77 are included in the operation unit 112.

The enlargement button 77 functions as a button for increasing the enlargement ratio of the image data displayed on the display unit 107 in the reproduction mode. The reduction button 78 is a button for reducing the enlargement ratio of the image data enlarged and displayed on the display unit 107. The play button 79 is an operation button for switching between a shooting mode and a play mode. When the user presses the play button 79 during the shooting mode, the image pickup device 100 shifts to the play mode and displays the image data recorded on the recording medium 110 on the display unit 107. The reduction button 78 and the play button 79 are included in the operation unit 112.

The quick return mirror 12 (hereinafter referred to as a mirror 12) is a mirror for switching the light beam incident from the optical lens mounted on the image pickup apparatus 100 so as to be incident on either the eyepiece finder 16 side or the image pickup unit 101 side. The mirror 12 is moved up and down by controlling an actuator (not shown) by the control unit 111 during exposure, live view shooting, and moving image shooting. The mirror 12 is normally arranged so that a light flux is incident on the eyepiece finder 16. In the case of shooting and live view display, the mirror 12 jumps upward so that a light flux is incident on the image pickup unit 101 (mirror lockup). Further, the central portion of the mirror 12 is a half mirror. A part of the light flux transmitted through the central portion of the mirror 12 is incident on a focal detection unit (not shown) for performing focus detection.

The communication terminal 10 is a communication terminal for communicating between the optical lens 300 mounted on the image pickup apparatus 100 and the image pickup apparatus 100. The terminal cover 40 is a cover that protects a connector (not shown) such as a connection cable for connecting a connection cable to an external device and the image pickup device 100. The lid 41 is a lid of a slot in which the recording medium 110 is stored. The lens mount 301 is a mounting portion to which an optical lens 300 (not shown) can be mounted.

The L microphone 201a and the R microphone 201b are microphones for collecting user's voice and the like. When viewed from the back surface of the image pickup apparatus 100, the L microphone 201a is arranged on the left side and the R microphone 201b is arranged on the right side.

<Structure of image pickup device 100>
FIG. 2 is a block diagram showing an example of the configuration of the image pickup apparatus 100 in this embodiment.

The optical lens 300 is a lens unit that can be attached to and detached from the image pickup device 100. For example, the optical lens 300 is a zoom lens or a varifocal lens. The optical lens 300 has an optical lens, a motor for driving the optical lens, and a communication unit that communicates with the lens control unit 102 of the image pickup apparatus 100 described later. The optical lens 300 can focus and zoom on the subject and correct camera shake by moving the optical lens by a motor based on the control signal received by the communication unit.

The image pickup unit 101 generates image data or moving image data from an image pickup element for converting an optical image of a subject imaged on an image pickup surface through an optical lens 300 into an electric signal, and an electric signal generated by the image pickup element. It has an image processing unit to output the image. The image pickup device is, for example, a CCD (Charge Coupled Device) and a CMOS (Complementary Metal Oxide Semiconductor). In this embodiment, a series of processes in which the image pickup unit 101 generates image data including still image data and moving image data and outputs the image data from the image pickup unit 101 is referred to as “shooting”. In the image pickup apparatus 100, the image data is recorded on the recording medium 110 described later in accordance with the DCF (Design rule for Camera File system) standard.

The lens control unit 102 transmits a control signal to the optical lens 300 via the communication terminal 10 based on the data output from the image pickup unit 101 and the control signal output from the control unit 111 described later, and causes the optical lens 300 to operate. Control.

The information acquisition unit 103 detects the tilt of the image pickup device 100, the temperature inside the housing of the image pickup device 100, and the like. For example, the information acquisition unit 103 detects the inclination of the image pickup device 100 by an acceleration sensor or a gyro sensor. Further, for example, the information acquisition unit 103 detects the temperature inside the housing of the image pickup apparatus 100 by a temperature sensor.

The voice input unit 104 generates voice data from the voice acquired by the microphone. The voice input unit 104 acquires voice around the image pickup device 100 by a microphone, performs analog-digital conversion (A / D conversion) and various voice processing on the acquired voice, and generates voice data. In this embodiment, the voice input unit 104 has a microphone. A detailed configuration example of the voice input unit 104 will be described later.

The volatile memory 105 temporarily records the image data generated by the imaging unit 101 and the audio data generated by the audio input unit 104. The volatile memory 105 is also used as a temporary recording area for image data displayed on the display unit 107, a work area for the control unit 111, and the like.

The display control unit 106 controls to display the image data output from the image pickup unit 101, characters for interactive operation, a menu screen, and the like on the display unit 107. Further, the display control unit 106 controls the display unit 107 to sequentially display the digital data output from the image pickup unit 101 on the display unit 107 during still image shooting and moving image shooting, so that the display unit 107 functions as an electronic viewfinder. Can be done. For example, the display unit 107 is a liquid crystal display or an organic EL display. Further, the display control unit 106 displays image data and moving image data output from the image pickup unit 101, characters for interactive operation, a menu screen, and the like on an external display via an external output unit 115, which will be described later. It can also be controlled to cause.

The coding processing unit 108 can encode the image data and the audio data temporarily recorded in the volatile memory 105, respectively. For example, the coding processing unit 108 can generate moving image data in which image data is encoded and data-compressed according to a JPEG standard or a RAW image format. For example, the coding processing unit 108 uses the MPEG2 standard or H.M. It is possible to generate video data encoded and data-compressed according to the 264 / MPEG4-AVC standard. Further, for example, the coding processing unit 108 can generate voice data in which voice data is coded and data-compressed according to the AC3AAC standard, ATRAC standard, or ADPCM method. Further, the coding processing unit 108 may encode the voice data so as not to compress the voice data according to, for example, the linear PCM method.

The recording control unit 109 can record data on the recording medium 110 and read the data from the recording medium 110. For example, the recording control unit 109 can record the still image data, the moving image data, and the audio data generated by the coding processing unit 108 on the recording medium 110 and read them from the recording medium 110. The recording medium 110 is, for example, an SD card, a CF card, an XQD memory card, an HDD (magnetic disk), an optical disk, and a semiconductor memory. The recording medium 110 may be configured to be detachable from the image pickup device 100, or may be built in the image pickup device 100. That is, the recording control unit 109 may have at least a means for accessing the recording medium 110.

The control unit 111 controls each component of the image pickup apparatus 100 via the data bus 116 according to the input signal and the program described later. The control unit 111 has a CPU, a ROM, and a RAM for executing various controls. Instead of the control unit 111 controlling the entire image pickup device 100, a plurality of hardware may share control of the entire image pickup device 100. The ROM included in the control unit 111 stores a program for controlling each component. Further, the RAM included in the control unit 111 is a volatile memory used for arithmetic processing and the like.

The operation unit 112 is a user interface for receiving an instruction to the image pickup apparatus 100 from the user. The operation unit 112 includes, for example, a power switch 72 for turning on or off the power of the image pickup apparatus 100, a release switch 61 for instructing shooting, a play button for instructing playback of image data or moving image data, and the like. And has a mode changeover switch 60 and the like.

The operation unit 112 outputs a control signal to the control unit 111 according to the operation of the user. Further, the touch panel formed on the display unit 107 can also be included in the operation unit 112. The release switch 61 has SW1 and SW2. When the release switch 61 is in the so-called half-pressed state, SW1 is turned on. As a result, it receives preparation instructions for performing preparatory operations for imaging such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-flash) processing. Further, when the release switch 61 is in the so-called fully pressed state, SW2 is turned on. By such a user operation, an imaging instruction for performing an imaging operation is received. Further, the operation unit 112 includes an operation member (for example, a button) capable of adjusting the volume of audio data reproduced from the speaker 114, which will be described later.

The audio output unit 113 can output audio data to the speaker 114 and the external output unit 115. The audio data input to the audio output unit 113 is audio data read from the recording medium 110 by the recording control unit 109, audio data output from the non-volatile memory 117, and audio data output from the coding processing unit. Is. The speaker 114 is an electroacoustic converter capable of reproducing audio data.

The external output unit 115 can output image data, moving image data, audio data, and the like to an external device. The external output unit 115 is composed of, for example, a video terminal, a microphone terminal, a headphone terminal, and the like.

The data bus 116 is a data bus for transmitting various data such as audio data, moving image data, and image data, and various control signals to each block of the image pickup apparatus 100.

The non-volatile memory 117 is a non-volatile memory, and stores a program or the like described later executed by the control unit 111. In addition, voice data is recorded in the non-volatile memory 117. This audio data includes, for example, a focusing sound output when the subject is in focus, an electronic shutter sound output when shooting is instructed, an operation sound output when the image pickup device 100 is operated, and the like. It is voice data of electronic sound.

<Operation of image pickup device 100>
From now on, the operation of the image pickup apparatus 100 of this Example will be described.

The image pickup apparatus 100 of this embodiment supplies electric power to each component of the image pickup apparatus from a power source (not shown) in response to the user operating the power switch 72 to turn on the power. For example, the power source is a battery such as a lithium ion battery or an alkaline manganese dry battery.

The control unit 111 determines, for example, which mode of the shooting mode and the reproduction mode operates based on the state of the mode changeover switch 60 according to the power supply. In the moving image recording mode, the control unit 111 records the moving image data output from the imaging unit 101 and the audio data output from the audio input unit 104 as one moving image data with audio. In the reproduction mode, the control unit 111 controls the recording control unit 109 to read out the image data or the moving image data recorded on the recording medium 110 and display the image data or the moving image data on the display unit 107.

First, the moving image recording mode will be described. In the moving image recording mode, first, the control unit 111 transmits a control signal to each component of the image pickup device 100 so as to shift the image pickup device 100 to the shooting standby state. For example, the control unit 111 controls the image pickup unit 101 and the voice input unit 104 to perform the following operations.

The image pickup unit 101 converts the optical image of the subject imaged on the image pickup surface through the optical lens 300 into an electric signal, and generates moving image data from the electric signal generated by the image pickup element. Then, the image pickup unit 101 transmits the moving image data to the display control unit 106, and the display unit 107 displays the moving image data. The user can prepare for shooting while viewing the moving image data displayed on the display unit 107.

The audio input unit 104 A / D-converts analog audio signals input from a plurality of microphones to generate a plurality of digital audio signals. Then, the voice input unit 104 generates voice data of a plurality of channels from the plurality of digital voice signals. The voice input unit 104 transmits the generated voice data to the voice output unit 113, and reproduces the voice data from the speaker 114. The user can adjust the volume of the audio data recorded in the moving image data with audio by the operation unit 112 while listening to the audio data reproduced from the speaker 114.

Next, in response to the user pressing the LV button 76, the control unit 111 transmits an instruction signal for starting shooting to each component of the image pickup apparatus 100. For example, the control unit 111 controls the image pickup unit 101, the voice input unit 104, the coding processing unit 108, and the recording control unit 109 to perform the following operations.

The image pickup unit 101 converts the optical image of the subject imaged on the image pickup surface through the optical lens 300 into an electric signal, and generates moving image data from the electric signal generated by the image pickup element. Then, the image pickup unit 101 transmits the moving image data to the display control unit 106, and the display unit 107 displays the moving image data. Further, the image pickup unit 101 also transmits the generated moving image data to the volatile memory 105.

The audio input unit 104 A / D-converts analog audio signals input from a plurality of microphones to generate a plurality of digital audio signals. Then, the voice input unit 104 generates multi-channel voice data from the plurality of digital voice signals. Then, the voice input unit 104 transmits the generated voice data to the volatile memory 105.

The coding processing unit 108 reads out the moving image data and the audio data temporarily recorded in the volatile memory 105 and encodes them respectively. The control unit 111 generates a data stream from the moving image data and the audio data encoded by the coding processing unit 108, and outputs the data stream to the recording control unit 109. The recording control unit 109 records the input data stream as moving image data with audio on the recording medium 110 according to a file system such as UDF or FAT.

Each component of the image pickup apparatus 100 continues the above operation during movie shooting.

Then, in response to the user pressing the LV button 76, the control unit 111 transmits an instruction signal for the end of shooting to each component of the image pickup apparatus 100. For example, the control unit 111 controls the image pickup unit 101, the voice input unit 104, the coding processing unit 108, and the recording control unit 109 to perform the following operations.

The image pickup unit 101 stops the generation of moving image data. The voice input unit 104 stops the generation of voice data.

The coding processing unit 108 reads out and encodes the remaining moving image data and audio data recorded in the volatile memory 105. The control unit 111 generates a data stream from the moving image data and the audio data encoded by the coding processing unit 108, and outputs the data stream to the recording control unit 109.

The recording control unit 109 records the data stream on the recording medium 110 as a file of moving image data with audio according to a file system such as UDF or FAT. Then, the recording control unit 109 completes the moving image data with audio in response to the stoppage of the input of the data stream. When the moving image data with sound is completed, the recording operation of the image pickup apparatus 100 is stopped.

The control unit 111 transmits a control signal to each component of the image pickup apparatus 100 so as to shift to the shooting standby state when the recording operation is stopped. As a result, the control unit 111 controls the image pickup apparatus 100 to return to the shooting standby state.

Next, the reproduction mode will be described. In the reproduction mode, the control unit 111 transmits a control signal to each component of the image pickup apparatus 100 so as to shift to the reproduction state. For example, the control unit 111 controls the coding processing unit 108, the recording control unit 109, the display control unit 106, and the voice output unit 113 to perform the following operations.

The recording control unit 109 reads out the moving image data with sound recorded on the recording medium 110 and transmits the read moving image data with sound to the coding processing unit 108.

The coding processing unit 108 decodes the image data and the audio data from the moving image data with audio. The coding processing unit 108 transmits the decoded moving image data to the display control unit 106 and the decoded audio data to the audio output unit 113, respectively.

The display control unit 106 displays the decoded image data by the display unit 107. The voice output unit 113 reproduces the decoded voice data by the speaker 114.

As described above, the image pickup apparatus 100 of this embodiment can record and reproduce image data and audio data.

In this embodiment, the voice input unit 104 executes voice processing such as adjustment processing of the level of the voice signal input from the microphone. In this embodiment, the voice input unit 104 executes this voice processing in response to the start of video recording. Note that this audio processing may be executed after the power of the image pickup apparatus 100 is turned on. Further, this audio processing may be executed depending on the shooting mode selected. Further, this voice processing may be executed depending on the selection of a mode related to voice recording such as a moving image recording mode and a voice memo function. Further, this voice processing may be executed in response to the start of recording of the voice signal.

<Structure of voice input unit 104>
FIG. 3 is a block diagram showing an example of a detailed configuration of the voice input unit 104 in this embodiment.

In this embodiment, the voice input unit 104 has three microphones, an L microphone 201a, an R microphone 201b, and a noise microphone 201c. The L microphone 201a and the R microphone 201b are examples of the first microphone, respectively. In this embodiment, the image pickup apparatus 100 picks up the ambient sound by the L microphone 201a and the R microphone 201b, and records the audio signals input from the L microphone 201a and the R microphone 201b in a stereo system. For example, the environmental sound is a user's voice, an animal's bark, a rain sound, and a sound generated outside the housing of the image pickup device 100 and the housing of the optical lens 300 such as music.

The noise microphone 201c is an example of the second microphone. The noise microphone 201c is a microphone for acquiring noise (noise) such as driving sound from a predetermined noise source (noise source) generated in the housing of the image pickup apparatus 100 and the housing of the optical lens 300. The noise source is, for example, a motor such as an ultrasonic motor (USM) and a stepper motor (STM). Noise (noise) is, for example, vibration noise generated by driving a motor such as USM and STM. For example, the motor is driven in an AF process for focusing on the subject. The image pickup device 100 acquires noise such as driving sound generated in the housing of the image pickup device 100 and the housing of the optical lens 300 by the noise microphone 201c, and uses the acquired noise voice data to generate noise described later. Generate parameters. In this embodiment, the L microphone 201a, the R microphone 201b, and the noise microphone 201c are omnidirectional microphones. An arrangement example of the L microphone 201a, the R microphone 201b, and the noise microphone 201c in this embodiment will be described later with reference to FIG.

The L microphone 201a, the R microphone 201b, and the noise microphone 201c each generate an analog audio signal from the acquired voice and input it to the A / D conversion unit 202. Here, the audio signal input from the L microphone 201a is referred to as Lch, the audio signal input from the R microphone 201b is referred to as Rch, and the audio signal input from the noise microphone 201c is referred to as Nch.

The A / D conversion unit 202 converts the analog audio signal input from the L microphone 201a, the R microphone 201b, and the noise microphone 201c into a digital audio signal. The A / D conversion unit 202 outputs the converted digital audio signal to the FFT unit 203. In this embodiment, the A / D conversion unit 202 converts an analog audio signal into a digital audio signal by executing sampling processing with a sampling frequency of 48 kHz and a bit depth of 16 bits.

The FFT unit 203 performs a fast Fourier transform process on the digital audio signal in the time domain input from the A / D conversion unit 202, and converts it into a digital audio signal in the frequency domain. In this embodiment, the digital audio signal in the frequency domain has a frequency spectrum of 1024 points in the frequency band from 0 Hz to 48 kHz. Further, the digital audio signal in the frequency domain has a frequency spectrum of 513 points in the frequency band from 0 Hz to 24 kHz, which is the Nyquist frequency. In this embodiment, the image pickup apparatus 100 performs noise reduction processing by using the frequency spectrum of 513 points from 0 Hz to 24 kHz in the audio data output from the FFT unit 203.

Here, the frequency spectrum of the Lch subjected to the fast Fourier transform is represented by the array data of 513 points from Lch_Before [0] to Lch_Before [512]. When these sequence data are generically referred to, they are described as Lch_Before. Further, the frequency spectrum of the Rch subjected to the fast Fourier transform is represented by the array data of 513 points from Rch_Before [0] to Rch_Before [512]. When these sequence data are generically referred to, they are described as Rch_Before. Note that Lch_Before and Rch_Before are examples of the first frequency spectrum data, respectively.

Further, the frequency spectrum of the Nch subjected to the fast Fourier transform is represented by the array data of 513 points from Nch_Before [0] to Nch_Before [512]. When these sequence data are generically referred to, they are described as Nch_Before. Note that Nch_Before is an example of the second frequency spectrum data.

The noise data generation unit 204 generates data for reducing noise contained in Lch_Before and Rch_Before based on Nch_Before. In this embodiment, the noise data generation unit 204 generates array data of NL [0] to NL [512] for reducing noise contained in Lch_Before [0] to Lch_Before [512] by using noise parameters. do. Further, the noise data generation unit 204 generates array data of NR [0] to NR [512] for reducing noise contained in Rch_Before [0] to Rch_Before [512], respectively. The frequency points in the sequence data of NL [0] to NL [512] are the same as the frequency points in the sequence data of Lch_Before [0] to Lch_Before [512]. Further, the frequency points in the sequence data of NR [0] to NR [512] are the same as the frequency points in the sequence data of Rch_Before [0] to Rch_Before [512].

In addition, when the sequence data of NL [0] to NL [512] is generically referred to, it is described as NL. When NR [0] to NR [512] are generically referred to, they are described as NR. NL and NR are examples of the third frequency spectrum data, respectively.

The noise parameter recording unit 205 records noise parameters for the noise data generation unit 204 to generate NL and NR from Nch_Before. The noise parameter recording unit 205 records a plurality of types of noise parameters according to the type of noise. When the noise parameters for generating NL from Nch_Before are generically referred to as PLx. When the noise parameters for generating NR from Nch_Before are generically referred to as PRx.

PLx and PRx have the same number of sequences as NL and NR, respectively. For example, PL1 is array data of PL1 [0] to PL1 [512]. Further, the frequency point of PL1 is the same as the frequency point of Lch_Before. Further, for example, PR1 is sequence data of PR1 [0] to PR1 [512]. The frequency point of PR1 is the same frequency point as Rch_Before. The noise parameters will be described later with reference to FIG.

The noise parameter selection unit 206 determines the noise parameter used in the noise data generation unit 204 from the noise parameter recorded in the noise parameter recording unit 205. The noise parameter selection unit 206 determines the noise parameter used in the noise data generation unit 204 based on the data received from Lch_Before, Rch_Before, Nch_Before, and the lens control unit 102. The operation of the noise parameter selection unit 206 will be described in detail later with reference to FIG.

In this embodiment, the noise parameter recording unit 205 records all the coefficients for each frequency spectrum of 513 points as noise parameters. However, it is sufficient that at least the coefficient of the frequency point necessary for reducing noise is recorded instead of the coefficient for all frequencies of 513 points. For example, the noise parameter recording unit 205 may record coefficients for each of the frequency spectra of 20 Hz to 20 kHz, which are considered to be typical audible frequencies, as noise parameters, and may not record coefficients of other frequency spectra. Further, for example, as a noise parameter, the coefficient for the frequency spectrum in which the value of the coefficient is zero may not be recorded in the noise parameter recording unit 205.

The subtraction processing unit 207 subtracts NL and NR from Lch_Before and Rch_Before, respectively. For example, the subtraction processing unit 207 has an L subtractor 207a that subtracts NL from Lch_Before, and an R subtractor 207b that subtracts NR from Rch_Before. The L subtractor 207a subtracts NL from Lch_Before and outputs 513 point sequence data of Lch_After [0] to Lch_After [512]. The R subtractor 207b subtracts NR from Rch_Before and outputs 513 point sequence data of Rch_After [0] to Rch_After [512]. In this embodiment, the subtraction processing unit 207 executes the subtraction processing by the spectral subtraction method.

The iFFT unit 208 converts the digital audio signal in the frequency domain input from the subtraction processing unit 207 into a digital audio signal in the time domain by performing an inverse fast Fourier transform (inverse Fourier transform).

The audio processing unit 209 executes audio processing for a digital audio signal in the time domain, such as an equalizer, an auto-level controller, and a stereo feeling enhancement process. The voice processing unit 209 outputs the voice-processed voice data to the volatile memory 105.

In this embodiment, the image pickup apparatus 100 has two microphones as the first microphone, but the image pickup apparatus 100 may use the first microphone as one microphone or three or more microphones. For example, when the image pickup apparatus 100 has one microphone as the first microphone in the voice input unit 104, the voice data picked up by one microphone is recorded in a monaural manner. Further, for example, when the image pickup apparatus 100 has three or more microphones as the first microphone in the voice input unit 104, the sound data picked up by the three or more microphones is recorded in a surround system.

In this embodiment, the L microphone 201a, the R microphone 201b, and the noise microphone 201c are omnidirectional microphones, but these microphones may be directional microphones.

<Arrangement of microphones in voice input unit 104>
Here, an example of arranging the microphone of the voice input unit 104 of this embodiment will be described. FIG. 4 shows an arrangement example of the L microphone 201a, the R microphone 201b, and the noise microphone 201c.

FIG. 4 is an example of a cross-sectional view of a portion of the image pickup apparatus 100 to which the L microphone 201a, the R microphone 201b, and the noise microphone 201c are attached. The portion of the image pickup apparatus 100 is composed of an exterior portion 302, a microphone bush 303, and a fixed portion 304.

The exterior portion 302 has a hole (hereinafter referred to as a microphone hole) for inputting an environmental sound into the microphone. In this embodiment, the microphone hole is formed above the L microphone 201a and the R microphone 201b. On the other hand, the noise microphone 201c is provided to acquire the driving sound generated in the housing of the image pickup apparatus 100 and the housing of the optical lens 300, and it is not necessary to acquire the environmental sound. Therefore, in this embodiment, the microphone hole is not formed above the noise microphone 201c in the exterior portion 302.

The drive sound generated in the housing of the image pickup apparatus 100 and the housing of the optical lens 300 is acquired by the L microphone 201a and the R microphone 201b through the microphone holes. When a driving sound or the like is generated in the housing of the image pickup apparatus 100 and the optical lens 300 in a state where the environmental sound is small, the sound acquired by each microphone is mainly the driving sound. Therefore, the sound level from the noise microphone 201c is higher than the sound level from the L microphone 201a and the R microphone 201b. That is, in this case, the relationship between the levels of the audio signals output from each microphone is as follows.
Lch ≒ Rch <Nch

Further, when the environmental sound becomes louder, the sound level of the environmental sound from the L microphone 201a and the R microphone 201b becomes louder than the sound level of the drive sound generated by the image pickup device 100 or the optical lens 300 from the microphone 201c. .. Therefore, in this case, the relationship between the levels of the audio signals output from each microphone is as follows.
Lch ≒ Rch> Nch

In this embodiment, the shape of the microphone hole formed in the exterior portion 302 is elliptical, but it may be another shape such as a circular shape or a square shape. Further, the shape of the microphone hole on the microphone 201a and the shape of the microphone hole on the microphone 201b may be different from each other.

In this embodiment, the noise microphone 201c is arranged so as to be close to the L microphone 201a and the R microphone 201b. Further, in this embodiment, the noise microphone 201c is arranged between the L microphone 201a and the R microphone 201b. As a result, the audio signal generated by the noise microphone 201c from the drive sound or the like generated in the housing of the image pickup apparatus 100 and the housing of the optical lens 300 is generated by the L microphone 201a and the R microphone 201b from the drive sound or the like. It becomes a signal similar to an audio signal.

The microphone bush 303 is a member for fixing the L microphone 201a, the R microphone 201b, and the noise microphone 201c. The fixing portion 304 is a member that fixes the microphone bush 303 to the exterior portion 302.

In this embodiment, the exterior portion 302 and the fixing portion 304 are made of a mold member such as a PC material. Further, the exterior portion 302 and the fixing portion 304 may be made of a metal member such as aluminum or stainless steel. Further, in this embodiment, the Mike Busch 303 is made of a rubber material such as ethylene propylene diene rubber.

<Noise parameter>
FIG. 5 is an example of noise parameters recorded in the noise parameter recording unit 205. The noise parameter is a parameter for correcting the audio signal generated by the noise microphone 201c acquiring the driving sound generated in the housing of the image pickup apparatus 100 and the housing of the optical lens 300. As shown in FIG. 5, in this embodiment, PLx and PRx are recorded in the noise parameter recording unit 205. In this embodiment, the source of the driving sound will be described as being inside the housing of the optical lens 300. The drive sound generated in the housing of the optical lens 300 is transmitted to the housing of the image pickup apparatus 100 via the lens mount 301, and is acquired by the L microphone 201a, the R microphone 201b, and the noise microphone 201c.

The frequency of the drive sound differs depending on the type of drive sound. Therefore, in this embodiment, the image pickup apparatus 100 records a plurality of noise parameters corresponding to the types of drive sounds (noise). Then, noise data is generated using any one of these plurality of noise parameters. In this embodiment, the image pickup apparatus 100 records a noise parameter with respect to white noise as constant noise. Further, the image pickup apparatus 100 records noise parameters for short-term noise generated by, for example, meshing of gears in the optical lens 300. Further, the image pickup apparatus 100 records noise parameters for sliding noise in the housing of the lens 300, for example, as long-term noise. In addition, the image pickup apparatus 100 may record noise parameters for each type of the optical lens 300, and for each temperature inside the housing of the image pickup apparatus 100 and the inclination of the image pickup apparatus 100 detected by the information acquisition unit 103.

<How to generate noise data>
The noise data generation process in the noise data generation unit 204 will be described with reference to FIGS. 6 and 7. Here, the noise data generation process for the Lch data will be described, but the noise data generation method for the Rch data is also the same.

First, a process of generating a noise parameter will be described in a situation where it can be considered that there is no environmental sound. FIG. 6A is an example of the frequency spectrum of Lch_Before when a driving sound is generated in the housing of the optical lens 300 in a situation where it can be considered that there is no environmental sound. FIG. 6B is an example of the frequency spectrum of Nch_Before when a driving sound is generated in the housing of the optical lens 300 in a situation where it can be considered that there is no environmental sound. The horizontal axis is the axis showing the frequency from the 0th point to the 512th point, and the vertical axis is the axis showing the amplitude of the frequency spectrum.

In Lch_Before and Nch_Before, the amplitude of the frequency spectrum in the same frequency band becomes large because the situation can be regarded as having no environmental sound. Further, since the driving sound is generated in the housing of the optical lens 300, the amplitude of each frequency spectrum with respect to the same driving sound tends to be larger in Nch_Before than in Lch_Before.

FIG. 6C is an example of PLx in this embodiment. In this embodiment, PLx is a coefficient of each frequency spectrum calculated by dividing the amplitude of each frequency spectrum of Lch_Before by the amplitude of each frequency spectrum of Nch_Before. The result of this division is described as Lch_Before / Nch_Before. That is, PLx is the ratio of the amplitudes of Lch_Before and Nch_Before. The noise parameter recording unit 205 records the value of Lch_Before / Nch_Before as the noise parameter PLx. As described above, since the amplitude of the frequency spectrum for the same drive sound tends to be larger in Nch_Before than in Lch_Before, the value of each coefficient of the noise parameter PLx tends to be smaller than 1. However, when the value of Nch_Before [n] is smaller than a predetermined threshold value, the noise parameter recording unit 205 records the noise parameter PLx with PLx [n] = 0.

Next, the process of applying the generated noise parameter to Nch_Before will be described. FIG. 7A is an example of the frequency spectrum of Lch_Before when the driving sound is generated in the housing of the optical lens 300 in the presence of the environmental sound. FIG. 7B is an example of the frequency spectrum of Nch_Before when the driving sound is generated in the housing of the optical lens 300 in the presence of the environmental sound. The horizontal axis is the axis showing the frequency from the 0th point to the 512th point, and the vertical axis is the axis showing the amplitude of the frequency spectrum.

FIG. 7C is an example of NL in the case where the driving sound is generated in the housing of the optical lens 300 in the presence of the environmental sound. The noise data generation unit 204 multiplies each frequency spectrum of Nch_Before by each coefficient of PLx to generate NL. NL is the frequency spectrum thus generated.

FIG. 7D is an example of Lch_After when a driving sound is generated in the housing of the optical lens 300 in a situation where an environmental sound is present. The subtraction processing unit 207 subtracts NL from Lch_Before to generate Lch_After. Lch_After is a frequency spectrum thus generated.

As a result, the image pickup apparatus 100 can reduce the noise caused by the driving sound in the housing of the optical lens 300 and record the environmental sound with less noise.

<Explanation of noise parameter selection unit 206>
FIG. 8 is a block diagram showing an example of a detailed configuration of the noise parameter selection unit 206.

Lch_Before, Rch_Before, Nch_Before, and a lens control signal are input to the noise parameter selection unit 206.

The Nch noise detection unit 2061 detects noise due to the driving sound generated in the housing of the optical lens 300 from Nch_Before. The Nch noise detection unit 2061 outputs data related to the noise detection result to the noise determination unit 2063 based on the noise detection result. In this embodiment, the Nch noise detection unit 2061 detects noise by using the data of 513 points of Nch_Before.

The environmental sound detection unit 2062 detects the level of the environmental sound from Lch_Before and Rch_Before. The environmental sound detection unit 2062 outputs data regarding the detection result of the environmental sound level to the noise determination unit 2063 based on the detection result of the environmental sound level.

The noise determination unit 2063 has a noise data generation unit 204 based on the lens control signal input from the lens control unit 102, the data input from the Nch noise detection unit 2061, and the data input from the environmental sound detection unit 2062. Determine the noise parameters to use. The noise determination unit 2063 outputs data indicating the type of the determined noise parameter to the noise data generation unit 204.

The Nch differential unit 2064 executes the differential process on Nch_Before. The Nch differentiation unit 2064 outputs data indicating the result of differential processing of Nch_Before to the short-term noise detection unit 2065. The short-term noise detection unit 2065 detects whether or not short-term noise is generated based on the data input from the Nch differentiation unit 2064. The short-term noise detection unit 2065 outputs data indicating whether or not short-term noise is generated to the noise determination unit 2063. The Nch differentiation unit 2064 and the short-term noise detection unit 2065 are included in the Nch noise detection unit 2061.

The Nch integration unit 2066 executes an integration process on Nch_Before. The Nch integration unit 2066 outputs data indicating the result of differential processing of Nch_Before to the long-term noise detection unit 2067. The long-term noise detection unit 2067 detects whether or not long-term noise is generated based on the data input from the Nch integration unit 2066. The long-term noise detection unit 2067 outputs data indicating whether or not long-term noise is generated to the noise determination unit 2063. The Nch integration unit 2066 and the long-term noise detection unit 2067 are included in the Nch noise detection unit 2061.

The environmental sound extraction unit 2068 extracts the environmental sound. In this embodiment, the environmental sound extraction unit 2068 extracts frequency data that is less affected by noise based on the noise meter. For example, the environmental sound extraction unit 2068 extracts frequency data having a noise parameter of a predetermined value or less. Then, the environmental sound extraction unit 2068 outputs data indicating the magnitude of the environmental sound based on the extracted frequency data. The environmental sound extraction unit 2068 is included in the environmental sound detection unit 2062.

The environmental sound determination unit 2069 determines the loudness of the environmental sound. The environmental sound determination unit 2069 inputs data indicating the loudness of the determined environmental sound to the Nch noise detection unit 2061 and the noise determination unit 2063. The Nch noise detection unit 2061 changes the first threshold value and the second threshold value, which will be described later, based on the data indicating the loudness of the environmental sound input from the environmental sound determination unit 2069. The environmental sound determination unit 2069 is included in the environmental sound detection unit 2062.

<Timing chart of noise reduction processing>
The noise reduction processing in this embodiment will be described with reference to FIG.

9 (a) to 9 (i) are examples of timing charts of voice processing in the noise data generation unit 204, the noise parameter selection unit 206, and the subtraction processing unit 207. In this embodiment, the Lch voice processing will be described for the sake of simplification of the description, but the same applies to the Rch voice processing. The horizontal axes of the graphs in FIGS. 9 (a) to 9 (i) are all time axes.

FIG. 9A shows an example of a lens control signal. The lens control signal is a signal instructing the lens control unit 102 to drive the optical lens 300. In this embodiment, the level of the lens control signal is represented by two values, High and Low. When the level of the lens control signal is High, the lens control unit 102 is instructing the optical lens 300 to drive the lens 300. When the level of the lens control signal is Low, the lens control unit 102 is in a state in which the optical lens 300 is not instructed to drive.

FIG. 9B is a graph showing an example of the value of Lch_Before [n]. The vertical axis is an axis indicating the value of Lch_Before [n]. In this embodiment, Lch_Before [n] is a signal at the nth frequency point in which the signal indicating the driving sound of the optical lens 300 is characteristically displayed among the Lch_Before output from the FFT unit 203. In this embodiment, the signal at the nth frequency point will be described, but voice processing is similarly executed for other frequencies. Further, the signals represented by the signal X and the signal Y are signals containing noise. In this embodiment, the signal X indicates a signal including short-term noise. The signal Y indicates a noise signal including long-term noise.

FIG. 9C is a graph showing an example of the magnitude of the environmental sound extracted by the environmental sound extraction unit 2068. The vertical axis shows the level of the audio signal generated from the acquired environmental sound. The threshold value Th1 and the threshold value Th2 are two threshold values used in the environmental sound determination unit 2069.

FIG. 9D is a graph showing an example of the value of Nch_Before [n]. Nch_Before [n] is a signal at the nth frequency point in which the signal indicating the driving sound of the optical lens 300 is characteristically displayed among the Nch_Before output from the FFT unit 203. The vertical axis is an axis indicating the value of Nch_Before [n]. In Nch_Before [n], the noise signal shown by the signal X and the signal Y in FIG. 9B appears more characteristically than in Lch_Before.

FIG. 9 (e) is a graph showing an example of the value of Ndiff [n]. Ndiff [n] indicates the value of the signal at the nth frequency point in the Ndiff output from the Nch differential unit 2064. The vertical axis is an axis indicating the value of Ndiff [n]. When the amount of change in the value of Nch_Before [n] per predetermined time is large, the value of Ndiff [n] becomes large. The short-term noise detection unit 2065 has a threshold value Th_Ndiff [n], which is a first threshold value, in order to detect short-term noise. The threshold value Th_Ndiff [n] changes between levels 1 to 3 based on the data indicating the magnitude of the environmental sound input from the environmental sound determination unit 2069 and the lens control signal. The initial value of the threshold value Th_Ndiff [n] is level 2. The level of the threshold value Th_Ndiff [n] is represented by a horizontal broken line.

FIG. 9 (f) is a graph showing an example of the value of Nint [n]. In this embodiment, Nint [n] indicates the value of the signal at the nth frequency point of the Nint output from the Nch integrating unit 2066. The vertical axis is an axis indicating the value of Nint [n]. When Nch_Before [n] is continuously large, the value of Nint [n] becomes large. The long-term noise detection unit 2067 has a threshold value Th_Nint [n], which is a second threshold value, in order to detect long-term noise. The threshold value Th_Nint [n] changes between levels 1 to 3 based on the data indicating the magnitude of the environmental sound input from the environmental sound determination unit 2069 and the lens control signal. The initial value of the threshold value Th_Nint [n] is level 2. The level of the threshold value Th_Nint [n] is represented by a horizontal broken line.

FIG. 9 (g) shows an example of the noise parameter selected by the noise parameter selection unit 206. In this embodiment, the plain portion indicates that only the noise parameter of PL1 is selected. The shaded area indicates that the noise parameters of PL1 and PL2 are selected. The plaid portion indicates that the noise parameters of PL1 and PL3 are selected.

FIG. 9 (h) is a graph showing an example of the value of NL [n]. In this embodiment, NL [n] indicates the value of the signal at the nth frequency point among the NLs generated by the noise data generation unit 204. The vertical axis is an axis indicating the value of NL [n].

FIG. 9 (i) is a graph showing an example of the value of Lch_After [n]. In this embodiment, Lch_After [n] indicates the value of the signal at the nth frequency point of the Lch_After output from the subtraction processing unit 207. The vertical axis is an axis indicating the value of Lch_After [n].

Next, the timing of each operation will be described using the times t701 to t709.

At time t701, the lens control unit 102 outputs a High signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9A). Since there is a high possibility that a driving sound is generated in the housing of the optical lens 300 at time t701, the short-term noise detection unit 2065 lowers the threshold value Th_Ndiff [n] to level 1 (FIG. 9 (e)). Further, at time t701, since there is a high possibility that a driving sound is generated in the housing of the optical lens 300, the long-term noise detection unit 2067 lowers the threshold value Th_Nint [n] to level 1 (FIG. 9 (f)).

At time t702, the optical lens 300 is driven, and a short-term driving sound such as a gear meshing sound is generated. The value of Ndiff [n] exceeds the threshold value Th_Ndiff [n] due to the noise microphone 201c picking up the short-term drive sound (FIG. 9 (e)). In response to this, the noise parameter selection unit 206 selects the noise parameters PL1 and PL2 (FIG. 9 (g)). The noise data generation unit 204 generates NL [n] based on Nch_Before [n] and the noise parameters PL1 and PL2 (FIG. 9 (h)). The subtraction processing unit 207 subtracts NL [n] from Lch_Before [n] and outputs Lch_After [n] (FIG. 9 (i)). In this case, Lch_After [n] becomes an audio signal in which constant noise and short-term noise are reduced.

At time t703, the optical lens 300 starts continuous driving, and a long-term driving sound such as a sliding sound is generated in the housing of the optical lens 300. The value of Nint [n] exceeds the threshold value Th_Nint [n] due to the noise microphone 201c picking up the long-term driving sound (FIG. 9 (f)). In response to this, the noise parameter selection unit 206 selects the noise parameters PL1 and PL3 (FIG. 9 (g)). The noise data generation unit 204 generates NL [n] based on Nch_Before [n] and the noise parameters PL1 and PL3 (FIG. 9 (h)). The subtraction processing unit 207 subtracts NL [n] from Lch_Before [n] and outputs Lch_After [n] (FIG. 9 (i)). In this case, Lch_After [n] becomes an audio signal in which constant noise and long-term noise are reduced.

At time t704, the optical lens 300 stops continuous driving. Since the noise microphone 201c does not collect the long-term driving sound, the value of Nint [n] becomes equal to or less than the threshold value Th_Nint [n] (FIG. 9 (f)). In response to this, the noise parameter selection unit 206 selects the noise parameter PL1 (FIG. 9 (g)). The noise data generation unit 204 generates NL [n] based on Nch_Before [n] and the noise parameter PL1 (FIG. 9 (h)). The subtraction processing unit 207 subtracts NL [n] from Lch_Before [n] and outputs Lch_After [n] (FIG. 9 (i)). In this case, Lch_After [n] becomes an audio signal with constant noise reduced.

At time t705, the lens control unit 102 outputs a Low signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9A). In this case, since the possibility that a driving sound is generated in the housing of the optical lens 300 is low, the short-term noise detection unit 2065 raises the threshold value Th_Ndiff [n] to level 2 (FIG. 9 (e)). Further, in this case, since the possibility that the driving sound is generated in the housing of the optical lens 300 is low, the long-term noise detection unit 2067 raises the threshold value Th_Nint [n] to level 2 (FIG. 9 (f)).

At time t706, the loudness of the environmental sound extracted by the environmental sound extraction unit 2068 exceeds the threshold value Th1. When the ambient sound is loud, the noise contained in the audio signal is less likely to be perceived by the user, so the short-term noise detection unit 2065 raises the threshold value Th_Ndiff [n] to level 3 (FIG. 9 (e)). Further, when the environmental sound is loud, the noise contained in the audio signal is less likely to be perceived by the user, so that the long-term noise detection unit 2067 raises the threshold value Th_Nint [n] to level 3 (FIG. 9 (f)).

At time t707, the lens control unit 102 outputs a High signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9A). In this case, since there is a high possibility that a driving sound is generated in the housing of the optical lens 300, the short-term noise detection unit 2065 lowers the threshold value Th_Ndiff [n] to level 2 (FIG. 9 (e)). Further, in this case, since there is a high possibility that a driving sound is generated in the housing of the optical lens 300, the long-term noise detection unit 2067 lowers the threshold value Th_Nint [n] to level 2 (FIG. 9 (f)).

At time t708, the loudness of the environmental sound extracted by the environmental sound extraction unit 2068 exceeds the threshold value Th2. When the ambient sound is louder, the user hardly feels the noise contained in the voice signal. Therefore, the noise parameter selection unit 206 selects only the noise parameter PL1 regardless of the data input from the Nch noise detection unit 2061.

As described above, the image pickup apparatus 100 can record the environmental sound with reduced noise by executing the noise reduction processing using the noise microphone 201c which is the second microphone.

In this embodiment, the image pickup device 100 reduces the drive sound generated in the housing of the optical lens 300, but the drive sound generated in the image pickup device 100 may be reduced. The driving sound generated in the image pickup apparatus 100 is, for example, the noise of the substrate and the radio wave noise. The squeal of the substrate is, for example, a sound generated by a squeak of the substrate generated when a voltage is applied to a capacitor on the substrate.

The threshold values Th1 and Th2 of the environmental sound determination unit 2069, the threshold value Th_Ndiff [n] of the short-term noise detection unit 2065, and the threshold value Th_Nint [n] of the long-term noise detection unit 2067 are based on the generated driving sound and the environmental sound. Will be decided. Therefore, the image pickup apparatus 100 may change these threshold values depending on the type of the optical lens 300, the inclination of the image pickup apparatus 100, and the like.

[Second Example]
Here, FIG. 10 is a block diagram showing a configuration example of the voice input unit 104 in the second embodiment. The parts different from the configuration of the voice input unit 104 shown in FIG. 3 are the subtraction processing unit 207 and the iFFT unit 208. Here, the description of the same processing unit as in FIG. 3 will be omitted.

The iFFT unit 208a performs inverse fast Fourier transforms on Lch_Before and Rch_Before input from the FFT unit 203, respectively, and converts the digital audio signal in the frequency domain into the digital audio signal in the time domain, respectively. Further, the iFFT unit 208b performs inverse fast Fourier transforms of NL and NR, respectively, and converts a digital audio signal in the frequency domain into a digital audio signal in the time domain.

The subtraction processing unit 207 subtracts the digital audio signal input from the iFFT unit 208b from the digital audio signal input from the iFFT unit 208a. The arithmetic processing in the subtraction processing unit 207 is a waveform subtraction method for subtracting a digital audio signal in the time domain.

When performing waveform subtraction, the image pickup apparatus 100 may also record a parameter related to the phase of the digital audio signal as a noise parameter.

The configuration and operation of the other image pickup apparatus 100 are the same as those in the first embodiment.

[Other Examples]
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or recording medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The present invention is not limited to the above embodiment as it is, and at the implementation stage, the components can be modified and embodied within a range that does not deviate from the gist thereof. In addition, various inventions can be formed by an appropriate combination of the plurality of components disclosed in the above examples. For example, some components may be removed from all the components shown in the examples. In addition, components from different embodiments may be combined as appropriate.

Claims (13)

With the first microphone to get the ambient sound,
With a second microphone to get the sound from the noise source,
The first conversion means for generating the first audio signal by Fourier transforming the audio signal from the first microphone,
A second conversion means that Fourier transforms the audio signal from the second microphone to generate a second audio signal,
A detection means for detecting the type of noise generated from the noise source, and
A generation means for generating noise data using the second audio signal and parameters related to noise of the noise source according to the type of noise detected by the detection means.
A subtraction means for subtracting the noise data from the first audio signal,
A third transforming means for inverse Fourier transforming the audio signal from the subtracting means,
A voice processing device characterized by having. The detection means detects at least one of a first type of noise and a second type of noise.
The generation means is provided by the detection means among a plurality of the parameters including the first parameter corresponding to the first type of noise and the second parameter corresponding to the second type of noise. The voice processing apparatus according to claim 1, wherein the noise data is generated by using the parameter corresponding to the detected type of noise and the second voice signal. The voice processing apparatus according to claim 2, further comprising a recording means for recording information of the plurality of parameters. The first microphone is composed of a plurality of microphones.
The voice processing device according to claim 3, wherein the recording means records the parameters for each microphone constituting the first microphone. The voice processing apparatus according to any one of claims 2 to 4, wherein the first type of noise is short-term noise, and the second type of noise is long-term noise. .. The voice processing apparatus according to any one of claims 1 to 5, wherein the detection means detects a type of noise from noise included in the second voice signal. It has a driving means as the noise source, and has
When the amplitude of the audio signal of the noise contained in the second audio signal is larger than a predetermined threshold value, the detection means detects the type of noise contained in the second audio signal.
The one according to any one of claims 1 to 6, wherein the predetermined threshold value in the state where the drive means is driven is smaller than the predetermined threshold value in the state where the drive means is not driven. Voice processing device. The detection means is characterized in that short-term noise is detected by differentiating the second voice signal, and long-term noise is detected by integrating the second voice signal. The voice processing apparatus according to any one of 7 to 7. The voice processing device is characterized in that a hole for inputting an environmental sound is formed above the first microphone, and a hole for inputting an environmental sound is not formed above the second microphone. The voice processing apparatus according to any one of claims 1 to 8. The voice processing apparatus according to any one of claims 1 to 9, wherein the parameter is a ratio of the amplitudes of the first voice signal and the second voice signal. It also has an imaging means,
The audio processing device according to any one of claims 1 to 10, wherein the noise source is a member that is driven in imaging in the imaging means. With the first microphone to get the ambient sound,
A method of controlling a voice processing device having a second microphone and a second microphone for acquiring sound from a noise source.
The step of Fourier transforming the audio signal from the first microphone to generate the first audio signal,
The step of Fourier transforming the audio signal from the second microphone to generate the second audio signal,
A detection step for detecting the type of noise generated from the noise source, and
A step of generating noise data using the second audio signal and parameters related to noise of the noise source according to the type of noise detected in the detection step.
A subtraction step of subtracting the noise data from the first audio signal,
A step of inverse Fourier transforming the audio signal generated by the subtraction step,
A control method characterized by having. A computer-readable program for operating a computer as each means of the voice processing device according to any one of claims 1 to 11.

Images