JP2022038611A - Sound processor, control method, and program - Google Patents

Abstract

To effectively reduce noise.SOLUTION: A sound processor comprises: mounting means on which a lens having a noise source is mounted; a first microphone for acquiring environmental sounds; a second microphone for acquiring sounds from the noise source; first transform means for performing Fourier transform on sound signals from the first microphone to generate first sound signals; second transform means for performing Fourier transform on sound signals from the second microphone to generate second sound signals; generation means for generating noise data using the second sound signals and a parameter related to the noise of the noise source; subtraction means for subtracting the noise data from the first sound signals; and third transform means for performing inverse Fourier transform on the sound signals from the subtraction means. The generation means uses the parameter in accordance with a kind of the lens mounted on the mounting means as a parameter.SELECTED DRAWING: Figure 12

Description

The present invention relates to a voice processing device capable of reducing noise contained in voice data.

When recording moving image data, a digital camera, which is an example of an audio processing device, can also record ambient audio. In addition, the digital camera has an autofocus function that focuses on the subject during recording of moving image data by driving an optical lens. In addition, the digital camera has a function of driving an optical lens to perform zooming while recording a moving image.

As described above, when the optical lens is driven during the recording of the moving image, the driving sound of the optical lens may be included as noise in the sound recorded together with the moving image. Therefore, conventionally, when a digital camera collects a sliding sound or the like generated when an optical lens is driven as noise, the noise can be reduced and the surrounding sound can be recorded. Patent Document 1 discloses a digital camera that reduces noise by a spectral subtraction method.

Japanese Unexamined Patent Publication No. 2011-205527

However, in Patent Document 1, since the digital camera creates a noise pattern from the noise collected by the microphone that records the surrounding sound, an accurate noise pattern is acquired from the sliding noise generated in the housing of the optical lens. It may not be possible. In this case, the digital camera may not be able to effectively reduce the noise contained in the collected sound.

Therefore, an object of the present invention is to effectively reduce noise.

From a mounting means to which a lens having a noise source is mounted, a first microphone for acquiring environmental sound, a second microphone for acquiring sound from the noise source, and the first microphone. A first conversion means for Fourier-converting an audio signal to generate a first audio signal, and a second conversion means for Fourier-converting an audio signal from the second microphone to generate a second audio signal. , A generation means for generating noise data using the second voice signal and parameters related to noise of the noise source, a subtraction means for subtracting the noise data from the first voice signal, and the subtraction means. A speech processing device comprising a third conversion means for inverse Fourier transforming an audio signal from the above, wherein the generation means has, as the parameter, the parameter according to the type of the lens mounted on the mounting means. A voice processing device characterized by using.

The voice processing apparatus of the present invention can effectively reduce noise.

It is a perspective view of the image pickup apparatus in 1st Example. It is a block diagram which shows the structure of the image pickup apparatus in 1st Example. It is a block diagram which shows the structure of the audio input part of the image pickup apparatus in the 1st Example. It is a figure which shows the arrangement of the microphone in the audio input part of the image pickup apparatus in the 1st Example. It is a figure which shows the noise parameter in 1st Example. It is a figure which shows the frequency spectrum of voice, and the frequency spectrum of a noise parameter in the case where the driving sound is generated in the situation which can be considered that there is no environmental sound in 1st Example. It is a figure which shows the frequency spectrum of the voice in the case where the driving sound is generated in the situation which there is an environmental sound in the 1st Example. It is a block diagram which shows the structure of the noise parameter selection part in 1st Example. It is a timing chart related to the voice noise reduction processing in the 1st Example. It is a figure which shows the noise parameter for each lens in 1st Example. It is a figure which shows the frequency spectrum of the voice in the case where the driving sound is generated in the situation which there is an environmental sound in the 1st Example. It is a flowchart which shows the process of the image pickup apparatus 100 in 1st Example. It is a figure which shows the posture information of the image pickup apparatus in the 2nd Example. It is a figure which shows the change of the frequency spectrum by the change of the posture of the image pickup apparatus in the 2nd Example. It is a correction noise parameter for each posture information of the image pickup apparatus in the second embodiment. In the second embodiment, it is a comparison of the lengths of the lens barrels at different zoom magnifications of the image pickup apparatus. It is a figure which shows the change of the frequency spectrum by the difference in the length of the lens barrel of the image pickup apparatus in the 2nd Example. It is a correction noise parameter for each zoom magnification of the image pickup apparatus in the second embodiment. It is a flowchart which shows the process of the image pickup apparatus 100 in the 2nd Example. It is a block diagram which shows the structure of the audio input part of the image pickup apparatus in the 3rd Example.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[First Example]
<External view of the image pickup device 100>
1A and 1B show an example of an external view of an image pickup apparatus 100 as an example of an audio processing apparatus to which the present invention can be applied. FIG. 1A is an example of a front perspective view of the image pickup apparatus 100. FIG. 1B is an example of a rear perspective view of the image pickup apparatus 100. In FIG. 1, an optical lens (not shown) is attached to the lens mount 301.

The display unit 107 displays image data, character information, and the like. The display unit 107 is provided on the back surface of the image pickup apparatus 100. The display unit 43 outside the viewfinder is a display unit provided on the upper surface of the image pickup apparatus 100. The out-of-finder display unit 43 displays the set values of the image pickup apparatus 100 such as the shutter speed and the aperture value. The eyepiece finder 16 is a peep-type finder. The user can confirm the focus and composition of the optical image of the subject by observing the focusing screen in the eyepiece finder 16.

The release switch 61 is an operating member for the user to give a shooting instruction. The mode changeover switch 60 is an operation member for the user to switch between various modes. The main electronic dial 71 is a rotation operation member. The user can change the setting values of the image pickup apparatus 100 such as the shutter speed and the aperture value by turning the main electronic dial 71. The release switch 61, the mode changeover switch 60, and the main electronic dial 71 are included in the operation unit 112.

The power switch 72 is an operating member that switches the power of the image pickup apparatus 100 on and off. The sub electronic dial 73 is a rotation operation member. The user can move the selection frame displayed on the display unit 107 by the sub-electronic dial 73, advance the image in the reproduction mode, and the like. The cross key 74 is a cross key (four-way key) capable of pushing up, down, left, and right portions, respectively. The image pickup apparatus 100 executes processing according to the pressed portion (direction) of the cross key 74. The power switch 72, the sub electronic dial 73, and the cross key 74 are included in the operation unit 112.

The SET button 75 is a push button. The SET button 75 is mainly used for the user to determine a selection item displayed on the display unit 107. The LV button 76 is a button used to switch the live view (hereinafter referred to as LV) on and off. The LV button 76 is used to instruct to start and stop moving image recording (recording) in the moving image recording mode. The magnifying button 77 is a push button for turning on and off the magnifying mode and changing the magnifying ratio during the magnifying mode in the live view display of the shooting mode. The SET button 75, the LV button 76, and the enlargement button 77 are included in the operation unit 112.

The enlargement button 77 functions as a button for increasing the enlargement ratio of the image data displayed on the display unit 107 in the reproduction mode. The reduction button 78 is a button for reducing the enlargement ratio of the image data enlarged and displayed on the display unit 107. The play button 79 is an operation button for switching between a shooting mode and a play mode. When the image pickup device 100 presses the play button 79 during the shooting mode, the image pickup device 100 shifts to the play mode and displays the image data recorded on the recording medium 110 on the display unit 107. The reduction button 78 and the play button 79 are included in the operation unit 112.

The quick return mirror 12 (hereinafter referred to as a mirror 12) is a mirror for switching the light beam incident from the optical lens mounted on the image pickup apparatus 100 so as to be incident on either the eyepiece finder 16 side or the image pickup unit 101 side. The mirror 12 is moved up and down by controlling an actuator (not shown) by the control unit 111 during exposure, live view shooting, and moving image shooting. The mirror 12 is normally arranged so that a light flux is incident on the eyepiece finder 16. In the case of shooting and live view display, the mirror 12 jumps upward so that a light flux is incident on the image pickup unit 101 (mirror lockup). Further, the central portion of the mirror 12 is a half mirror. A part of the light flux transmitted through the central portion of the mirror 12 is incident on a focal detection unit (not shown) for performing focus detection.

The communication terminal 10 is a communication terminal for communicating between the optical lens 300 mounted on the image pickup apparatus 100 and the image pickup apparatus 100. The terminal cover 40 is a cover that protects a connector (not shown) such as a connection cable for connecting a connection cable to an external device and the image pickup device 100. The lid 41 is a lid of a slot in which the recording medium 110 is stored. The lens mount 301 is a mounting portion to which an optical lens 300 (not shown) can be mounted.

The L microphone 201a and the R microphone 201b are microphones for collecting user's voice and the like. When viewed from the back surface of the image pickup apparatus 100, the L microphone 201a is arranged on the left side and the R microphone 201b is arranged on the right side.

<Structure of image pickup device 100>
FIG. 2 is a block diagram showing an example of the configuration of the image pickup apparatus 100 in this embodiment.

The optical lens 300 is a lens unit that can be attached to and detached from the image pickup device 100. For example, the optical lens 300 is a zoom lens or a varifocal lens. The optical lens 300 has an optical lens, a motor for driving the optical lens, and a communication unit that communicates with the lens control unit 102 of the image pickup apparatus 100 described later. The optical lens 300 can focus and zoom on the subject and correct camera shake by moving the optical lens by a motor based on the control signal received by the communication unit.

The image pickup unit 101 generates image data or moving image data from an image pickup element for converting an optical image of a subject imaged on an image pickup surface through an optical lens 300 into an electric signal, and an electric signal generated by the image pickup element. It has an image processing unit to output the image. The image pickup device is, for example, a CCD (Charge Coupled Device) and a CMOS (Complementary Metal Oxide Semiconductor). In this embodiment, a series of processes in which the image pickup unit 101 generates image data including still image data and moving image data and outputs the image data from the image pickup unit 101 is referred to as “shooting”. In the image pickup apparatus 100, the image data is recorded on the recording medium 110 described later in accordance with the DCF (Design rule for Camera File system) standard.

The lens control unit 102 transmits a control signal to the optical lens 300 via the communication terminal 10 based on the data output from the image pickup unit 101 and the control signal output from the control unit 111 described later, and causes the optical lens 300 to operate. Control. Further, the lens control unit 102 receives lens information from the optical lens 300 mounted on the image pickup apparatus 100. The lens information is, for example, the type of lens, the model number of the lens, the type of noise source, and the like.

The information acquisition unit 103 detects the tilt of the image pickup device 100, the temperature inside the housing of the image pickup device 100, and the like. For example, the information acquisition unit 103 detects the inclination of the image pickup device 100 by an acceleration sensor or a gyro sensor. Further, for example, the information acquisition unit 103 detects the temperature inside the housing of the image pickup apparatus 100 by a temperature sensor.

The voice input unit 104 generates voice data from the voice acquired by the microphone. The voice input unit 104 acquires voice around the image pickup device 100 by a microphone, performs analog-digital conversion (A / D conversion) and various voice processing on the acquired voice, and generates voice data. In this embodiment, the voice input unit 104 has a microphone. A detailed configuration example of the voice input unit 104 will be described later.

The volatile memory 105 temporarily records the image data generated by the imaging unit 101 and the audio data generated by the audio input unit 104. The volatile memory 105 is also used as a temporary recording area for image data displayed on the display unit 107, a work area for the control unit 111, and the like.

The display control unit 106 controls to display the image data output from the image pickup unit 101, characters for interactive operation, a menu screen, and the like on the display unit 107. Further, the display control unit 106 controls the display unit 107 to display the digital data output from the image pickup unit 101 on the sequential display unit 107 at the time of still image shooting and moving image shooting, so that the display unit 107 functions as an electronic viewfinder. Can be done. For example, the display unit 107 is a liquid crystal display or an organic EL display. Further, the display control unit 106 displays image data and moving image data output from the image pickup unit 101, characters for interactive operation, a menu screen, and the like on an external display via an external output unit 115, which will be described later. It can also be controlled to cause.

The coding processing unit 108 can encode the image data and the audio data temporarily recorded in the volatile memory 105, respectively. For example, the coding processing unit 108 can generate moving image data in which image data is encoded and data-compressed according to a JPEG standard or a RAW image format. For example, the coding processing unit 108 uses the MPEG2 standard or H.M. It is possible to generate video data encoded and data-compressed according to the 264 / MPEG4-AVC standard. Further, for example, the coding processing unit 108 can generate voice data in which voice data is coded and data-compressed according to the AC3AAC standard, ATRAC standard, or ADPCM method. Further, the coding processing unit 108 may encode the voice data so as not to compress the voice data according to, for example, the linear PCM method.

The recording control unit 109 can record data on the recording medium 110 and read the data from the recording medium 110. For example, the recording control unit 109 can record the still image data, the moving image data, and the audio data generated by the coding processing unit 108 on the recording medium 110, and can read the data from the recording medium 110. The recording medium 110 is, for example, an SD card, a CF card, an XQD memory card, an HDD (magnetic disk), an optical disk, and a semiconductor memory. The recording medium 110 may be configured to be detachable from the image pickup device 100, or may be built in the image pickup device 100. That is, the recording control unit 109 may have at least a means for accessing the recording medium 110.

The control unit 111 controls each component of the image pickup apparatus 100 via the data bus 116 according to the input signal and the program described later. The control unit 111 has a CPU, a ROM, and a RAM for executing various controls. Instead of the control unit 111 controlling the entire image pickup device 100, a plurality of hardware may share control of the entire image pickup device 100. The ROM included in the control unit 111 stores a program for controlling each component. Further, the RAM included in the control unit 111 is a volatile memory used for arithmetic processing and the like.

The operation unit 112 is a user interface for receiving an instruction to the image pickup apparatus 100 from the user. The operation unit 112 includes, for example, a power switch 72 for turning on or off the power of the image pickup apparatus 100, a release switch 61 for instructing shooting, a play button for instructing playback of image data or moving image data, and the like. And has a mode changeover switch 60 and the like.

The operation unit 112 outputs a control signal to the control unit 111 according to the operation of the user. Further, the touch panel formed on the display unit 107 can also be included in the operation unit 112. The release switch 61 has SW1 and SW2. When the release switch 61 is in the so-called half-pressed state, SW1 is turned on. As a result, it receives preparation instructions for performing preparatory operations for imaging such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-flash) processing. Further, when the release switch 61 is in the so-called fully pressed state, SW2 is turned on. By such a user operation, an image pickup instruction for performing an image pickup operation is received. Further, the operation unit 112 includes an operation member (for example, a button) capable of adjusting the volume of audio data reproduced from the speaker 114, which will be described later.

The audio output unit 113 can output audio data to the speaker 114 and the external output unit 115. The audio data input to the audio output unit 113 is audio data read from the recording medium 110 by the recording control unit 109, audio data output from the non-volatile memory 117, and audio data output from the coding processing unit. Is. The speaker 114 is an electroacoustic converter capable of reproducing audio data.

The external output unit 115 can output image data, moving image data, audio data, and the like to an external device. The external output unit 115 is composed of, for example, a video terminal, a microphone terminal, a headphone terminal, and the like.

The data bus 116 is a data bus for transmitting various data such as audio data, moving image data, and image data, and various control signals to each block of the image pickup apparatus 100.

The non-volatile memory 117 is a non-volatile memory, and stores a program or the like described later executed by the control unit 111. In addition, voice data is recorded in the non-volatile memory 117. This audio data includes, for example, a focusing sound output when the subject is in focus, an electronic shutter sound output when shooting is instructed, an operation sound output when the image pickup device 100 is operated, and the like. It is voice data of electronic sound.

<Operation of image pickup device 100>
From now on, the operation of the image pickup apparatus 100 of this Example will be described.

The image pickup apparatus 100 of the present embodiment supplies electric power to each component of the image pickup apparatus from a power source (not shown) in response to the user operating the power switch 72 to turn on the power. For example, the power source is a battery such as a lithium ion battery or an alkaline manganese dry battery.

The control unit 111 determines, for example, which mode of the shooting mode and the reproduction mode operates based on the state of the mode changeover switch 60 according to the power supply. In the moving image recording mode, the control unit 111 records the moving image data output from the imaging unit 101 and the audio data output from the audio input unit 104 as one moving image data with audio. In the reproduction mode, the control unit 111 controls the recording control unit 109 to read out the image data or the moving image data recorded on the recording medium 110 and display the image data or the moving image data on the display unit 107.

First, the moving image recording mode will be described. In the moving image recording mode, first, the control unit 111 transmits a control signal to each component of the image pickup device 100 so as to shift the image pickup device 100 to the shooting standby state. For example, the control unit 111 controls the image pickup unit 101 and the voice input unit 104 to perform the following operations.

The image pickup unit 101 converts the optical image of the subject imaged on the image pickup surface through the optical lens 300 into an electric signal, and generates moving image data from the electric signal generated by the image pickup element. Then, the image pickup unit 101 transmits the moving image data to the display control unit 106, and the display unit 107 displays the moving image data. The user can prepare for shooting while viewing the moving image data displayed on the display unit 107.

The audio input unit 104 A / D-converts analog audio signals input from a plurality of microphones to generate a plurality of digital audio signals. Then, the voice input unit 104 generates voice data of a plurality of channels from the plurality of digital voice signals. The voice input unit 104 transmits the generated voice data to the voice output unit 113, and reproduces the voice data from the speaker 114. The user can adjust the volume of the audio data recorded in the moving image data with audio by the operation unit 112 while listening to the audio data reproduced from the speaker 114.

Next, in response to the user pressing the LV button 76, the control unit 111 transmits an instruction signal for starting shooting to each component of the image pickup apparatus 100. For example, the control unit 111 controls the image pickup unit 101, the voice input unit 104, the coding processing unit 108, and the recording control unit 109 to perform the following operations.

The image pickup unit 101 converts the optical image of the subject imaged on the image pickup surface through the optical lens 300 into an electric signal, and generates moving image data from the electric signal generated by the image pickup element. Then, the image pickup unit 101 transmits the moving image data to the display control unit 106, and the display unit 107 displays the moving image data. Further, the imaging unit 101 transmits the generated moving image data to the volatile memory 105.

The audio input unit 104 A / D-converts analog audio signals input from a plurality of microphones to generate a plurality of digital audio signals. Then, the voice input unit 104 generates multi-channel voice data from the plurality of digital voice signals. Then, the voice input unit 104 transmits the generated voice data to the volatile memory 105.

The coding processing unit 108 reads out the moving image data and the audio data temporarily recorded in the volatile memory 105 and encodes them respectively. The control unit 111 generates a data stream from the moving image data and the audio data encoded by the coding processing unit 108, and outputs the data stream to the recording control unit 109. The recording control unit 109 records the input data stream as moving image data with audio on the recording medium 110 according to a file system such as UDF or FAT.

Each component of the image pickup apparatus 100 continues the above operation during movie shooting.

Then, in response to the user pressing the LV button 76, the control unit 111 transmits an instruction signal for the end of shooting to each component of the image pickup apparatus 100. For example, the control unit 111 controls the image pickup unit 101, the voice input unit 104, the coding processing unit 108, and the recording control unit 109 to perform the following operations.

The image pickup unit 101 stops the generation of moving image data. The voice input unit 104 stops the generation of voice data.

The coding processing unit 108 reads out and encodes the remaining moving image data and audio data recorded in the volatile memory 105. The control unit 111 generates a data stream from the moving image data and the audio data encoded by the coding processing unit 108, and outputs the data stream to the recording control unit 109.

The recording control unit 109 records the data stream on the recording medium 110 as a file of moving image data with audio according to a file system such as UDF or FAT. Then, the recording control unit 109 completes the moving image data with audio in response to the stoppage of the input of the data stream. When the moving image data with sound is completed, the recording operation of the image pickup apparatus 100 is stopped.

The control unit 111 transmits a control signal to each component of the image pickup apparatus 100 so as to shift to the shooting standby state when the recording operation is stopped. As a result, the control unit 111 controls the image pickup apparatus 100 to return to the shooting standby state.

Next, the reproduction mode will be described. In the reproduction mode, the control unit 111 transmits a control signal to each component of the image pickup apparatus 100 so as to shift to the reproduction state. For example, the control unit 111 controls the coding processing unit 108, the recording control unit 109, the display control unit 106, and the voice output unit 113 to perform the following operations.

The recording control unit 109 reads out the moving image data with sound recorded on the recording medium 110 and transmits the read moving image data with sound to the coding processing unit 108.

The coding processing unit 108 decodes the image data and the audio data from the moving image data with audio. The coding processing unit 108 transmits the decoded moving image data to the display control unit 106 and the decoded audio data to the audio output unit 113, respectively.

The display control unit 106 displays the decoded image data by the display unit 107. The voice output unit 113 reproduces the decoded voice data by the speaker 114.

As described above, the image pickup apparatus 100 of this embodiment can record and reproduce image data and audio data.

In this embodiment, the voice input unit 104 executes voice processing such as adjustment processing of the level of the voice signal input from the microphone. In this embodiment, the voice input unit 104 executes this voice processing in response to the start of video recording. Note that this audio processing may be executed after the power of the image pickup apparatus 100 is turned on. Further, this audio processing may be executed depending on the shooting mode selected. Further, this voice processing may be executed depending on the selection of a mode related to voice recording such as a moving image recording mode and a voice memo function. Further, this voice processing may be executed in response to the start of recording of the voice signal.

<Structure of voice input unit 104>
FIG. 3 is a block diagram showing an example of a detailed configuration of the voice input unit 104 in this embodiment.

In this embodiment, the voice input unit 104 has three microphones, an L microphone 201a, an R microphone 201b, and a noise microphone 201c. The L microphone 201a and the R microphone 201b are examples of the first microphone, respectively. In this embodiment, the image pickup apparatus 100 picks up the ambient sound by the L microphone 201a and the R microphone 201b, and records the audio signals input from the L microphone 201a and the R microphone 201b in a stereo system. For example, the environmental sound is a user's voice, an animal's bark, a rain sound, and a sound generated outside the housing of the image pickup device 100 and the housing of the optical lens 300 such as music.

The noise microphone 201c is an example of the second microphone. The noise microphone 201c is a microphone for collecting noise such as drive sound from a predetermined noise source (noise source) generated in the housing of the image pickup apparatus 100 and the housing of the optical lens 300. .. The noise source is, for example, a motor such as an ultrasonic motor (USM) and a stepper motor (STM). Noise (noise) is, for example, vibration noise generated by driving a motor such as USM and STM. For example, the motor is driven in an AF process for focusing on the subject. The image pickup device 100 acquires noise such as driving sound generated in the housing of the image pickup device 100 and the housing of the optical lens 300 by the noise microphone 201c, and uses the acquired noise voice data to generate noise described later. Generate parameters. In this embodiment, the L microphone 201a, the R microphone 201b, and the noise microphone 201c are omnidirectional microphones. An arrangement example of the L microphone 201a, the R microphone 201b, and the noise microphone 201c in this embodiment will be described later with reference to FIG.

The L microphone 201a, the R microphone 201b, and the noise microphone 201c each generate an analog audio signal from the collected sound and input it to the A / D conversion unit 202. Here, the audio signal input from the L microphone 201a is referred to as Lch, the audio signal input from the R microphone 201b is referred to as Rch, and the audio signal input from the noise microphone 201c is referred to as Nch.

The A / D conversion unit 202 converts the analog audio signal input from the L microphone 201a, the R microphone 201b, and the noise microphone 201c into a digital audio signal. The A / D conversion unit 202 outputs the converted digital audio signal to the FFT unit 203. In this embodiment, the A / D conversion unit 202 converts an analog audio signal into a digital audio signal by executing sampling processing with a sampling frequency of 48 kHz and a bit depth of 16 bits.

The FFT unit 203 performs a fast Fourier transform process on the digital audio signal in the time domain input from the A / D conversion unit 202, and converts it into a digital audio signal in the frequency domain. In this embodiment, the digital audio signal in the frequency domain has a frequency spectrum of 1024 points in the frequency band from 0 Hz to 48 kHz. Further, the digital audio signal in the frequency domain has a frequency spectrum of 513 points in the frequency band from 0 Hz to 24 kHz, which is the Nyquist frequency. In this embodiment, the image pickup apparatus 100 performs noise reduction processing by using the frequency spectrum of 513 points from 0 Hz to 24 kHz in the audio data output from the FFT unit 203.

Here, the frequency spectrum of the Lch subjected to the fast Fourier transform is represented by the array data of 513 points from Lch_Before [0] to Lch_Before [512]. When these sequence data are generically referred to, they are described as Lch_Before. Further, the frequency spectrum of the Rch subjected to the fast Fourier transform is represented by the array data of 513 points from Rch_Before [0] to Rch_Before [512]. When these sequence data are generically referred to, they are described as Rch_Before. Here, in this embodiment, Lch_Before [0] is the frequency spectrum of the voice of 0 Hz, and Lch_Before [512] is the frequency spectrum of the voice of 24 kHz. Note that Lch_Before and Rch_Before are examples of the first frequency spectrum data, respectively.

Further, the frequency spectrum of the Nch subjected to the fast Fourier transform is represented by the array data of 513 points from Nch_Before [0] to Nch_Before [512]. When these sequence data are generically referred to, they are described as Nch_Before. Note that Nch_Before is an example of the second frequency spectrum data.

The noise data generation unit 204 generates data for reducing noise contained in Lch_Before and Rch_Before based on Nch_Before. In this embodiment, the noise data generation unit 204 generates array data of NL [0] to NL [512] for reducing noise contained in Lch_Before [0] to Lch_Before [512] by using noise parameters. do. Further, the noise data generation unit 204 generates array data of NR [0] to NR [512] for reducing noise contained in Rch_Before [0] to Rch_Before [512], respectively. The frequency points in the sequence data of NL [0] to NL [512] are the same as the frequency points in the sequence data of Lch_Before [0] to Lch_Before [512]. Further, the frequency points in the sequence data of NR [0] to NR [512] are the same as the frequency points in the sequence data of Rch_Before [0] to Rch_Before [512].

In addition, when the sequence data of NL [0] to NL [512] is generically referred to, it is described as NL. When NR [0] to NR [512] are generically referred to, they are described as NR. NL and NR are examples of the third frequency spectrum data, respectively.

The noise parameter recording unit 205 records noise parameters for the noise data generation unit 204 to generate NL and NR from Nch_Before.

In this embodiment, a plurality of types of lenses can be attached to the image pickup apparatus 100. Multiple types of lenses differ from each other in focal length, lens configuration, motor type, and the like. Therefore, the characteristics of the noise generated by each lens are different. The noise parameter recording unit 205 records a plurality of noise parameters corresponding to a plurality of types of lenses. For example, the noise parameter recording unit 205 records the first noise parameter corresponding to the first type lens and the second noise parameter corresponding to the second type lens. When the noise parameters for generating NL from Nch_Before are generically referred to as PLx. When the noise parameters for generating NR from Nch_Before are generically referred to as PRx.

PLx and PRx have the same number of sequences as NL and NR, respectively. For example, PL1 is array data of PL1 [0] to PL1 [512]. Further, the frequency point of PL1 is the same as the frequency point of Lch_Before. Further, for example, PR1 is sequence data of PR1 [0] to PR1 [512]. The frequency point of PR1 is the same frequency point as Rch_Before. The noise parameters will be described later with reference to FIG.

The noise parameter selection unit 206 determines the noise parameter used in the noise data generation unit 204 from the noise parameter recorded in the noise parameter recording unit 205. The noise parameter selection unit 206 determines the noise parameter used in the noise data generation unit 204 based on data such as lens information received from Lch_Before, Rch_Before, Nch_Before, and the lens control unit 102. The operation of the noise parameter selection unit 206 will be described in detail later with reference to FIG.

In this embodiment, the noise parameter recording unit 205 records all the coefficients for each frequency spectrum of 513 points as noise parameters. However, it is sufficient that at least the coefficient of the frequency point necessary for reducing noise is recorded instead of the coefficient for all frequencies of 513 points. For example, the noise parameter recording unit 205 may record coefficients for each of the frequency spectra of 20 Hz to 20 kHz, which are considered to be typical audible frequencies, as noise parameters, and may not record coefficients of other frequency spectra. Further, for example, as a noise parameter, the coefficient for the frequency spectrum in which the value of the coefficient is zero may not be recorded in the noise parameter recording unit 205.

The subtraction processing unit 207 subtracts NL and NR from Lch_Before and Rch_Before, respectively. For example, the subtraction processing unit 207 has an L subtractor 207a that subtracts NL from Lch_Before, and an R subtractor 207b that subtracts NR from Rch_Before. The L subtractor 207a subtracts NL from Lch_Before and outputs 513 point sequence data of Lch_After [0] to Lch_After [512]. The R subtractor 207b subtracts NR from Rch_Before and outputs 513 point sequence data of Rch_After [0] to Rch_After [512]. In this embodiment, the subtraction processing unit 207 executes the subtraction processing by the spectral subtraction method.

The iFFT unit 208 converts the digital audio signal in the frequency domain input from the subtraction processing unit 207 into a digital audio signal in the time domain by performing an inverse fast Fourier transform (inverse Fourier transform).

The audio processing unit 209 executes audio processing for a digital audio signal in the time domain, such as an equalizer, an auto-level controller, and a stereo feeling enhancement process. The voice processing unit 209 outputs the voice-processed voice data to the volatile memory 105.

In this embodiment, the image pickup apparatus 100 has two microphones as the first microphone, but the image pickup apparatus 100 may use the first microphone as one microphone or three or more microphones. For example, when the image pickup apparatus 100 has one microphone as the first microphone in the voice input unit 104, the voice data acquired by one microphone is recorded in a monaural manner. Further, for example, when the image pickup apparatus 100 has three or more microphones as the first microphone in the voice input unit 104, the sound data acquired by the three or more microphones is recorded in a surround system.

In this embodiment, the L microphone 201a, the R microphone 201b, and the noise microphone 201c are omnidirectional microphones, but these microphones may be directional microphones.

<Arrangement of microphones in voice input unit 104>
Here, an example of arranging the microphone of the voice input unit 104 of this embodiment will be described. FIG. 4 shows an arrangement example of the L microphone 201a, the R microphone 201b, and the noise microphone 201c.

FIG. 4 is an example of a cross-sectional view of a portion of the image pickup apparatus 100 to which the L microphone 201a, the R microphone 201b, and the noise microphone 201c are attached. The portion of the image pickup apparatus 100 is composed of an exterior portion 302, a microphone bush 303, and a fixed portion 304.

The exterior portion 302 has a hole (hereinafter referred to as a microphone hole) for inputting an environmental sound into the microphone. In this embodiment, the microphone hole is formed above the L microphone 201a and the R microphone 201b. On the other hand, the noise microphone 201c is provided to acquire the driving sound generated in the housing of the image pickup apparatus 100 and the housing of the optical lens 300, and it is not necessary to acquire the environmental sound. Therefore, in this embodiment, the microphone hole is not formed above the noise microphone 201c in the exterior portion 302.

The drive sound generated in the housing of the image pickup apparatus 100 and the housing of the optical lens 300 is acquired by the L microphone 201a and the R microphone 201b through the microphone holes. When a driving sound or the like is generated in the housing of the image pickup apparatus 100 and the optical lens 300 in a state where the environmental sound is small, the sound acquired by each microphone is mainly the driving sound. Therefore, the sound level from the noise microphone 201c is higher than the sound level from the L microphone 201a and the R microphone 201b. That is, in this case, the relationship between the levels of the audio signals output from each microphone is as follows.
Lch ≒ Rch <Nch
Further, when the environmental sound becomes louder, the sound level of the environmental sound from the L microphone 201a and the R microphone 201b becomes louder than the sound level of the drive sound generated by the image pickup device 100 or the optical lens 300 from the microphone 201c. .. Therefore, in this case, the relationship between the levels of the audio signals output from each microphone is as follows.
Lch ≒ Rch> Nch
In this embodiment, the shape of the microphone hole formed in the exterior portion 302 is elliptical, but it may be another shape such as a circular shape or a square shape. Further, the shape of the microphone hole on the microphone 201a and the shape of the microphone hole on the microphone 201b may be different from each other.

In this embodiment, the noise microphone 201c is arranged so as to be close to the L microphone 201a and the R microphone 201b. Further, in this embodiment, the noise microphone 201c is arranged between the L microphone 201a and the R microphone 201b. As a result, the audio signal generated by the noise microphone 201c from the drive sound or the like generated in the housing of the image pickup apparatus 100 and the housing of the optical lens 300 is generated by the L microphone 201a and the R microphone 201b from the drive sound or the like. It becomes a signal similar to an audio signal.

The microphone bush 303 is a member for fixing the L microphone 201a, the R microphone 201b, and the noise microphone 201c. The fixing portion 304 is a member that fixes the microphone bush 303 to the exterior portion 302.

In this embodiment, the exterior portion 302 and the fixing portion 304 are made of a mold member such as a PC material. Further, the exterior portion 302 and the fixing portion 304 may be made of a metal member such as aluminum or stainless steel. Further, in this embodiment, the Mike Busch 303 is made of a rubber material such as ethylene propylene diene rubber.

<Noise parameter>
FIG. 5 is an example of noise parameters recorded in the noise parameter recording unit 205. The noise parameter shown in FIG. 5 is a noise parameter for noise generated from a certain lens. The noise parameter is a parameter for correcting the audio signal generated by the noise microphone 201c acquiring the driving sound generated in the housing of the image pickup apparatus 100 and the housing of the optical lens 300. As shown in FIG. 5, in this embodiment, PLx and PRx are recorded in the noise parameter recording unit 205. In this embodiment, the source of the driving sound will be described as being inside the housing of the optical lens 300. The drive sound generated in the housing of the optical lens 300 is transmitted to the housing of the image pickup apparatus 100 via the lens mount 301, and is acquired by the L microphone 201a, the R microphone 201b, and the noise microphone 201c.

The frequency of the drive sound differs depending on the type of drive sound. Therefore, in this embodiment, the image pickup apparatus 100 records a plurality of noise parameters corresponding to the types of drive sounds (noise). Then, noise data is generated using any one of these plurality of noise parameters. In this embodiment, the image pickup apparatus 100 records a noise parameter with respect to white noise as constant noise. Further, the image pickup apparatus 100 records noise parameters for short-term noise generated by, for example, meshing of gears in the optical lens 300. Further, the image pickup apparatus 100 records noise parameters for sliding noise in the housing of the lens 300, for example, as long-term noise. In addition, the image pickup apparatus 100 may record noise parameters for each type of the optical lens 300, and for each temperature inside the housing of the image pickup apparatus 100 and the inclination of the image pickup apparatus 100 detected by the information acquisition unit 103.

<How to generate noise data>
The noise data generation process in the noise data generation unit 204 will be described with reference to FIGS. 6 and 7. Here, the noise data generation process for the Lch data will be described, but the noise data generation method for the Rch data is also the same.

First, a process of generating a noise parameter will be described in a situation where it can be considered that there is no environmental sound. FIG. 6A is an example of the frequency spectrum of Lch_Before when a driving sound is generated in the housing of the optical lens 300 in a situation where it can be considered that there is no environmental sound. FIG. 6B is an example of the frequency spectrum of Nch_Before when a driving sound is generated in the housing of the optical lens 300 in a situation where it can be considered that there is no environmental sound. The horizontal axis is the axis showing the frequency from the 0th point to the 512th point, and the vertical axis is the axis showing the amplitude of the frequency spectrum.

In Lch_Before and Nch_Before, the amplitude of the frequency spectrum in the same frequency band becomes large because the situation can be regarded as having no environmental sound. Further, since the driving sound is generated in the housing of the optical lens 300, the amplitude of each frequency spectrum with respect to the same driving sound tends to be larger in Nch_Before than in Lch_Before.

FIG. 6C is an example of PLx in this embodiment. In this embodiment, PLx is a coefficient of each frequency spectrum calculated by dividing the amplitude of each frequency spectrum of Lch_Before by the amplitude of each frequency spectrum of Nch_Before. The result of this division is described as Lch_Before / Nch_Before. That is, PLx is the ratio of the amplitudes of Lch_Before and Nch_Before. The noise parameter recording unit 205 records the value of Lch_Before / Nch_Before as the noise parameter PLx. As described above, since the amplitude of the frequency spectrum for the same drive sound tends to be larger in Nch_Before than in Lch_Before, the value of each coefficient of the noise parameter PLx tends to be smaller than 1. However, when the value of Nch_Before [n] is smaller than a predetermined threshold value, the noise parameter recording unit 205 records the noise parameter PLx with PLx [n] = 0.

Next, the process of applying the generated noise parameter to Nch_Before will be described. FIG. 7A is an example of the frequency spectrum of Lch_Before when the driving sound is generated in the housing of the optical lens 300 in the presence of the environmental sound. FIG. 7B is an example of the frequency spectrum of Nch_Before when the driving sound is generated in the housing of the optical lens 300 in the presence of the environmental sound. The horizontal axis is the axis showing the frequency from the 0th point to the 512th point, and the vertical axis is the axis showing the amplitude of the frequency spectrum.

FIG. 7C is an example of NL in the case where the driving sound is generated in the housing of the optical lens 300 in the presence of the environmental sound. The noise data generation unit 204 multiplies each frequency spectrum of Nch_Before by each coefficient of PLx to generate NL. NL is the frequency spectrum thus generated.

FIG. 7D is an example of Lch_After when a driving sound is generated in the housing of the optical lens 300 in a situation where an environmental sound is present. The subtraction processing unit 207 subtracts NL from Lch_Before to generate Lch_After. Lch_After is a frequency spectrum thus generated.

As a result, the image pickup apparatus 100 can reduce the noise caused by the driving sound in the housing of the optical lens 300 and record the environmental sound with less noise.

<Explanation of noise parameter selection unit 206>
FIG. 8 is a block diagram showing an example of a detailed configuration of the noise parameter selection unit 206.

Lch_Before, Rch_Before, Nch_Before, lens information, and a lens control signal are input to the noise parameter selection unit 206.

The Nch noise detection unit 2061 detects noise due to the driving sound generated in the housing of the optical lens 300 from Nch_Before. In this embodiment, the Nch noise detection unit 2061 detects noise by using the data of 513 points of Nch_Before.

The environmental sound detection unit 2062 detects the level of the environmental sound from Lch_Before and Rch_Before.

The noise determination unit 2063 determines the noise parameters used by the noise data generation unit 204 based on the lens information and the lens control signal, the data input from the Nch noise detection unit 2061, and the data input from the environmental sound detection unit. Among the noise parameters recorded in the noise parameter recording unit 205, the noise determination unit 2063 uses the noise parameters corresponding to the lens type indicated by the lens information from the lens control unit 102 as the noise parameters used by the noise data generation unit 204. decide. The noise determination unit 2063 outputs data indicating the type of the determined noise parameter to the noise data generation unit 204.

The Nch differential unit 2064 executes the differential process on Nch_Before. The Nch differentiation unit 2064 outputs data indicating the result of differential processing of Nch_Before to the short-term noise detection unit 2065. The short-term noise detection unit 2065 detects whether or not short-term noise is generated based on the data input from the Nch differentiation unit 2064. The short-term noise detection unit 2065 outputs data indicating whether or not short-term noise is generated to the noise determination unit 2063. The Nch differentiation unit 2064 and the short-term noise detection unit 2065 are included in the Nch noise detection unit 2061.

The Nch integration unit 2066 executes an integration process on Nch_Before. The Nch integration unit 2066 outputs data indicating the result of differential processing of Nch_Before to the long-term noise detection unit 2067. The long-term noise detection unit 2067 detects whether or not long-term noise is generated based on the data input from the Nch integration unit 2066. The long-term noise detection unit 2067 outputs data indicating whether or not long-term noise is generated to the noise determination unit 2063. The Nch integration unit 2066 and the long-term noise detection unit 2067 are included in the Nch noise detection unit 2061.

The environmental sound extraction unit 2068 extracts the environmental sound. In this embodiment, the environmental sound extraction unit 2068 extracts frequency data that is less affected by noise based on the noise meter. For example, the environmental sound extraction unit 2068 extracts frequency data having a noise parameter of a predetermined value or less. Then, the environmental sound extraction unit 2068 outputs data indicating the magnitude of the environmental sound based on the extracted frequency data. The environmental sound extraction unit 2068 is included in the environmental sound detection unit 2062.

The environmental sound determination unit 2069 determines the loudness of the environmental sound. The environmental sound determination unit 2069 inputs data indicating the loudness of the determined environmental sound to the Nch noise detection unit 2061 and the noise determination unit 2063. The Nch noise detection unit 2061 changes the first threshold value and the second threshold value, which will be described later, based on the data indicating the loudness of the environmental sound input from the environmental sound determination unit 2069. The environmental sound determination unit 2069 is included in the environmental sound detection unit 2062.

<Timing chart of noise reduction processing>
The noise reduction processing in this embodiment will be described with reference to FIG.

9 (a) to 9 (i) are examples of timing charts of voice processing in the noise data generation unit 204, the noise parameter selection unit 206, and the subtraction processing unit 207. In this embodiment, the Lch voice processing will be described for the sake of simplification of the description, but the same applies to the Rch voice processing. The horizontal axes of the graphs in FIGS. 9 (a) to 9 (i) are all time axes.

FIG. 9A shows an example of a lens control signal. The lens control signal is a signal instructing the lens control unit 102 to drive the optical lens 300. In this embodiment, the level of the lens control signal is represented by two values, High and Low. When the level of the lens control signal is High, the lens control unit 102 is instructing the optical lens 300 to drive the lens 300. When the level of the lens control signal is Low, the lens control unit 102 is in a state in which the optical lens 300 is not instructed to drive.

FIG. 9B is a graph showing an example of the value of Lch_Before [n]. The vertical axis is an axis indicating the value of Lch_Before [n]. In this embodiment, Lch_Before [n] is a signal at the nth frequency point in which the signal indicating the driving sound of the optical lens 300 is characteristically displayed among the Lch_Before output from the FFT unit 203. In this embodiment, the signal at the nth frequency point will be described, but voice processing is similarly executed for other frequencies. Further, the signals represented by the signal X and the signal Y are signals containing noise. In this embodiment, the signal X indicates a signal including short-term noise. The signal Y indicates a noise signal including long-term noise.

FIG. 9C is a graph showing an example of the magnitude of the environmental sound extracted by the environmental sound extraction unit 2068. The vertical axis shows the level of the audio signal generated from the acquired environmental sound. The threshold value Th1 and the threshold value Th2 are two threshold values used in the environmental sound determination unit 2069.

FIG. 9D is a graph showing an example of the value of Nch_Before [n]. Nch_Before [n] is a signal at the nth frequency point in which the signal indicating the driving sound of the optical lens 300 is characteristically displayed among the Nch_Before output from the FFT unit 203. The vertical axis is an axis indicating the value of Nch_Before [n]. In Nch_Before [n], the noise signal shown by the signal X and the signal Y in FIG. 9B appears more characteristically than in Lch_Before.

FIG. 9 (e) is a graph showing an example of the value of Ndiff [n]. Ndiff [n] indicates the value of the signal at the nth frequency point in the Ndiff output from the Nch differential unit 2064. The vertical axis is an axis indicating the value of Ndiff [n]. When the amount of change in the value of Nch_Before [n] per predetermined time is large, the value of Ndiff [n] becomes large. The short-term noise detection unit 2065 has a threshold value Th_Ndiff [n], which is a first threshold value, in order to detect short-term noise. The threshold value Th_Ndiff [n] changes between levels 1 to 3 based on the data indicating the magnitude of the environmental sound input from the environmental sound determination unit 2069 and the lens control signal. The initial value of the threshold value Th_Ndiff [n] is level 2. The level of the threshold value Th_Ndiff [n] is represented by a horizontal broken line.

FIG. 9 (f) is a graph showing an example of the value of Nint [n]. In this embodiment, Nint [n] indicates the value of the signal at the nth frequency point of the Nint output from the Nch integrating unit 2066. The vertical axis is an axis indicating the value of Nint [n]. When Nch_Before [n] is continuously large, the value of Nint [n] becomes large. The long-term noise detection unit 2067 has a threshold value Th_Nint [n], which is a second threshold value, in order to detect long-term noise. The threshold value Th_Nint [n] changes between levels 1 to 3 based on the data indicating the loudness of the environmental sound input from the environmental sound determination unit 2069 and the lens control signal. The initial value of the threshold value Th_Nint [n] is level 2. The level of the threshold value Th_Nint [n] is represented by a horizontal broken line.

FIG. 9 (g) shows an example of the noise parameter selected by the noise parameter selection unit 206. In this embodiment, the plain portion indicates that only the noise parameter of PL1 is selected. The shaded area indicates that the noise parameters of PL1 and PL2 are selected. The plaid portion indicates that the noise parameters of PL1 and PL3 are selected.

FIG. 9 (h) is a graph showing an example of the value of NL [n]. In this embodiment, NL [n] indicates the value of the signal at the nth frequency point among the NLs generated by the noise data generation unit 204. The vertical axis is an axis indicating the value of NL [n].

FIG. 9 (i) is a graph showing an example of the value of Lch_After [n]. In this embodiment, Lch_After [n] indicates the value of the signal at the nth frequency point of the Lch_After output from the subtraction processing unit 207. The vertical axis is an axis indicating the value of Lch_After [n].

Next, the timing of each operation will be described using the times t701 to t709.

At time t701, the lens control unit 102 outputs a High signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9A). Since there is a high possibility that a driving sound is generated in the housing of the optical lens 300 at time t701, the short-term noise detection unit 2065 lowers the threshold value Th_Ndiff [n] to level 1 (FIG. 9 (e)). Further, at time t701, since there is a high possibility that a driving sound is generated in the housing of the optical lens 300, the long-term noise detection unit 2067 lowers the threshold value Th_Nint [n] to level 1 (FIG. 9 (f)).

At time t702, the optical lens 300 is driven, and a short-term driving sound such as a gear meshing sound is generated. The value of Ndiff [n] exceeds the threshold value Th_Ndiff [n] due to the noise microphone 201c picking up the short-term drive sound (FIG. 9 (e)). In response to this, the noise parameter selection unit 206 selects the noise parameters PL1 and PL2 (FIG. 9 (g)). The noise data generation unit 204 generates NL [n] based on Nch_Before [n] and the noise parameters PL1 and PL2 (FIG. 9 (h)). The subtraction processing unit 207 subtracts NL [n] from Lch_Before [n] and outputs Lch_After [n] (FIG. 9 (i)). In this case, Lch_After [n] becomes an audio signal in which constant noise and short-term noise are reduced.

At time t703, the optical lens 300 starts continuous driving, and a long-term driving sound such as a sliding sound is generated in the housing of the optical lens 300. The value of Nint [n] exceeds the threshold value Th_Nint [n] due to the noise microphone 201c picking up the long-term driving sound (FIG. 9 (f)). In response to this, the noise parameter selection unit 206 selects the noise parameters PL1 and PL3 (FIG. 9 (g)). The noise data generation unit 204 generates NL [n] based on Nch_Before [n] and the noise parameters PL1 and PL3 (FIG. 9 (h)). The subtraction processing unit 207 subtracts NL [n] from Lch_Before [n] and outputs Lch_After [n] (FIG. 9 (i)). In this case, Lch_After [n] becomes an audio signal in which constant noise and long-term noise are reduced.

At time t704, the optical lens 300 stops continuous driving. Since the noise microphone 201c does not collect the long-term driving sound, the value of Nint [n] becomes equal to or less than the threshold value Th_Nint [n] (FIG. 9 (f)). In response to this, the noise parameter selection unit 206 selects the noise parameter PL1 (FIG. 9 (g)). The noise data generation unit 204 generates NL [n] based on Nch_Before [n] and the noise parameter PL1 (FIG. 9 (h)). The subtraction processing unit 207 subtracts NL [n] from Lch_Before [n] and outputs Lch_After [n] (FIG. 9 (i)). In this case, Lch_After [n] becomes an audio signal with constant noise reduced.

At time t705, the lens control unit 102 outputs a Low signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9A). In this case, since the possibility that a driving sound is generated in the housing of the optical lens 300 is low, the short-term noise detection unit 2065 raises the threshold value Th_Ndiff [n] to level 2 (FIG. 9 (e)). Further, in this case, since the possibility that the driving sound is generated in the housing of the optical lens 300 is low, the long-term noise detection unit 2067 raises the threshold value Th_Nint [n] to level 2 (FIG. 9 (f)).

At time t706, the loudness of the environmental sound extracted by the environmental sound extraction unit 2068 exceeds the threshold value Th1. When the ambient sound is loud, the noise contained in the audio signal is less likely to be perceived by the user, so the short-term noise detection unit 2065 raises the threshold value Th_Ndiff [n] to level 3 (FIG. 9 (e)). Further, when the environmental sound is loud, the noise contained in the audio signal is less likely to be perceived by the user, so that the long-term noise detection unit 2067 raises the threshold value Th_Nint [n] to level 3 (FIG. 9 (f)).

At time t707, the lens control unit 102 outputs a High signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9A). In this case, since there is a high possibility that a driving sound is generated in the housing of the optical lens 300, the short-term noise detection unit 2065 lowers the threshold value Th_Ndiff [n] to level 2 (FIG. 9 (e)). Further, in this case, since there is a high possibility that a driving sound is generated in the housing of the optical lens 300, the long-term noise detection unit 2067 lowers the threshold value Th_Nint [n] to level 2 (FIG. 9 (f)).

At time t708, the loudness of the environmental sound extracted by the environmental sound extraction unit 2068 exceeds the threshold value Th2. Here, the noise parameter selection unit 206 selects only the noise parameter PL1 regardless of the data input from the Nch noise detection unit 2061. As described above, when the environmental sound is very loud, it is difficult for the user to perceive the noise contained in the audio signal. Therefore, the image pickup apparatus 100 further reduces the short-term noise and the long-term noise by reducing only the constant noise. Record more natural environmental sounds than when processing is performed.

As described above, the image pickup apparatus 100 can record the environmental sound with reduced noise by executing the noise reduction processing using the noise microphone 201c which is the second microphone.

Further, the image pickup apparatus 100 detects that noise is generated by using the output signal of the noise microphone 201c, and sets the noise parameter according to the timing at which the generation of noise is detected. Therefore, the image pickup apparatus 100 can appropriately set the noise parameter in synchronization with the generation of noise and appropriately reduce the noise.

Further, when the loudness of the environmental sound is equal to or less than the threshold value Th2, the image pickup apparatus 100 executes noise reduction processing according to the noise detected by the Nch noise detection unit 206, and when the environmental sound is larger than the threshold value Th2, it is constant. Noise is reduced. This makes it possible for the image pickup apparatus 100 to record noise-reduced environmental sounds according to the loudness of the environmental sounds so that the user does not feel uncomfortable.

In this embodiment, the image pickup device 100 reduces the drive sound generated in the housing of the optical lens 300, but the drive sound generated in the image pickup device 100 may be reduced. The driving sound generated in the image pickup apparatus 100 is, for example, the noise of the substrate and the radio wave noise. The squeal of the substrate is, for example, a sound generated by a squeak of the substrate generated when a voltage is applied to a capacitor on the substrate.

The threshold values Th1 and Th2 of the environmental sound determination unit 2069, the threshold value Th_Ndiff [n] of the short-term noise detection unit 2065, and the threshold value Th_Nint [n] of the long-term noise detection unit 2067 are based on the generated driving sound and the environmental sound. Will be decided. Therefore, the image pickup apparatus 100 may change these threshold values depending on the type of the optical lens 300, the inclination of the image pickup apparatus 100, and the like.

<Noise parameter for each lens>
First, when the optical lens 300 is replaced, the case where the same noise parameter is applied to each lens will be described with reference to FIGS. 7 and 11. In FIGS. 7 and 11, it is assumed that the image pickup apparatus 100 is equipped with different lenses. In addition, it is assumed that the same environmental sound is generated in FIGS. 7 and 11.

7 (a) and 11 (a) show the amplitude of each frequency spectrum of Lch_Before. 7 (b) and 11 (b) show the amplitude of each frequency spectrum of Nch_Before. Here, when comparing the respective frequency spectra, it can be read that they are different from each other in at least a part. This is because the noise picked up by each of the L microphone 201a, the R microphone 201b, and the noise microphone 201c differs depending on the shape and structure of the lens and the number and position of noise sources such as the drive body included in the lens. Is.

7 (c) and 11 (c) show the amplitude of each frequency spectrum of NL. Here, the same noise parameter is used to generate the NL regardless of the lens mounted on the image pickup apparatus 100.

7 (d) and 11 (d) show the amplitude of each frequency spectrum of Lch_After. Since Lch_After is an audio signal recorded as an environmental sound, ideally, the frequency spectra shown in FIGS. 7 (d) and 11 (d) are the same. However, when comparing the frequency spectra of FIGS. 7 (d) and 11 (d), it can be read that they are different from each other in at least a part. This is because the frequency spectrum of NL subtracted from Lch_Before is different from the frequency spectrum of the noise generated in each lens. The reason why the frequency spectrum of NL is different from the frequency spectrum of noise is that the noise parameter is not a parameter created for the noise generated from the lens, but a general-purpose parameter for a certain kind of noise. That is, when the same noise parameter is applied to the noise generated from various lenses, the audio data recorded as the environmental sound on the recording medium 110 becomes different audio data for each lens mounted on the image pickup apparatus 100. It ends up.

Therefore, in this embodiment, the image pickup apparatus 100 effectively reduces the noise and records the environmental sound more accurately by making the noise parameter different for each lens. FIG. 10 is an example of noise parameters for each lens that can be attached to the image pickup apparatus 100 as the optical lens 300. In FIG. 10, the noise parameters of PL2 and PR2 are shown by default values and for each lens. The noise parameter recording unit 205 of this embodiment records the noise parameters for each lens in advance. The noise parameter recording unit 205 also records the other lens parameters PLx and PRx for each lens.

In this embodiment, the noise parameter recording unit 205 records the noise parameter for a lens other than the lens in which the noise parameter is recorded in advance as a default value. This is because the image pickup apparatus 100 can reduce the noise propagated from the lens to some extent by applying the general-purpose noise parameter to the noise uttered from the lens in which the noise parameter is not recorded in advance. ..

From now on, the processing of the image pickup apparatus 100 in the case of exchanging the optical lens 300 will be described with reference to the flowchart of FIG. The processing of the image pickup apparatus 100 is started, for example, when the power switch 72 is operated by the user and the power is turned on.

In step S1201, the control unit 111 controls to supply electric power to each component of the image pickup apparatus 100 from a power source (not shown).

In step S1202, the lens control unit 102 receives lens information from the optical lens 300. The lens information is, for example, the type of lens, the model number of the lens, the type of noise source, and the like. The control unit 111 records the received lens information in the non-volatile memory 117.

In step S1203, the lens control unit 102 determines whether or not the lens has been replaced. For example, the lens control unit 102 compares the lens information recorded in the non-volatile memory 117 prior to the processing in step S1202 with the lens information received in step S1202. When it is determined that the two lens information match, the lens control unit 102 determines that the lenses have not been exchanged. If it is determined that the two lens information do not match, the lens control unit 102 determines that the lenses have been exchanged. Further, for example, when the lens information is not recorded in the non-volatile memory 117 before the process of step S1202, the lens control unit 102 determines that the lens has been replaced. When the lens control unit 102 determines that the lens has been replaced, the process of step S1204 is executed. If the lens control unit 102 determines that the lens has not been replaced, the process of step S1208 is executed.

In step S1204, the noise parameter selection unit 206 receives lens information from the lens control unit 102.

In step S1205, the noise parameter selection unit 206 determines whether or not the noise parameter for the optical lens 300 mounted on the image pickup apparatus 100 is recorded in the noise parameter recording unit 205. For example, the noise parameter selection unit 206 determines whether or not the noise parameter for the lens is recorded in the noise parameter recording unit 205 based on the model number of the lens included in the lens information received in step S1204. When the noise parameter selection unit 206 determines that the noise parameter of the optical lens 300 is registered in the noise parameter recording unit 205, the process of step S1206 is executed. When the noise parameter selection unit 206 determines that the noise parameter of the optical lens 300 is not registered in the noise parameter recording unit 205, the process of step S1207 is executed.

In step S1206, the noise parameter selection unit 206 determines to use the noise parameter for the optical lens 300, and instructs the noise data generation unit 204 to use the noise parameter. For example, the noise parameter selection unit 206 transmits data indicating the model number of the optical lens 300 to the noise data generation unit 204. In this case, when the noise parameter is read from the noise parameter recording unit 205, the noise data generation unit 204 reads the noise parameter for the optical lens 300 instructed by the noise parameter selection unit 206.

In step S1207, the noise parameter selection unit 206 determines to use the default noise parameter, and instructs the noise data generation unit 204 to use the noise parameter. For example, the noise parameter selection unit 206 transmits data indicating a default value to the noise data generation unit 204. In this case, when the noise parameter is read from the noise parameter recording unit 205, the noise data generation unit 204 reads the noise parameter of the default value instructed by the noise parameter selection unit 206.

In step S1208, the control unit 111 determines whether or not to turn off the power. For example, the control unit 111 determines that the power is turned off when the power switch is operated to be turned off. If the control unit 111 determines that the power is turned off, the process of step S1212 is executed. If the control unit 111 determines that the power is not turned off, the process of step S1209 is executed.

In step S1209, the control unit 111 determines whether or not to start video recording. For example, the control unit 111 determines that the moving image recording is started in response to the pressing of the release switch 61. On the contrary, the control unit 111 determines that the moving image recording is not started when the release switch 61 is not pressed. If the control unit 111 determines to start video recording, the process of step S1209 is executed. If the control unit 111 determines that the moving image recording is not started, the process of step S1203 is executed.

In step S1210, the control unit 111 records a moving image. For example, as described above, the control unit 111 records the moving image data with audio on the recording medium 110. In the audio processing during the moving image recording, the noise parameters determined in the processing of step S1206 or the processing of step S1207 are used.

In step S1211, the control unit 111 determines whether or not to end the moving image recording. For example, the control unit 111 determines that the moving image recording is finished in response to the pressing of the release switch 61. On the contrary, the control unit 111 determines that the moving image recording is not finished when the release switch 61 is not pressed. When the control unit 111 determines that the moving image recording is finished, the process of step S1203 is executed. If the control unit 111 determines that the moving image recording is not completed, the process of step S1210 is executed.

In step S1212, the control unit 111 records the lens information in the non-volatile memory 117. The lens information recorded in this step is used when the process of step S1203 is executed next.

The processing of the image pickup apparatus 100 when the optical lens 300 is replaced has been described above.

As described above, by changing the noise parameter according to the optical lens 300 mounted on the image pickup apparatus 100, effective noise reduction processing can be performed.

In this embodiment, the noise parameter is recorded for each lens, but it may be recorded for each type of noise source. For example, the image pickup apparatus 100 prepares noise parameters for each lens of the lens in which USM is used and the lens in which STM is used. In this case, the noise parameter selection unit 206 instructs the noise data generation unit 204 to use the noise parameter for USM or the noise parameter for STM from the type of noise source included in the lens information received from the lens control unit 102. ..

[Second Example]
In the first embodiment, the image pickup apparatus 100 uses noise parameters according to the type of the mounted lens. In the second embodiment, in addition to using the noise parameter according to the type of the mounted lens, the process of using the noise parameter according to the posture of the image pickup apparatus 100 and the state of the lens will be described.

In the second embodiment, the same image pickup apparatus 100 as in the first embodiment will be used. Since the configuration of the image pickup apparatus 100 is the same as that of the first embodiment, the description thereof will be omitted.

FIG. 13 is a diagram showing posture information of the image pickup apparatus 100 in this embodiment. In this embodiment, an angle of 0 ° is defined as a state in which the image pickup direction of the image pickup apparatus 100 is horizontal to the ground. Further, the case where the image pickup direction of the image pickup apparatus 100 is perpendicular to the ground upward is defined as an angle of 90 °. Further, the case where the image pickup direction of the image pickup apparatus 100 is perpendicular to the ground downward is defined as an angle −90 °. Here, the angle θ indicates the angle in the image pickup direction of the image pickup apparatus 100. In this embodiment, the image pickup device 100 acquires the posture information of the image pickup device 100 by the information acquisition unit 103. For example, the information acquisition unit 103 detects the inclination of the image pickup device 100 by an acceleration sensor or a gyro sensor.

In this embodiment, the range of the image pickup device 100 in the image pickup direction is divided into five. Range a is 60 ° to 90 °, range b is 30 ° to 60 °, range c is -30 ° to 30 °, range d is -60 ° to -30 °, and range e is -90 ° to -60 °. do. For example, when the angle θ, which is the image pickup direction of the image pickup apparatus 100, is 50 °, the range b is set. In this embodiment, the range of the image pickup direction of the image pickup apparatus 100 is divided into five, but the range of the image pickup direction of the image pickup apparatus 100 may be a plurality.

The gravity applied to the motor in the lens, which is a noise source in this embodiment, changes according to the posture of the image pickup apparatus 100. When gravity changes, the holding force of the holding mechanism that holds the lens changes, and the movement of the motor that drives this holding mechanism also changes. Therefore, the driving sound generated when the motor is driven also changes according to gravity, and the frequency characteristic of the noise collected by each microphone also changes according to gravity.

FIG. 14A shows Lch_Before when the angles θ indicating the posture of the image pickup apparatus 100 are 0 ° and 50 °. In this embodiment, the amplitude of the frequency spectrum of Lch_Before [400] to Lch_Before [450] changes. Here, in Lch_Before [400] to Lch_Before [450], the peak value of the amplitude of the frequency spectrum at an angle θ of 0 ° is A, and the peak value of the amplitude of the frequency spectrum at an angle θ of 50 ° is A'.

FIG. 14B shows Nch_Before when the angles θ indicating the posture of the image pickup apparatus 100 are 0 ° and 50 °. In this embodiment, the amplitude of the frequency spectrum of Nch_Before [400] to Nch_Before [450] changes. In this embodiment, it is assumed that the positions of the frequency spectra whose amplitudes change are the same in Lch_Before and Nch_Before. Here, in Nch_Before [400] to Nch_Before [450], the peak value of the amplitude of the frequency spectrum at an angle θ of 0 ° is B, and the peak value of the amplitude of the frequency spectrum at an angle θ of 50 ° is B'.

Here, the amount of change in the peak value of the amplitude of the frequency spectrum of Lch_Before due to the change in the posture of the image pickup apparatus 100 | A'-A | and the amount of change in the peak value of the amplitude of the frequency spectrum of Nch_Before | B'-B. Compare with |. With reference to FIG. 14, it can be read that | B'-B |> | A'-A |. This indicates that the amount of change in the volume of the noise picked up by the noise microphone 201c is larger than the amount of change in the volume of the noise picked up by the L microphone 201a. Therefore, in the present embodiment, in order to effectively reduce the noise, the image pickup apparatus 100 corrects the noise parameter in the range in which the amplitude of the frequency spectrum changes according to the change in the posture of the image pickup apparatus 100.

FIG. 15 shows correction noise parameters for each posture information of the image pickup apparatus 100. In this embodiment, the amplitudes of the frequency spectra of Lch_Before, Rch_Before, and Nch_Before will be described as varying in the range of N = 400 to 450. On the other hand, in this embodiment, it is assumed that the amplitudes of the frequency spectra of Lch_Before, Rch_Before, and Nch_Before are sufficiently small changes in the other N ranges so that the noise parameters do not need to be corrected. As shown in FIG. 15, only the frequency spectrum that needs to be corrected is recorded in the noise parameter recording unit 205 as a noise parameter for correction. For example, when the angle θ is 50 °, the noise parameter selection unit 206 instructs the noise data generation unit 204 to use the correction noise parameter in the range b together with the noise parameter to be used. When the angle θ is in the range c, the noise parameter selection unit 206 determines that the noise parameter correction is not necessary, and may not instruct the noise data generation unit 204 to instruct the noise parameter for correction.

FIG. 16 is a comparison of the lengths of zoom lenses when the lens magnifications are different. As shown in FIG. 16, the zoom lens can change the focal length by expanding and contracting the lens barrel and moving the lens position. As a result, the shape and structure of the lens change, so that the noise propagation path and propagation distance from the motor, which is the noise source, to each microphone change, and the frequency spectrum of the noise picked up by the microphone changes. In this embodiment, the image pickup apparatus 100 acquires the zoom magnification of the optical lens 300 by receiving the zoom magnification as lens information from the optical lens 300 by the lens control unit 102.

FIG. 17A shows Lch_Before when the zoom magnification of the optical lens 300 is 1x and 5x. In this embodiment, the amplitude of the frequency spectrum of Lch_Before [200] to Lch_Before [250] changes. Here, in Lch_Before [200] to Lch_Before [250], the peak value of the amplitude of the frequency spectrum when the zoom magnification is 5 times is C, and the peak value of the amplitude of the frequency spectrum when the zoom magnification is 1 times is C'.

FIG. 17B shows Nch_Before when the zoom magnification of the optical lens 300 is 1x and 5x. In this embodiment, the amplitude of the frequency spectrum of Nch_Before [200] to Nch_Before [250] changes. Here, in Nch_Before [200] to Nch_Before [250], the peak value of the amplitude of the frequency spectrum when the zoom magnification is 5 times is D, and the peak value of the amplitude of the frequency spectrum when the zoom magnification is 1 times is D'.

Here, the amount of change in the peak value of the amplitude of the frequency spectrum of Lch_Before due to the change in the zoom magnification of the optical lens 300 | C'-C | and the amount of change in the peak value of the amplitude of the frequency spectrum of Nch_Before | Compare with D |. With reference to FIG. 17, it can be read that | D'-D |> | C'-C |. This is because the amount of change in the volume of the noise picked up by the noise microphone 201c is larger than the amount of change in the volume of the noise picked up by the L microphone 201a, as in the case where the posture of the image pickup apparatus 100 changes. Is shown. Therefore, in the present embodiment, in order to effectively reduce noise, the image pickup apparatus 100 corrects the noise parameter in a range in which the amplitude of the frequency spectrum changes according to the change in the zoom magnification of the optical lens 300. ..

FIG. 18 shows a correction noise parameter for each zoom magnification of the optical lens 300. In this embodiment, the amplitudes of the frequency spectra of Lch_Before, Rch_Before, and Nch_Before will be described as varying in the range of N = 200-250. On the other hand, in this embodiment, it is assumed that the amplitudes of the frequency spectra of Lch_Before, Rch_Before, and Nch_Before are sufficiently small changes in the other N ranges so that the noise parameters do not need to be corrected. As shown in FIG. 18, only the frequency spectrum that needs to be corrected is recorded in the noise parameter recording unit 205 as a noise parameter for correction. For example, when the zoom magnification is 2.3 times, the noise parameter selection unit 206 instructs the noise data generation unit 204 to use the correction noise parameter having a zoom magnification of 2 to 3 times together with the noise parameter to be used. .. The noise parameter selection unit 206 determines that correction of the noise parameter is not necessary when the zoom magnification is 1x, and may not instruct the noise data generation unit 204 to instruct the noise parameter for correction.

From now on, the correction processing of the noise parameter of the image pickup apparatus 100 during the moving image recording will be described with reference to the flowchart of FIG. The processing of the image pickup apparatus 100 is started, for example, when the release switch 61 is pressed by the user.

In step S1901, the noise parameter selection unit 206 determines to use the noise parameter for the optical lens 300 mounted on the image pickup apparatus 100.

In step S1902, the noise parameter selection unit 206 determines whether or not noise has been detected. If it is determined that the noise parameter selection unit 206 has detected noise, the process of step S1903 is executed. If it is determined that the noise parameter selection unit 206 has not detected noise, the process of step S1909 is executed.

In step S1903, the noise parameter selection unit 206 acquires the posture information of the image pickup apparatus 100 from the information acquisition unit 103.

In step S1904, the noise parameter selection unit 206 acquires the zoom magnification of the optical lens 300 from the lens control unit 102.

In step S1905, the noise parameter selection unit 206 determines the correction noise parameter based on the attitude information acquired in step S1903 and the zoom magnification acquired in step S1904.

In step S1906, the noise parameter selection unit 206 instructs the noise data generation unit 204 of the correction noise parameter to be used for the noise reduction processing. The noise data generation unit 204 generates noise data by using the noise parameter for the optical lens 300 and the correction parameter.

In step S1907, the voice input unit 104 records the noise-reduced voice data in the volatile memory 105. The control unit 111 uses this audio data to record moving image data with audio on the recording medium 110.

In step S1908, the control unit 111 determines whether or not to end the moving image recording. For example, the control unit 111 determines that the moving image recording is finished in response to the pressing of the release switch 61. On the contrary, the control unit 111 determines that the moving image recording is not finished when the release switch 61 is not pressed. When the control unit 111 determines that the moving image recording is finished, the process of step S1909 is executed. When the control unit 111 determines that the moving image recording is not started, the processing of this flowchart ends.

The correction processing of the noise parameter of the image pickup apparatus 100 has been described above.

In this way, by correcting the noise parameter during video recording, effective noise reduction processing can be performed.

In this embodiment, as an example, it is assumed that the amplitudes of the frequency spectra of Lch_Before [200] to Lch_Before [250] and Nch_Before [200] to Nch_Before [250] change according to the posture of the image pickup apparatus 100. However, since the position of the frequency spectrum that changes according to the posture of the image pickup apparatus 100 depends on the shape and structure of the lens and the like, it differs depending on the type of the optical lens 300. Similarly, since the position of the frequency spectrum that changes according to the zoom magnification of the optical lens 300 depends on the shape and structure of the lens, the position of the frequency spectrum differs depending on the type of the optical lens 300. That is, the correction noise parameter is a noise parameter at a position other than the frequency spectrum shown in this embodiment, depending on the type of the optical lens 300.

In addition, the image pickup device 100 may change the noise parameter depending on whether it is held by the user's hand or fixed to a tripod or the like. This is a correction because the frequency spectrum of the noise input to the microphone differs depending on the transfer function or the like between the state of being held by the user's hand and the state of being fixed.

In addition to this, the configuration may be such that the state of the device that affects the frequency spectrum of the noise picked up by each microphone is detected and the noise parameter is corrected according to the detection result. Specifically, it detects a state related to the voice processing device that affects the voice transmission path and transmission function from the noise source to each microphone, and corrects the noise parameter according to the detected state.

[Third Example]
Here, FIG. 20 is a block diagram showing a configuration example of the voice input unit 104 in the second embodiment. The parts different from the configuration of the voice input unit 104 shown in FIG. 3 are the subtraction processing unit 207 and the iFFT unit 208. Here, the description of the same processing unit as in FIG. 3 will be omitted.

The iFFT unit 208a performs inverse fast Fourier transforms on Lch_Before and Rch_Before input from the FFT unit 203, respectively, and converts the digital audio signal in the frequency domain into the digital audio signal in the time domain, respectively. Further, the iFFT unit 208b performs inverse fast Fourier transforms of NL and NR, respectively, and converts the digital audio signal in the frequency domain into the digital audio signal in the time domain.

The subtraction processing unit 207 subtracts the digital audio signal input from the iFFT unit 208b from the digital audio signal input from the iFFT unit 208a. The arithmetic processing in the subtraction processing unit 207 is a waveform subtraction method for subtracting a digital audio signal in the time domain.

The waveform subtraction method in the third embodiment can be applied instead of the spectral subtraction method in the first embodiment or the second embodiment.

When performing waveform subtraction, the image pickup apparatus 100 may also record a parameter related to the phase of the digital audio signal as a noise parameter.

[Other Examples]
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or recording medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The present invention is not limited to the above embodiment as it is, and at the implementation stage, the components can be modified and embodied within a range that does not deviate from the gist thereof. In addition, various inventions can be formed by an appropriate combination of the plurality of components disclosed in the above examples. For example, some components may be removed from all the components shown in the examples. In addition, components from different embodiments may be combined as appropriate.

Claims (20)

A mounting means on which a lens having a noise source is mounted, and
With the first microphone to get the ambient sound,
A second microphone for acquiring sound from the noise source,
The first conversion means for generating the first audio signal by Fourier transforming the audio signal from the first microphone,
A second conversion means that Fourier transforms the audio signal from the second microphone to generate a second audio signal,
A generation means for generating noise data using the second audio signal and parameters related to noise of the noise source, and
A subtraction means for subtracting the noise data from the first audio signal,
A speech processing device comprising a third transforming means for inverse Fourier transforming a speech signal from the subtracting means.
The voice processing device is characterized in that the generation means uses the parameter according to the type of the lens mounted on the mounting means as the parameter. The voice processing apparatus according to claim 1, wherein the generation means determines the parameters used for generating the noise data based on information about the type of the lens mounted on the mounting means. A detection means for detecting that the lens has been replaced, and
In response to the detection that the lens has been replaced by the detection means, the determination according to the type of the lens mounted on the mounting means by the replacement is determined as the parameter used by the generation means. The voice processing apparatus according to claim 1 or 2, further comprising means. The determination means determines from the plurality of parameters recorded in the recording means that the parameters corresponding to the type of the lens mounted on the mounting means are used as parameters for generating the noise data. The voice processing device according to claim 3. When the parameter corresponding to the lens mounted on the mounting means is not recorded on the recording means, the determining means determines the parameter having a default value as a parameter used to generate the noise data. The voice processing apparatus according to claim 3 or 4. The voice processing device according to any one of claims 1 to 5, wherein the noise source is a motor, and the type of the motor included in the lens differs depending on the type of the lens. The voice processing device according to any one of claims 1 to 6, wherein the type of the lens is a model number of the lens. It ’s a voice processing device,
With the first microphone to get the ambient sound,
With a second microphone to get the sound from the noise source,
A detection means for detecting a state related to the voice processing device, and
The first conversion means for generating the first audio signal by Fourier transforming the audio signal from the first microphone,
A second conversion means that Fourier transforms the audio signal from the second microphone to generate a second audio signal,
A generation means for generating noise data using the second audio signal and parameters related to noise of the noise source, and
A subtraction means for subtracting the noise data from the first audio signal,
It has a third conversion means for inverse Fourier transforming the voice signal from the subtraction means, and the generation means corrects the parameter according to the state of the voice processing device detected by the detection means. Characterized voice processing device. The state relating to the voice processing device is the posture of the voice processing device.
The voice processing device according to claim 8, wherein the detection means is at least one of an acceleration sensor and a gyro sensor. Further having a mounting means for mounting the lens having the noise source,
The detection means detects information related to the length of the lens barrel of the lens mounted on the mounting means as information on the voice processing device.
The voice processing apparatus according to claim 8 or 9, wherein the generation means corrects the parameters according to the information related to the length of the lens barrel of the lens detected by the detection means. The audio processing device according to claim 10, wherein the information relating to the length of the lens barrel of the lens is the zoom magnification of the lens. The voice processing apparatus according to any one of claims 8 to 11, wherein the generation means corrects the parameter in at least a part of the frequency spectrum. The generation means detects whether or not noise is generated from the noise source, and determines whether or not noise is generated.
When it is detected that noise is generated from the noise source, the generation means corrects the parameter and corrects the parameter.
The voice processing apparatus according to any one of claims 8 to 12, wherein when it is not detected that noise is generated from the noise source, the generation means does not correct the parameter. The generation means includes at least one of the plurality of the parameters and the second voice, including a first parameter corresponding to the first kind of noise and a second parameter corresponding to the second kind of noise. The voice processing apparatus according to any one of claims 1 to 13, wherein the noise data is generated by using a signal. The voice processing apparatus according to claim 14, further comprising a recording means for recording information of the plurality of parameters. The first microphone is composed of a plurality of microphones.
The voice processing device according to claim 15, wherein the recording means records the parameters for each microphone constituting the first microphone. The generation means is characterized in that the noise data is generated by using the parameter corresponding to the type of noise included in the second voice signal among the plurality of parameters and the second voice signal. The voice processing apparatus according to any one of claims 14 to 16. A mounting means on which a lens having a noise source is mounted, and
With the first microphone to get the ambient sound,
A method of controlling a voice processing device having a second microphone for acquiring sound from the noise source.
The step of Fourier transforming the audio signal from the first microphone to generate the first audio signal,
The step of Fourier transforming the audio signal from the second microphone to generate the second audio signal,
A generation step of generating noise data using the second audio signal and parameters related to noise of the noise source.
A subtraction step of subtracting the noise data from the first audio signal,
It has a step of inverse Fourier transforming the audio signal generated in the subtraction step.
A control method characterized in that, in the generation step, the parameter corresponding to the type of the lens mounted on the mounting means is used as the parameter. With the first microphone to get the ambient sound,
A method of controlling a voice processing device having a second microphone for acquiring sound from a noise source.
A detection step for detecting a state related to the voice processing device, and
The step of Fourier transforming the audio signal from the first microphone to generate the first audio signal,
The step of Fourier transforming the audio signal from the second microphone to generate the second audio signal,
A generation step of generating noise data using the second audio signal and parameters related to noise of the noise source.
A subtraction step of subtracting the noise data from the first audio signal,
It has a third transformation step of inverse Fourier transforming the audio signal generated in the subtraction step.
The control method, characterized in that, in the generation step, the parameter is corrected according to the state of the voice processing device detected by the detection step. A computer-readable program for operating a computer as each means of the voice processing device according to any one of claims 1 to 17.

Images