JP6497878B2 - Electronic device and control method - Google Patents

Description

  The present invention relates to a technique for reducing noise.

  Imaging apparatuses represented by recent digital cameras have not only still images but also moving image recording functions with sound. That is, a moving image obtained by continuously capturing an image of a subject along a time axis and surrounding audio data can be recorded together in a storage medium such as a memory card. Hereinafter, the sound to be recorded, such as the sound around the subject, is referred to as “environmental sound”.

  In addition, the imaging apparatus can focus or zoom the subject during imaging by moving the optical lens. However, driving sound is generated during driving for moving the optical lens. In recent digital cameras, the housing has been downsized, and the distance between the drive sound source and the microphone has been shortened. Therefore, the microphone in the digital camera acquires driving sound, and as a result, there is a high possibility that the driving sound is superimposed as noise on the environmental sound.

  Conventionally, a technique called “spectral subtraction method” is known in order to reduce the noise as described above (for example, Patent Document 1). This spectral subtraction method will be briefly described with reference to FIG. This figure is a part of a block configuration of a digital camera. This apparatus includes a control unit 2109 that controls the entire apparatus, an operation unit 2110 that receives instructions from a user, an optical lens, a lens control unit, and the like. Further, this apparatus includes an imaging unit 2101 that captures images to obtain image data, a microphone 2205, an audio input unit 2102 that acquires audio as audio data, and a memory 2103 that stores image data and audio data. Note that, normally, image data and audio data stored in the memory 2103 are encoded and stored in the storage medium as encoded data.

  When the control unit 2109 detects an instruction such as zoom-in or zoom-out by the user via the operation unit 2110 during the recording of the moving image with sound, the control unit 2109 changes the position of the optical lens to change the position of the optical lens. 101 is controlled. In accordance with this, the imaging unit 2101 drives a drive source such as a motor in order to change the position of the optical lens. At this time, the microphone 2205 picks up the driving sound of the optical lens, and as a result, the acoustic data obtained from the microphone 2205 is data in which the environmental sound and the driving sound (noise) are synthesized. The illustrated voice input unit 2102 has a function of reducing the drive sound.

  The sound detected by the microphone 2205 is converted into 16-bit digital data (hereinafter referred to as acoustic data) by an ADC (Analog / Digital Converter) 2206 at a sampling rate of 48 kHz, for example. The FFT 2207 converts the acoustic data arranged in time series (for example, 1024 pieces of acoustic data) into FFT data (amplitude spectrum) by performing FFT (fast Fourier transform) processing. The noise reduction unit 2200 performs noise reduction processing by subtracting each frequency data of noise from each frequency data. For this reason, the noise reduction unit 2200 includes a profile storage unit 2210 that stores amplitude data (noise profile) for each frequency of noise in advance, and an amplitude spectrum subtraction unit 2211. The amplitude spectrum subtraction unit 2211 subtracts the amplitude data of each frequency of noise recorded in the profile storage unit 2210 from the amplitude spectrum. Thereafter, the amplitude spectrum from which the noise has been subtracted is subjected to inverse FFT processing by IFFT 2214 and returned to the original time-series acoustic data. Thereafter, the voice processing unit 2216 performs various processes on the acoustic data. An ALC (auto level controller) 2217 adjusts the level of the acoustic data, and the result is stored in the memory 2103.

  The above is the outline of the “spectral subtraction method”. As described above, it is desirable that the noise profile stored in advance in the profile storage unit 2210 represents the driving sound actually generated by the imaging unit 2101.

JP 2006-279185 A

When the technique disclosed in Patent Document 1 is applied to an imaging apparatus, an error occurs between the driving sound actually generated in the imaging apparatus and the driving sound indicated by the noise profile stored in advance in the following situation. .
・ Individual differences in sound noise generation of motors, gears, etc. ・ Imaging device installation status Differences in audio noise ・ Differences in audio noise due to zoom position ・ Wear and aging of parts ・ Temperature conditions during operation ・ Image sensor attitude -Replacement of parts of the drive unit or the recording unit after being shipped to the market, such as in response to a failure For this reason, there is a problem that it is difficult to reduce noise by one noise profile stored in advance. In addition, when recording left and right channel sounds such as stereo sound, there is a problem in that the influence of the drive sound on the microphone corresponding to each channel changes due to changes in environmental sound and drive sound.

  In view of the above-described problems, the present invention reduces noise with high accuracy with respect to the sound of each channel even when the environmental sound and driving sound change when recording the sound of the left and right channels. The purpose is to.

The electronic device according to the present invention is
A first microphone,
A second microphone,
Input means for inputting a drive instruction for driving the drive means ;
First conversion means for acquiring sound spectrum data by performing Fourier transform on sound data obtained from the first microphone;
Second conversion means for acquiring sound spectrum data by performing Fourier transform on the sound data obtained from the second microphone;
A first average value is an average value of the voice spectrum data obtained by converting by said first of said first converting means the audio data obtained from the microphone before the driving instruction is input The voice data obtained from the first microphone is converted by the first conversion means after a predetermined stabilization period until the voice spectrum data is stabilized after the drive instruction is input. a first generation means for generating a first noise spectrum data representing the driving noise of the difference in basis, the driving means and the second average value is an average value of the voice spectrum data obtained by,
A third average value is an average value of the voice spectrum data obtained by converting by said second of said second converting means the audio data obtained from the microphone before the driving instruction is input The voice data obtained from the second microphone is converted by the second conversion means between the input of the driving instruction and the elapse of a predetermined stabilization period until the voice spectrum data is stabilized. a second generation means for generating a second noise spectrum data representing the driving noise of the difference in basis, the driving means and the fourth average value is an average value of the voice spectrum data obtained by,
The voice spectrum data obtained from the first conversion means and the second voice spectrum data obtained from the first conversion means and the sum of the voice spectrum data obtained from the second conversion means. When the value indicating the ratio of the difference from the audio spectrum data obtained from the conversion means is equal to or less than a predetermined value, control is performed so that the first noise spectrum data is used to reduce the driving noise, and the value is If the not less than a predetermined value, characterized by chromatic and control means for controlling so as not using the first noise spectrum data in order to reduce the driving noise.

  According to the present invention, when recording the sound of the left and right channels, noise can be reduced with high accuracy with respect to the sound of each channel even if the environmental sound or the driving sound changes.

1 is a block diagram illustrating an example of an imaging apparatus according to an embodiment. The block diagram which shows an example of a structure of the imaging part and audio | voice input part of embodiment. The flowchart which shows an example of the moving image recording process in embodiment. The figure showing an example of the amplitude spectrum for every frequency before the zoom operation of the embodiment and during the zoom operation. An example of the timing chart which shows the noise profile creation process of embodiment. An example of the timing chart which shows the noise profile creation process of embodiment. The flowchart which shows an example of the noise profile creation process of embodiment. An example of the timing chart which shows the expansion correction process of the noise profile of embodiment. An example of the timing chart which shows the reduction correction process of the noise profile of embodiment. The flowchart which shows an example of the noise profile correction process of embodiment. It is a figure which shows an example of the setting of the time constant regarding the noise profile correction process in embodiment. The figure which shows an example of the relationship between the external sound source and audio | voice input part of embodiment. 6 is a timing chart showing an example of noise profile correction processing for Rch and Lch in the embodiment. 6 is a flowchart illustrating an example of noise profile correction processing for Rch and Lch according to the embodiment. The timing chart which shows an example of the noise reduction process in embodiment. The flowchart which shows an example of the noise reduction process in embodiment. The figure which shows an example of the relationship between the coefficient (alpha) and environmental sound in embodiment. The timing chart which shows an example of the post-process of embodiment. The flowchart which shows an example of the post-process in embodiment. The block which shows an example of the audio | voice input part in embodiment. FIG. 10 is a block diagram illustrating an example of a conventional imaging device.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the embodiment, the imaging apparatus 100 such as a digital camera is taken as an example of the electronic device and will be described below. However, the electronic device is not limited to the imaging device 100, and may be a mobile phone or an IC recorder as long as the device has a microphone.

  FIG. 1 is a block diagram illustrating an example of the configuration of the imaging apparatus 100. The imaging apparatus 100 includes an imaging unit 101, an audio input unit 102, a memory 103, a display control unit 104, a display unit 105, an encoding processing unit 106, a recording / playback unit 107, a recording medium 108, and a control unit 109. Furthermore, the imaging apparatus 100 includes an operation unit 110, an audio output unit 111, a speaker 112, an external output unit 113, and a system bus 114 that connects each component.

  The imaging unit 101 converts an optical image of a subject into an image signal, performs image processing on the image signal, and generates image data. The sound input unit 102 collects sound around the imaging apparatus 100, performs sound processing on the sound, and generates sound data.

  The memory 103 stores image data supplied from the imaging unit 101 and audio data supplied from the audio input unit 102. The display control unit 104 causes the display unit 105 to display the image data obtained from the imaging unit 101, the menu screen of the imaging device 100, and the like. The encoding processing unit 106 performs predetermined encoding on the image data stored in the memory 103 to generate compressed image data. In addition, the encoding processing unit 106 performs predetermined encoding on the audio data stored in the memory 103 to generate compressed audio data. The recording / reproducing unit 107 records at least one of the compressed image data, the compressed audio data, and the compressed moving image data generated by the encoding processing unit 106 on the recording medium 108. The recording / reproducing unit 107 reads at least one of image data, audio data, and moving image data recorded on the recording medium 108 from the recording medium 108.

  The control unit 109 controls each unit of the imaging apparatus 100 via the system bus 114. The control unit 109 has a CPU and a memory. A program for controlling each unit of the imaging apparatus 100 is recorded in the memory of the control unit 109.

  The operation unit 110 receives an operation for inputting an instruction from the user to the imaging apparatus 100. The operation unit 110 transmits a signal corresponding to a specific operation performed by the user to the control unit 109. The operation unit 110 includes a button for instructing photographing of a still image, a recording button for instructing start and stop of moving image recording, a zoom button for instructing the imaging apparatus 100 to optically perform a zoom operation on the image, and the like. Have. Furthermore, the operation unit 110 includes a mode selection button for selecting an operation mode of the imaging apparatus 100 from among a still image shooting mode, a moving image shooting mode, and a playback mode.

  The audio output unit 111 outputs the audio data read by the recording / reproducing unit 107 to the speaker 112. The external output unit 113 outputs the audio data read by the recording / playback unit 107 to an external device.

  Next, an operation when the imaging apparatus 100 is in the moving image shooting mode will be described. When the image capturing apparatus 100 is in the moving image capturing mode, the control unit 109 controls the image capturing unit 101 to capture images at a predetermined frame rate in response to the recording button of the operation unit 110 being turned on, and the audio data The voice input unit 102 is controlled so as to acquire. In this case, the image data and audio data captured by the imaging unit 101 are compressed and recorded as moving image data on the recording medium 108 by the recording / reproducing unit 107. Thereafter, the control unit 109 closes the moving image data recorded on the recording medium 108 in response to the recording button of the operation unit 110 being turned off, and generates one moving image file. When the imaging apparatus 100 is in the moving image shooting mode, it is assumed that the recording button of the operation unit 110 is OFF until the recording button of the operation unit 110 is turned ON by the user.

FIG. 20 shows the relationship between the imaging unit 101 and the audio input unit 102.
The imaging unit 101 includes an optical lens 201, an imaging element 202, a lens control unit 203, and an image processing unit 204.
The optical lens 201 is a focus lens or a zoom lens for optically focusing on a subject. The optical lens 201 can perform zooming optically. Hereinafter, the optical zooming using the optical lens 201 is referred to as “zoom operation”. In the zoom operation, the lens control unit 203 moves the optical lens 201 in response to an instruction from the control unit 109 to zoom the optical image of the subject. The image sensor 202 converts an optical image of a subject into an image signal and outputs the image signal. The lens control unit 203 drives a motor or the like for moving the optical lens 201. The image processing unit 204 performs image processing on the image signal output from the image sensor 202 to generate image data.

  For example, when an instruction for causing the imaging apparatus 100 to start zoom operation, focus adjustment, or the like is input via the operation unit 110, the control unit 109 controls the lens control unit 203 to move the optical lens 201. The zoom control signal for turning on is changed to ON. When the zoom control signal is changed to ON, the lens control unit 203 drives a motor or the like to move the optical lens 201.

  When the optical lens 201 is moved, in the imaging apparatus 100, noise accompanying movement of the optical lens 201 and noise accompanying driving of a motor for moving the optical lens 201 are generated. Hereinafter, the noise accompanying the movement of the optical lens 201 and the noise accompanying the driving of the motor for moving the optical lens 201 are referred to as “driving noise”.

  In FIG. 20, the imaging apparatus 100 has been described as including the optical lens 201 and the lens control unit 203, but is not limited thereto. The optical lens 201 and the lens control unit 203 may be detachable from the imaging apparatus 100.

  The audio input unit 102 of the imaging apparatus 100 includes an R (Right) channel audio input unit 102a and an L (Left) channel audio input unit 102b in order to realize stereo recording. Since the R channel audio input unit 102a and the L channel audio input unit 102b have the same configuration, the configuration of the R channel audio input unit 102a will be described below. The R channel audio input unit 102a includes a microphone 205a, an ADC 206a, an FFT 207a, a noise reduction unit 200a, an IFFT 214a, a noise application unit 215a, an audio processing unit 216a, and an ALC 217a. The R channel is hereinafter referred to as “Rch”, and the L channel is hereinafter referred to as “Lch”.

  The microphone 205a converts audio vibration into an electrical signal and outputs an analog audio signal. The ADC (analog / digital converter) 206a converts the analog audio signal obtained by the microphone 205a into a digital audio signal. For example, the sampling frequency of the ADC 206a is 48 KHz, and 16-bit time-series digital data is output per sample. An FFT (Fast Fourier Transformer) 207a inputs, for example, 1024 time-series audio data output from the ADC 206a as one frame. Then, the FFT 207a performs fast Fourier transform on the audio data for one frame, generates an amplitude level (amplitude spectrum data) for each frequency, and supplies it to the noise reduction unit 200a. Note that the amplitude spectrum generated by the FFT 207a is composed of amplitude data for each frequency of 1024 points from 0 to 48 KHz. In the embodiment, the number of audio data of one frame is 1024, but the first half of 512 data in one frame to be processed next and the latter half of 512 data in the immediately preceding frame are the same, and are identical to each other. The parts are overlapping.

  The noise reduction unit 200a subtracts the amplitude data of the noise for each frequency representing the driving noise generated when the imaging apparatus 100 performs the zoom operation from the amplitude data of the corresponding frequency output from the FFT 207a. The noise reduction unit 200a supplies the amplitude spectrum data after the subtraction to an IFFT (Inverse Fast Fourier Transform) 214a.

  The IFFT (Inverse Fast Fourier Transform) 214a uses the phase information supplied from the FFT 207a to perform an inverse fast Fourier transform (inverse transform) on the amplitude spectrum supplied from the noise reduction unit 200a, thereby Generate audio data in time series format. The IFFT 214a uses the phase information before being subjected to the fast Fourier transform in the FFT 207a to return it to a time-series audio signal.

  The noise applying unit 215a applies a noise signal to the time-series audio signal supplied from the IFFT 214a. It is assumed that the noise signal applied by the noise applying unit 215a is a noise floor level signal. The sound processing unit 216a performs processing for reducing wind noise, processing for enhancing a stereo feeling, equalizer processing, and the like. The ALC (auto gain controller) 217 a adjusts the amplitude of the time-series audio signal to a predetermined level and outputs the adjusted audio data to the memory 103.

  Next, the noise reduction unit 200a of the R channel audio input unit 102a in the embodiment will be described below with reference to FIG.

  FIG. 2 is a block diagram illustrating an example of the configuration of the noise reduction unit 200a. The noise reduction unit 200a includes an integration circuit 250a, a memory 251a, a profile creation unit 252a, a profile storage unit 253a, an amplitude spectrum subtraction unit 254a, a post-processing unit 255a, and a profile correction unit 256a.

The noise reduction unit 200a performs an operation for reducing drive noise that occurs when the imaging apparatus 100 performs a zoom operation. Hereinafter, the operation performed by the noise reduction unit 200a will be described with reference to FIG.
FIG. 4 is a diagram illustrating an example of an amplitude spectrum for each frequency before the zoom operation is performed by the imaging apparatus 100 and during the zoom operation is performed by the imaging apparatus 100. The horizontal axis in FIG. 4 indicates the frequency and indicates 1024 points in the interval from 0 to 48 Khz (provided that the Nyquist frequency of up to 24 kHz has a 512-point frequency spectrum). Reference numeral 401 in FIG. 4 indicates an amplitude spectrum indicating environmental sound before the imaging apparatus 100 performs a zoom operation (in a state where the optical lens 201 is not moved). Reference numeral 402 in FIG. 4 indicates an amplitude spectrum indicating environmental sound when the imaging apparatus 100 is performing a zoom operation (a state in which the optical lens 201 is moving). The amplitude spectrum indicated by 402 includes drive noise. The noise reduction unit 200a creates a noise profile that is used to reduce drive noise from the difference between the amplitude spectrum indicated by 401 and the amplitude spectrum indicated by 402. Hereinafter, each part of the noise reduction unit 200a will be described.

In response to an instruction from the control unit 109, the integration circuit 250a integrates the amplitude value for each frequency of the amplitude spectrum subjected to the fast Fourier transform by the FFT 207a on the time axis. At this time, the integration circuit 250a counts the number of integrated frames. The amplitude value of the frequency fi (where i = 0, 1,..., 1023) in the amplitude spectrum data obtained from one frame from the FFT 207a is represented as A (fi). In this case, the integration circuit 250a obtains an integral value (cumulative addition value) S (fi) for each frequency as in the following equation.
S (fi) = ΣA (fi)

When the lens control unit 203 has not moved the optical lens 201, the integration circuit 250a integrates the amplitude value of each frequency as described above. Then, the integration circuit 250a outputs a result obtained by dividing the integration value of each frequency by the number of frames n representing the integration period. That is, the integration circuit 250a calculates (calculates) the average amplitude value Aave (fi) for each frequency as in the following equation, and outputs the calculation result.
Aave (fi) = S (fi) / n
The data indicated by the average amplitude value Aave (fi) (i = 0, 1,..., 1023) corresponds to the amplitude spectrum indicated by 401 in FIG. The integrating circuit 250a stores the calculated average amplitude value Aave (fi) in the memory 251a.

  The integration circuit 250a integrates the amplitude value of each frequency as described above from when the lens control unit 203 starts moving the optical lens 201 until the stabilization period elapses. The stabilization period is a period until the amplitude spectrum input to the integration circuit 250a is stabilized by the time constant of the integration circuit 250a. Until the stabilization period elapses, the amplitude spectrum output from the FFT 207a includes drive noise. When a stabilization period (for example, corresponding to m frames) has elapsed, the integration circuit 250a outputs S (fi) / m to the profile creation unit 252a. S (fi) / m corresponds to the amplitude spectrum indicated by 402 in FIG.

The profile creation unit 252a subtracts S (fi) / n stored in the memory 251a from S (fi) / m supplied from the integration circuit 250a, thereby obtaining an amplitude value corresponding to driving noise for each frequency. A certain N (fi) is calculated as follows.
N (fi) = S (fi) / m-S (fi) / n

  After N (fi) is calculated, the profile creation unit 252a stores N (fi) as a noise profile in the profile storage unit 253a. The noise profile is data indicating drive noise that occurs when a zoom operation is performed.

After that, the amplitude spectrum subtraction unit 254a performs a process of subtracting the drive noise amplitude value N (fi) read from the profile storage unit 253a from the amplitude spectrum A (fi) supplied from the FFT 207a. Hereinafter, the process of subtracting the amplitude value N (fi), which is the noise profile read from the profile storage unit 253a, from A (fi) of the amplitude spectrum supplied from the FFT 207a is referred to as “subtraction process”. The amplitude spectrum subtraction unit 254a outputs the amplitude spectrum A _NR (fi) obtained by the following equation to the IFFT 214a or the IFFT 214b.
A _NR (fi) = A (fi) −N (fi)

  It should be noted that a state in which the creation of the noise profile is not completed by the profile creation unit 252a occurs after the stabilization period elapses after the user receives an instruction to start the zoom operation to the imaging apparatus 100. Thereby, in order to shorten the period until the creation of the noise profile by the profile creation unit 252a is completed, it is necessary to reduce “m”. However, when “m” is extremely small, there is a possibility that the drive noise reduction accuracy by the noise profile is lowered. When the lens control unit 203 starts control for moving the optical lens 201, a movement sound, sound vibration, and the like of the optical lens 201, which are a kind of driving noise, are generated for about 70 ms. In order to reduce the starting sound and vibration of the optical lens 201, “m” is set to, for example, “15” in order to cause the profile creation unit 252a to create a noise profile in a period exceeding 70 ms.

In the embodiment, although one frame is 1024 time-series audio data, half of each frame is superimposed on each other. Since the sampling rate of the audio data is 48 kHz, if m = 15, the noise profile creation period T is
T = m frame period = m × (1024/2) / 48 kHz = 160 ms
It becomes. The profile creation unit 252a creates a noise profile during the period from when the user receives an instruction to start the zoom operation to the imaging apparatus 100 until the creation period T elapses. For this reason, the profile creation unit 252a can create a highly accurate noise profile for reducing the movement sound and sound fluctuation of the optical lens 201.

  The post-processing unit 255a corrects the amplitude spectrum after the subtraction process is performed by the amplitude spectrum subtracting unit 254a and outputs the corrected amplitude spectrum to the IFFT 214a.

  The profile correction unit 256a performs a process of correcting the noise profile stored in the profile storage unit 253a according to the magnitude of the environmental sound. As correction of the noise profile performed by the profile correction unit 256a, there are enlargement correction (increase correction) and reduction correction (decrease correction). The profile correction unit 256a includes a profile enlargement unit 271a that performs noise profile enlargement correction and a profile reduction unit 272a that performs noise profile reduction correction.

The noise profile enlargement correction is correction for increasing the amplitude spectrum of the noise profile created by the profile creation unit 252a or the noise profile corrected by the profile correction unit 256a. That is, the amplitude spectrum A _NR (fi) after the subtraction process is performed by the amplitude spectrum subtracting unit 254a is reduced by performing the correction of the noise profile. The noise profile reduction correction is correction for reducing the amplitude profile of the noise profile created by the profile creation unit 252a or the noise profile corrected by the profile correction unit 256a. That is, the amplitude spectrum A _NR (fi) after the subtraction process is performed by the amplitude spectrum subtracting unit 211 is increased by performing the noise profile reduction correction. The noise profile correction performed by the profile correction unit 256a is performed on the amplitude spectrum A (fi) for each frame supplied from the FFT 207a as necessary. When the zoom operation is performed by the imaging apparatus 100, the profile correction unit 256a can appropriately correct the noise profile in accordance with fluctuations in environmental sound and driving noise.

  Similarly to the R channel audio input unit 102a, the L channel audio input unit 102b includes a microphone 205b, an ADC 206b, an FFT 207b, a noise reduction unit 200b, an IFFT 214b, a noise application unit 215b, an audio processing unit 216b, and an ALC 217b. The microphone 205a and the microphone 205b have the same configuration, the FFT 207a and the FFT 207b have the same configuration, and the noise reduction unit 200a and the noise reduction unit 200b have the same configuration. Furthermore, IFFT 214a and IFFT 214b have the same configuration, and noise application unit 215a and noise application unit 215b have the same configuration. Furthermore, the audio processing unit 216a and the audio processing unit 216b have the same configuration, and the ALC 217a and the ALC 217b have the same configuration.

  FIG. 3 is a flowchart illustrating an example of a moving image recording process performed by the control unit 109 when the moving image shooting mode is selected as the mode of the imaging apparatus 100. The moving image recording process will be described below by taking an example in which an analog audio signal is output from the microphone 205a to the ADC 206a.

  When the imaging apparatus 100 is changed to the moving image shooting mode, the control unit 109 clears the profile storage unit 253a in the noise reduction unit 200a to zero (S301). Thereafter, the control unit 109 causes the integrating circuit 250a to start integrating the amplitude spectrum input from the FFT 207a (S302). Then, the control unit 109 determines whether or not the recording button of the operation unit 110 is turned on, that is, whether or not an instruction for starting the recording of moving image data to the imaging device 100 is input (S303). When an instruction to start recording moving image data is input to the imaging apparatus 100 (Yes in S303), the control unit 109 starts recording moving image data (S304). In this case, the control unit 109 starts encoding processing of image data and audio data for generating moving image data stored in the memory 103 from the imaging unit 101 and the audio input unit 102, and stores the recording medium in the recording / playback unit 107. Recording to 108 is started.

  In step S <b> 305, the control unit 109 determines whether an instruction to start a zoom operation is input via the operation unit 110. When an instruction to start a zoom operation is not input via the operation unit 110, the control unit 109 determines whether an instruction to end recording of moving image data is input to the imaging apparatus 100 via the operation unit 110. (S306). When an instruction to end recording of moving image data is input to the imaging apparatus 100 via the operation unit 110 (Yes in S306), the control unit 109 starts encoding of moving image data stored in the memory 103, Recording on the recording medium 108 is performed. Further, the control unit 109 performs a closing process on the moving image data stored in the recording medium 108 and completes it as a moving image file (S312). When an instruction to end recording of moving image data is not input to the imaging apparatus 100 via the operation unit 110 (No in S306), the process returns from S306 to S302.

  On the other hand, when an instruction to start the zoom operation is input from the operation unit 110, the process proceeds from S305 to S307. In step S307, the control unit 109 performs noise profile creation processing to cause the noise reduction unit 200a to create a noise profile. The noise profile creation process performed in S307 will be described later. The noise profile created by executing the noise profile creation process is stored in the profile storage unit 253a.

  Next, the control unit 109 performs a noise reduction process of subtracting the amplitude value of a specific frequency included in the noise profile from the amplitude value for each frequency of the amplitude spectrum for each frequency that has been fast Fourier transformed by the FFT 207a (S308). ). When the noise reduction process is performed, the control unit 109 controls the amplitude spectrum subtraction unit 254a to perform the subtraction process. Next, the control unit 109 performs a noise profile correction process for controlling the profile correction unit 256a so as to correct the noise profile stored in the profile storage unit 210a (S309). The noise profile correction process performed in S309 will be described later. The noise profile corrected by the profile correction unit 256a is applied in the subtraction process for the next frame. Next, the control unit 109 outputs the Rch amplitude spectrum obtained after the subtraction process is performed by the amplitude spectrum subtraction unit 254a and the Lch amplitude spectrum obtained after the subtraction process is performed by the amplitude spectrum subtraction unit 254b. If so, post-processing is performed (S310). The post-processing is a process of correcting the Rch amplitude spectrum and the Lch amplitude spectrum to be the same. The post-processing performed in S310 will be described later.

  Then, it is determined whether or not an instruction to stop the zoom operation is input from the operation unit 110 (S311). When an instruction to stop the zoom operation is not input from the operation unit 110, the zoom operation is continuously executed in the imaging apparatus 100, and thus the control unit 109 repeats the processing from S308 to S310. In addition, when an instruction to stop the zoom operation is input from the operation unit 110, the control unit 109 stops the zoom operation in the imaging apparatus 100 and returns to the process of S301.

  Note that the moving image recording process in FIG. 3 has been described by taking an example in which an analog audio signal is output from the microphone 205a to the ADC 206a. However, even when an analog audio signal is output from the microphone 205b to the ADC 206b, a moving image is recorded in the same manner as the moving image recording process of FIG.

[Noise Profile Creation Processing (S307)]
The noise profile creation process executed by the control unit 109 in S307 will be described with reference to FIGS. The case where an analog audio signal is output from the microphone 205a to the ADC 206a will be described as an example, and the noise profile creation process will be described below.

  FIG. 5 is a timing chart showing a noise profile creation process for the amplitude spectrum for each frequency. FIG. 6 is a timing chart illustrating a noise profile creation process for the amplitude spectrum for each frequency when the environmental sound is large before the zoom operation is started by the imaging apparatus 100.

  Hereinafter, FIGS. 5 and 6 will be described. 5 and 6, the timing at which the control unit 109 controls the lens control unit 203 to move the optical lens 201 is “t1”. The control unit 109 changes the zoom control signal to ON in order to control the lens control unit 203 to move the optical lens 201. If the control unit 109 does not control the lens control unit 203 so as to move the optical lens 201, the zoom control signal is not turned on. In this case, the zoom control signal is turned off. In addition, when the control unit 109 controls the lens control unit 203 to stop the movement of the optical lens 201, the control unit 109 changes the zoom control signal to OFF. When the zoom control signal is changed to ON, the lens control unit 203 starts to move the optical lens 201. When the zoom control signal is changed to OFF, the lens control unit 203 stops the movement of the optical lens 201. Further, in FIGS. 5 and 6, the timing at which the amplitude spectrum subtraction unit 254a starts the subtraction process using the noise profile is “t2”. Further, in FIGS. 5 and 6, the timing at which the control unit 109 turns off the zoom control signal is “t3”. The subtraction process using the noise profile is performed by the amplitude spectrum subtraction unit 254a during the period from the timing t2 to the timing t3.

  5 and 6, the amplitude spectrum of the predetermined frequency fi that has been fast Fourier transformed by the FFT 207a is assumed to be “It”. Further, in FIGS. 5 and 6, the amplitude spectrum indicating the amplitude of the predetermined frequency fi integrated by the integration circuit 250a is “Dt”. 5 and 6, the noise profile corresponding to the predetermined frequency fi created by the profile creation unit 252a is “Pt”, and the amplitude spectrum of the predetermined frequency fi output from the amplitude spectrum subtraction unit 254a is “Ut”. To do. Further, in FIGS. 5 and 6, a time-series digital audio signal having a predetermined frequency fi after the noise signal is applied by the noise applying unit 215a is represented as “Nt”.

  The noise profile Pt has an amplitude spectrum of 512 points up to the Nyquist frequency of 24 kHz. A 512-point amplitude spectrum Dt1 in FIGS. 5 and 6 indicates an amplitude spectrum indicating an environmental sound before the imaging apparatus 100 performs a zoom operation (a state in which the optical lens 201 is not moved), and corresponds to 401 in FIG. . A 512-point amplitude spectrum (Dt2) in FIGS. 5 and 6 indicates an amplitude spectrum indicating an environmental sound when the imaging apparatus 100 is performing a zoom operation (a state in which the optical lens 201 is moving). Corresponding to 402.

  FIG. 7 is a flowchart showing a noise profile creation process to be performed by the control unit 109. Next, the noise profile creation process performed by the control unit 109 will be described with reference to FIG. The case where the profile creating unit 252a creates a noise profile will be described as an example, and the noise profile creating process will be described below. When an instruction to start the zoom operation is input from the operation unit 110 (Yes in S305), the zoom control signal is changed from OFF to ON by the control unit 109 (timing t1). At timing t1, the control unit 109 controls the integration circuit 250a so as to calculate Aave (fi) as described above. When Aave (fi) is calculated by the integrating circuit 250a, the control unit 109 stores Aave (fi) in the memory 251a as the amplitude spectrum Dt1 (S701).

  Next, the control unit 109 integrates the integration circuit for each frequency in the amplitude spectrum when the lens control unit 203 moves the optical lens 201 until the stabilization period elapses from the timing t1. 250a is controlled. Thereafter, the control unit 109 determines whether or not the stabilization period has elapsed (S702). When the stabilization period has elapsed (timing t2) (Yes in S702), the control unit 109 controls the integration circuit 250a to calculate S (fi) / m as described above. When S (fi) / m is calculated by the integration circuit 250a, the control unit 109 stores S (fi) / m as the amplitude spectrum Dt2 in the memory 251a (S703).

  Next, the control unit 109 determines whether or not the amplitude spectrum Dt1 is equal to or less than a predetermined amplitude spectrum Dtth (S704). It is assumed that the predetermined amplitude spectrum Dtth is stored in the memory 103 in advance. The predetermined amplitude spectrum Dtth is set so that driving noise can be reduced even when the environmental sound before the zoom operation is started by the imaging apparatus 100 is loud. The predetermined amplitude spectrum Dtth is set to be a level that is lower than the noise level predicted as noise noise of the imaging apparatus 100 by a certain level.

  When it is determined that the amplitude spectrum Dt1 is larger than the predetermined amplitude spectrum Dtth (No in S704), the control unit 109 determines that the environmental sound before the zoom operation is started is large. When it is determined that the amplitude spectrum Dt1 is larger than the predetermined amplitude spectrum Dtth (No in S704), the noise profile creation process is a timing chart as shown in FIG. In this case (No in S704), the control unit 109 deletes Aave (fi) stored in the memory 251a as the amplitude spectrum Dt1 so that the predetermined amplitude spectrum Dtth is stored in the memory 251a as the amplitude spectrum Dt1. (S705). When the predetermined amplitude spectrum Dtth is stored in the memory 251a as the amplitude spectrum Dt1, the control unit 109 performs the process of S706. When it is determined that the amplitude spectrum Dt1 is equal to or less than the predetermined amplitude spectrum Dtth (Yes in S704), the noise profile creation process is a timing chart as shown in FIG. In this case (Yes in S704), the control unit 109 performs the process of S706.

  Next, the control unit 109 controls the profile creation unit 252a to create the noise profile Pt by subtracting the amplitude spectrum Dt1 from the amplitude spectrum Dt2 (S706). When the amplitude spectrum Dt1 is equal to or smaller than the predetermined amplitude spectrum Dtth, the profile creation unit 252a creates a noise profile Pt by subtracting Aave (fi) from the amplitude spectrum Dt2. When the amplitude spectrum Dt1 is larger than the predetermined amplitude spectrum Dtth, the profile creation unit 252a creates a noise profile Pt by subtracting the predetermined amplitude spectrum Dtth from the amplitude spectrum Dt2. The noise profile Pt created by the profile creation unit 252a is stored in the profile storage unit 253a.

  When the stabilization period has not elapsed (No in S702), since the noise profile Pt is not stored in the profile storage unit 253a, driving noise cannot be reduced using the noise profile Pt. Therefore, the control unit 109 controls the amplitude spectrum subtraction unit 254a so that the amplitude spectrum Ut is the same as the amplitude spectrum Dt1. When the stabilization period has not elapsed, the environmental sound may fluctuate abruptly from the middle as indicated by reference numeral 501 in FIG. In this case, the control unit 109 determines whether or not the amplitude spectrum It is greater than or equal to the amplitude spectrum Dt1 (S707).

  When it is determined that the amplitude spectrum It is equal to or greater than the amplitude spectrum Dt1 (Yes in S707), the control unit 109 controls the amplitude spectrum subtracting unit 254a so that the amplitude spectrum Ut is the same as the amplitude spectrum Dt1 (S708). . When it is determined that the amplitude spectrum It is equal to or greater than the amplitude spectrum Dt1 (Yes in S707), the amplitude spectrum Ut is the same as the amplitude spectrum Dt1 until the stabilization period elapses (from timing t1 to timing t2). To be controlled. When it is determined that the amplitude spectrum It is not equal to or greater than the amplitude spectrum Dt1 (No in S707), the control unit 109 controls the amplitude spectrum subtracting unit 254a so that the amplitude spectrum Ut is the same as the amplitude spectrum It (S709). When it is determined that the amplitude spectrum It is smaller than the amplitude spectrum Dt1 (No in S707), the amplitude spectrum Ut is the same as the amplitude spectrum It until the stabilization period elapses (from timing t1 to timing t2). Controlled.

  Note that the noise profile creation processing in FIG. 7 has been described by taking as an example the case where the profile creation unit 252a creates a noise profile. However, even when the profile creation unit 252b creates a noise profile, it creates a noise profile in the same manner as the noise profile creation processing of FIG.

  In the case shown in FIG. 6, the amplitude spectrum It may be in a state equal to or greater than the amplitude spectrum Dt1, or the amplitude spectrum It may be smaller than the amplitude spectrum Dt1. Even in such a case, the amplitude spectrum Ut is controlled so as not to exceed the amplitude spectrum Dt1. Thereby, the imaging device 100 can reduce drive noise during a period until the stabilization period elapses (from timing t1 to timing t2).

  As described above, during the period from when the zoom control signal is turned ON until the stabilization period elapses (from timing t1 to timing t2), the control unit 109 causes the amplitude spectrum Ut to be the amplitude spectrum It or the amplitude spectrum Dt1. To control. As a result, the imaging apparatus 100 can reduce drive noise in a period from when the zoom control signal is turned on until the stabilization period elapses (from timing t1 to timing t2). Further, during the period after the stabilization period has elapsed (from timing t2 to timing t3), the control unit 109 uses the noise profile Pt in the period after the stabilization period has elapsed (from timing t2 to timing t3). Drive noise can be reduced. Thereby, the imaging device 100 can seamlessly reduce drive noise.

[Noise Profile Correction Processing (S309)]
The noise profile correction process executed by the control unit 109 in S309 will be described with reference to FIGS. The case where the profile correction unit 256a corrects the noise profile created by the profile creation unit 252a will be described as an example, and the noise profile correction processing will be described below.

  FIG. 8 is a timing chart showing a process for enlarging and correcting the noise profile Pt. FIG. 9 is a timing chart showing a process for reducing and correcting the noise profile Pt. Since t1, t2, t3, It, Dt, Pt, Ut, and Nt in FIGS. 8 and 9 are the same as t1, t2, t3, It, Dt, Pt, Ut, and Nt in FIGS. Omitted. FIG. 11 is a diagram illustrating setting of a time constant related to the noise profile correction process.

  FIG. 10 is a flowchart showing a noise profile correction process performed by the control unit 109. Next, the noise profile correction process performed by the control unit 109 will be described with reference to FIG. The noise profile correction process will be described below by taking as an example a case where the profile correction unit 256a corrects the noise profile created by the profile creation unit 252a. The integration circuit 250a integrates the amplitude value of each frequency of a preset number of frames after the noise profile creation processing in S307 is performed, and divides the integrated amplitude value by the preset number of frames. Then, the average amplitude value for each frequency is calculated. The number of frames set in advance may be set by the user. When the preset number of frames is “1”, the value of the amplitude spectrum output from the integration circuit 250a is equal to the value output from the FFT 207a. The integrating circuit 250a outputs an average amplitude value for each frequency as an amplitude spectrum Dt.

  The control unit 109 determines whether or not the amplitude spectrum Dt output from the integration circuit 250a is equal to or less than the amplitude spectrum Dt2 (S1001). When it is determined that the amplitude spectrum Dt is larger than the amplitude spectrum Dt2 (No in S1001), the control unit 109 determines whether or not the noise profile Pt stored in the profile storage unit 253a is equal to or less than the first value Pmax. Is determined (S1002). The first value Pmax is a threshold value for limiting the expansion correction of the noise profile Pt. Furthermore, the first value Pmax is used to prevent a sense of incongruity due to excessive reduction of drive noise.

  As shown in FIG. 8, the amplitude spectrum Dt becomes larger than the amplitude spectrum Dt2 as the drive noise increases during the period from the timing t2 to the timing t3. For this reason, the drive noise corresponding to the difference between the amplitude spectrum Dt and the amplitude spectrum Dt2 is not reduced only by causing the amplitude spectrum subtraction unit 211a to perform the subtraction process using the noise profile Pt generated by the profile creation unit 252a. It was. Therefore, when it is determined that the noise profile Pt is equal to or less than the first value Pmax (Yes in S1002), the control unit 109 performs the expansion correction of the noise profile Pt according to the time constant inc (fi). (S1003). After the enlargement correction of the noise profile Pt is performed, the control unit 109 performs the process of S1004.

  When it is determined that the noise profile Pt is not equal to or less than the first value Pmax (No in S1002), the control unit 109 performs enlargement correction of the noise profile Pt to prevent the drive noise from being excessively reduced. 271a should not be performed. When it is determined that the noise profile Pt is not equal to or less than the first value Pmax (No in S1002), the control unit 109 performs the process of S1004. When it is determined that the amplitude spectrum Dt is equal to or smaller than the amplitude spectrum Dt2 (Yes in S1001), the control unit 109 performs the process of S1004.

  The control unit 109 determines whether or not the amplitude spectrum Ut output from the amplitude spectrum subtraction unit 254a is greater than or equal to the second value Umin (S1004). The second value Umin is a threshold value that limits the reduction correction of the noise profile Pt. The second value Umin is a noise floor level, and is a minimum noise value that is recorded even when no sound is input to the sound input unit 102. When it is determined that the noise profile Pt is equal to or greater than the second value Umin (Yes in S1004), the control unit 109 prevents the profile reduction unit 272a from performing reduction correction of the noise profile Pt, and performs noise profile correction processing. Exit.

  As shown in FIG. 9, the amplitude spectrum Ut becomes smaller than the second value Utmin as the drive noise becomes smaller during the period from the timing t2 to the timing t3. For this reason, the sound corresponding to the difference between the amplitude spectrum Ut and the second value Utmin is erased only by causing the amplitude spectrum subtraction unit 254a to perform the subtraction process using the noise profile Pt created by the profile creation unit 252a. There was a case. Therefore, when it is determined that the noise profile Pt is not equal to or greater than the second value Umin (No in S1004), the control unit 109 causes the profile reduction unit 272a to perform reduction correction of the noise profile Pt according to the time constant dec (fi). (S1005). After the reduction correction of the noise profile Pt is performed, the control unit 109 ends the noise profile correction process.

  Note that the noise profile correction processing of FIG. 10 has been described by taking as an example the case where the profile correction unit 256a corrects the noise profile created by the profile creation unit 252a. However, when the profile correction unit 256b corrects the noise profile created by the profile creation unit 252b, the noise profile is corrected similarly to the noise profile correction process of FIG.

  Next, referring to FIG. 11, a method of setting the time constant inc (fi) for the expansion correction of the noise profile Pt by the profile expansion unit 271a and the time constant dec (fi) for the reduction correction of the noise profile Pt by the profile reduction unit 272a. Will be described.

  FIG. 11A is a diagram illustrating characteristics of driving noise for each frequency. FIG. 11B is a diagram showing the setting of the time constant inc (fi) corresponding to the frequency when the noise profile Pt is enlarged and corrected. FIG. 11C is a diagram illustrating the setting of the time constant dec (fi) according to the frequency when the noise profile Pt is reduced and corrected.

  In FIG. 11A, reference numeral 1101 denotes an amplitude spectrum when the zoom operation is performed by the imaging apparatus 100 as an amplitude spectrum of 512 points. Reference numeral 1102 denotes a change in the amplitude spectrum of the drive noise when the zoom operation is performed by the imaging apparatus 100. As 1102 indicates, the higher the frequency band, the greater the change in drive noise when the zoom operation is performed by the imaging apparatus 100.

  Thus, as shown in FIG. 11B, when the noise profile Pt is enlarged and corrected by the profile enlargement unit 271a, the time constant inc (fi) is set to be smaller as the frequency band is higher. This is to prevent the drive noise from being left unreduced by causing the enlargement correction of the noise profile Pt to follow the change in the drive noise quickly.

  In addition, as shown in FIG. 11C, when the noise profile Pt is reduced and corrected by the profile reduction unit 272a, the time constant dec (fi) is set to increase as the frequency band increases. This is to prevent the drive noise from remaining unreduced by causing the reduction correction of the noise profile Pt to follow the change of the drive noise slowly.

  In the present embodiment, the time constant dec (fi) for correcting the noise profile Pt to be reduced is set larger than the time constant inc (fi) for correcting the noise profile Pt to be enlarged.

  After being returned to the time-series audio signal by the IFFT 214a, the noise applying unit 215a applies the noise signal to the audio signal supplied from the IFFT 214a. The noise application unit 215a applies a noise signal in order to prevent a sense of incongruity due to excessive reduction of drive noise by the noise reduction unit 200a. It is assumed that the noise signal applied by the noise applying unit 215a is a noise floor level signal. As a result, in the subtraction processing by the amplitude spectrum subtraction unit 254a, reduction of driving noise is emphasized.

  FIG. 12 is a diagram illustrating an example of the relationship between the external sound source 1201 and the audio input unit 102. As shown in FIG. 12, when the distance between the external sound source 1201 and the imaging device 100 is sufficiently large, the distance between the external sound source 1201 and the R channel sound input unit 102a, the external sound source 1201 and the L channel sound input unit 102b, Is substantially the same. For this reason, the difference between the environmental sound acquired by the microphone 205a and the environmental sound acquired by the microphone 205b is small.

  However, the influence of the drive noise due to the difference between the distance between the optical lens 201 and the R channel audio input unit 102a and the distance between the optical lens 201 and the L channel audio input unit 102b is different. Therefore, it is necessary to consider the influence of driving noise on the R channel audio input unit 102a and the influence of driving noise on the L channel audio input unit 102b, respectively.

As shown in the following equation, the difference between the influence of driving noise on the R channel audio input unit 102a and the influence of driving noise on the L channel audio input unit 102b becomes large. In the following equation, “DtL” is the amplitude value of the L (Left) channel before the noise reduction processing is performed, and “DtR” is the amplitude of the R (Right) channel before the noise reduction processing is performed. Value. Further, “βt” in the following expression is a left-right correlation amplitude spectrum.
βt = | DtL−DtR | / (DtL + DtR)

  The difference between Lch and Rch increases as the volume of the environmental sound increases. However, in the case as shown in FIG. 12, the distance between the external sound source 1201 and the R channel sound input unit 102a and the distance between the external sound source 1201 and the L channel sound input unit 102b are substantially the same. Get smaller. Regarding the drive noise, the left-right correlation amplitude spectrum βt increases due to the difference between the distance between the optical lens 201 and the R channel audio input unit 102a and the distance between the optical lens 201 and the L channel audio input unit 102b. Whether the driving noise is dominant with respect to the environmental sound can be determined from the left-right correlation amplitude spectrum βt.

  Next, a noise profile correction process that takes into account the influence of drive noise on Rch and the influence of drive noise on Lch will be described with reference to FIGS. 13 and 14.

FIG. 13 is a timing chart showing a process for correcting a noise profile for Rch and Lch. Since t1, t2, and t3 in FIG. 13 are the same as t1, t2, and t3 in FIGS. 5 and 6, description thereof is omitted.
In FIG. 13, the amplitude spectrum of the predetermined frequency fi that is fast Fourier transformed by the FFT 207a is “ItR”, and the amplitude spectrum of the predetermined frequency fi that is fast Fourier transformed by the FFT 207b is “ItL”. The amplitude spectrum ItR is indicated by a dotted line, and the amplitude spectrum ItL is indicated by a solid line. Further, in FIG. 13, the amplitude spectrum indicating the amplitude of the predetermined frequency fi integrated by the integration circuit 250a is “DtR”, and the amplitude spectrum indicating the amplitude of the predetermined frequency fi integrated by the integration circuit 250b is “DtL”. And The amplitude spectrum DtR is indicated by a dotted line, and the amplitude spectrum DtL is indicated by a solid line. In FIG. 13, the noise profile corresponding to the predetermined frequency fi created by the profile creation unit 252a is “PtR”, and the noise profile corresponding to the predetermined frequency fi created by the profile creation unit 252b is “PtL”. . The noise profile PtR is indicated by a dotted line, and the noise profile PtL is indicated by a solid line. At timing t2, a noise profile PtR is created by the profile creation unit 252a, and a noise profile PtL is created by the profile creation unit 252b.

  In FIG. 13, the amplitude spectrum of the predetermined frequency fi output from the amplitude spectrum subtraction unit 254a is “UtR”, and the amplitude spectrum of the predetermined frequency fi output from the amplitude spectrum subtraction unit 254b is “UtL”. The amplitude spectrum UtR is indicated by a dotted line, and the amplitude spectrum UtL is indicated by a solid line. In FIG. 13, a time-series digital audio signal having a predetermined frequency fi after the noise signal is applied by the noise applying unit 215 a is referred to as “NtR”. In FIG. 13, a time-series digital audio signal having a predetermined frequency fi after the noise signal is applied by the noise applying unit 215 b is “NtL”. The amplitude spectrum NtR is indicated by a dotted line, and the amplitude spectrum NtL is indicated by a solid line. In FIG. 13, | ItL−ItR |, which is the absolute value of the difference between the amplitude spectrum ItL and the amplitude spectrum ItR, is indicated by a solid line. In FIG. 13, | UtL−UtR |, which is the absolute value of the difference between the amplitude spectrum UtL and the amplitude spectrum UtR, is indicated by a dotted line.

  As illustrated in FIG. 13, | UtL−UtR | may exceed | ItL−ItR |. This indicates that the drive noise of one of the amplitude spectrum UtL and the amplitude spectrum UtR is excessively reduced by the subtraction process. This is because one of the noise profile PtL and the noise profile PtR is too large.

  FIG. 14 is a flowchart illustrating an example of noise profile correction processing for Rch and Lch. Next, the noise profile correction processing for Rch and Lch performed by the control unit 109 will be described with reference to FIG. 10 is performed on the noise profile PtR, and after the noise profile correction process of FIG. 10 is performed on the noise profile PtL, the noise profile correction process on Rch and Lch in FIG. 14 is performed. Done.

Thereafter, the control unit 109 detects the amplitude spectrum ItL, the amplitude spectrum ItR, the amplitude spectrum UtL, and the amplitude spectrum UtR, and determines whether or not the following conditions are satisfied (S1401).
Condition: | ItL-ItR | ≦ | UtL-UtR |
When it is determined that the condition | ItL−ItR | ≦ | UtL−UtR | is satisfied (Yes in S1401), the control unit 109 determines whether or not the amplitude spectrum UtL is greater than or equal to the amplitude spectrum UtR ( S1402). When the amplitude spectrum UtL is equal to or larger than the amplitude spectrum UtR (Yes in S1402), the control unit 109 causes the profile enlargement unit 271b to perform the enlargement correction of the noise profile PtL according to the time constant inc_L (fi) (S1403). The time constant inc_L (fi) is a time constant corresponding to the profile enlarging unit 271b. Thereafter, the control unit 109 causes the profile reduction unit 272a to perform reduction correction of the noise profile PtR according to the time constant dec_R (fi) (S1404). The time constant dec_R (fi) is a time constant corresponding to the profile reduction unit 272a. After the processing of S1404 is performed, the noise profile correction processing for Rch and Lch ends. The time constant dec_R (fi) is larger than the time constant inc_L (fi).

  When it is determined that the amplitude spectrum UtL is smaller than the amplitude spectrum UtR (No in S1402), the control unit 109 causes the profile expansion unit 271a to perform expansion correction of the noise profile PtR according to the time constant inc_R (fi) ( S1405). The time constant inc_R (fi) is a time constant corresponding to the profile enlarging unit 271a. Thereafter, the control unit 109 causes the profile reduction unit 272b to perform reduction correction of the noise profile PtL according to the time constant dec_L (fi) (S1406). The time constant dec_L (fi) is a time constant corresponding to the profile reduction unit 272b. After the processing of S1406 is performed, the noise profile correction processing for Rch and Lch ends. The time constant dec_L (fi) is larger than the time constant inc_R (fi).

  When the condition | ItL−ItR | ≦ | UtL−UtR | is not satisfied, | ItL−ItR |> | UtL−UtR | If it is determined that the condition | ItL−ItR | ≦ | UtL−UtR | is not satisfied (No in S1401), the noise profile correction processing for Rch and Lch ends.

  As described above, the control unit 109 corrects the noise profile PtR and corrects the noise profile PtL in accordance with changes in environmental sound and driving noise. Accordingly, the imaging apparatus 100 can appropriately perform the noise reduction process for the Rch sound and the noise reduction process for the Lch sound. Therefore, the imaging apparatus 100 can prevent a situation in which the environmental sound is uncomfortable due to the drive noise remaining unerased or the drive noise being excessively reduced.

[Noise reduction processing (S308)]
The noise reduction process executed by the control unit 109 in S308 will be described with reference to FIGS.

  FIG. 15 is a timing chart showing noise reduction processing for Rch and Lch. Since t1, t2, and t3 in FIG. 15 are the same as t1, t2, and t3 in FIGS. 5 and 6, description thereof is omitted. In FIG. 15, ItR, ItL, DtR, DtL, PtR, PtL, UtR, UtL, NtR and NtL are the same as ItR, ItL, DtR, DtL, PtR, PtL, UtR, UtL, NtR and NtL in FIG. Therefore, the description is omitted.

  If the environmental sound or driving noise changes abruptly while the zoom operation is being performed by the imaging apparatus 100, even if the driving noise is reduced using the noise profile PtR and the noise profile PtL, the remaining noise of driving noise and the environment The sound may be uncomfortable. In order to prevent this, the control unit 109 performs noise reduction processing according to the left-right correlation amplitude spectrum βt.

  FIG. 16 is a flowchart illustrating an example of noise reduction processing. FIG. 17 is a diagram illustrating the relationship between the coefficient α and the environmental sound. The horizontal axis in FIG. 17 indicates the environmental sound level, and the vertical axis in FIG. 17 indicates the value of the coefficient α. In FIG. 17, the coefficient α is associated with the environmental sound level. A solid line 1701 in FIG. 17 indicates the value of the coefficient α corresponding to the environmental sound level. A broken line 1702 is a drive noise level, and a broken line 1703 is a level at which the drive noise is erased by the environmental sound. The coefficient α decreases as the environmental sound level increases. When the environmental sound level is the level indicated by the broken line 1702, the coefficient α is 0.125.

  Next, the noise reduction process performed by the control unit 109 will be described with reference to FIGS. Note that the noise reduction processing in FIG. 16A will be described using an example in which the amplitude spectrum subtraction unit 254a performs subtraction processing.

  At timing t1, the control unit 109 determines the coefficient α according to the amplitude spectrum Dt1 stored in the memory 251a (S1601). The coefficient α is a coefficient by which the noise profile is multiplied. In S1601, the control unit 109 detects the level of the environmental sound in FIG. 17 corresponding to the amplitude spectrum Dt1, and determines the value of the coefficient α corresponding to the detected level of the environmental sound.

  Next, the control unit 109 calculates the left-right correlation amplitude spectrum βt as described above (S1602). Thereafter, the control unit 109 determines whether or not the left-right correlation amplitude spectrum βt is equal to or smaller than the third value βth (S1603). Note that the third value βth is set according to the value of the left-right correlation amplitude spectrum βt calculated when there is no environmental sound. The greater the level of environmental sound, the closer the left-right correlation amplitude spectrum βt is to zero. When the driving noise is dominant with respect to the environmental sound, the left-right correlation amplitude spectrum βt is 0.2 or more.

When it is determined that the left-right correlation amplitude spectrum βt is equal to or smaller than the third value βth (Yes in S1603), the control unit 109 performs the process of S1604. In S1604, the control unit 109 multiplies the noise profile PtR by the coefficient α determined in S1601, and controls the amplitude spectrum subtraction unit 254a to subtract this from the amplitude spectrum ItR. In S1604, when the subtraction process is performed by the amplitude spectrum subtraction unit 254a, the amplitude spectrum UtR output from the amplitude spectrum subtraction unit 254a is expressed by the following equation.
UtR = ItR−α · PtR

When it is determined that the left-right correlation amplitude spectrum βt is larger than the third value βth (No in S1603), the control unit 109 performs the process of S1605. In step S1605, the control unit 109 multiplies the first value Pmax by the coefficient α determined in step S1601, and controls the amplitude spectrum subtraction unit 254a to subtract this from the amplitude spectrum ItR. In S1605, when the subtraction process is performed by the amplitude spectrum subtraction unit 254a, the amplitude spectrum UtR output from the amplitude spectrum subtraction unit 254a is expressed by the following equation.
UtR = ItR−α · Pmax

  When it is determined that the left-right correlation amplitude spectrum βt is not equal to or smaller than the third value βth (No in S1603), the control unit 109 does not use the noise profile PtR. Note that the noise reduction processing in FIG. 16A has been described by taking as an example the case where the amplitude spectrum subtraction unit 254a performs the subtraction processing. However, also when the amplitude spectrum subtraction unit 254b performs the subtraction process, the drive noise is reduced in the same manner as the noise reduction process of FIG.

  Next, the noise reduction process performed by the control unit 109 will be described with reference to FIGS. Note that the noise reduction processing in FIG. 16B will be described by taking the case where the amplitude spectrum subtraction unit 254a performs the subtraction processing as an example.

  Since S1602, S1603, and S1604 in FIG. 16B are the same processes as S1602, S1603, and S1604 in FIG. The control unit 109 determines the coefficient α according to the amplitude spectrum Ut−1 after the subtraction process of the previous frame (S1606). In S1606, the control unit 109 detects the level of the environmental sound in FIG. 17 corresponding to the amplitude spectrum Ut-1, and determines the value of the coefficient α corresponding to the detected level of the environmental sound. When the amplitude spectrum subtraction unit 254a performs the subtraction process, in step S1606, the control unit 109 determines the coefficient α according to the amplitude spectrum Ut-1R after the subtraction process for the previous frame performed by the amplitude spectrum subtraction unit 254a. . Thereafter, the control unit 109 performs the processes of S1602 and S1603. When it is determined that the left-right correlation amplitude spectrum βt is equal to or smaller than the third value βth (Yes in S1603), the control unit 109 performs the process of S1604. When it is determined that the left-right correlation amplitude spectrum βt is not equal to or smaller than the third value βth (No in S1603), the control unit 109 performs the process of S1607.

In S1607, the control unit 109 multiplies the second value Umin by the coefficient α determined in S1606, and controls the amplitude spectrum subtraction unit 254a to subtract this from the amplitude spectrum ItR. In S1607, when the subtraction process is performed by the amplitude spectrum subtraction unit 254a, the amplitude spectrum UtR output from the amplitude spectrum subtraction unit 254a is expressed by the following equation.
UtR = ItR−α · Umin

  When it is determined that the left-right correlation amplitude spectrum βt is not equal to or smaller than the third value βth (No in S1603), the control unit 109 does not use the noise profile PtR. Note that the noise reduction process in FIG. 16B has been described by taking as an example the case where the amplitude spectrum subtraction unit 254a performs the subtraction process. However, even when the amplitude spectrum subtraction unit 254b performs the subtraction process, the drive noise is reduced in the same manner as the noise reduction process of FIG.

  Although the noise reduction processing has been described with reference to FIGS. 16A and 16B, any one of the noise reduction processing in FIGS. 16A and 16B may be performed by the control unit 109. And

  As described above, the control unit 109 changes the process for reducing noise according to the left-right correlation amplitude spectrum βt. Thereby, the imaging device 100 can appropriately reduce the drive noise depending on whether the drive noise is dominant with respect to the environmental sound.

[Post-processing (S310)]
In S310, post-processing executed by the control unit 109 will be described with reference to FIGS.

  FIG. 18 is a timing chart showing post-processing. Since t1, t2, and t3 in FIG. 18 are the same as t1, t2, and t3 in FIGS. 5 and 6, description thereof is omitted. UtR and UtL in FIG. 18 are the same as UtR and UtL in FIG. The amplitude spectrum Qt in FIG. 18 is an amplitude spectrum that is output after post-processing.

  Next, post-processing performed by the control unit 109 will be described with reference to FIGS. 19A and 18A. When the amplitude spectrum UtR is output from the amplitude spectrum subtraction unit 254a and the amplitude spectrum UtL is output from the amplitude spectrum subtraction unit 254b, the control unit 109 determines whether or not the amplitude spectrum UtL is equal to or less than the amplitude spectrum UtR ( S1901).

  When it is determined that the amplitude spectrum UtL is equal to or smaller than the amplitude spectrum UtR (Yes in S1901), the control unit 109 controls the post-correction unit 255b to output the amplitude spectrum UtL as the amplitude spectrum Qt to the IFFT 214b. Thereafter, the control unit 109 controls the post-correction unit 255a to output the amplitude spectrum UtL as the amplitude spectrum Qt to the IFFT 214a without outputting the amplitude spectrum UtR to the IFFT 214a (S1902). After the process of S1902 is performed, the control unit 109 ends the post-process.

  When it is determined that the amplitude spectrum UtL is larger than the amplitude spectrum UtR (No in S1901), the control unit 109 controls the post-correction unit 255a to output the amplitude spectrum UtR to the IFFT 214a as the amplitude spectrum Qt. Thereafter, the control unit 109 controls the post-processing unit 255b to output the amplitude spectrum UtR as the amplitude spectrum Qt to the IFFT 214b without outputting the amplitude spectrum UtL to the IFFT 214b (S1903). After the processing of S1903 is performed, the control unit 109 ends the post processing.

  When the post-processing of FIG. 19A is performed, as shown in FIG. 18A, the smaller one of the amplitude spectrum UtL and the amplitude spectrum UtR is input to the IFFT 214a and IFFT 214b as the amplitude spectrum Qt.

  Next, post-processing performed by the control unit 109 will be described with reference to FIGS. 19B and 18B. When the amplitude spectrum UtR is output from the amplitude spectrum subtraction unit 254a and the amplitude spectrum UtL is output from the amplitude spectrum subtraction unit 254b, the control unit 109 performs the process of S1910. In step S1910, the control unit 109 determines whether any one of the amplitude spectrum UtL and the amplitude spectrum UtR is equal to or less than the fourth value Qmin. Note that the fourth value Qmin is used to prevent a sense of incongruity due to post-processing. The fourth value Qmin may be the same value as the second value Umin.

  When any one of the amplitude spectrum UtL and the amplitude spectrum UtR is equal to or smaller than the fourth value Qmin (Yes in S1910), the control unit 109 determines whether the amplitude spectrum UtL is equal to or smaller than the amplitude spectrum UtR (S1914). ). When it is determined that the amplitude spectrum UtL is equal to or smaller than the amplitude spectrum UtR (Yes in S1914), the control unit 109 controls the post-correction unit 255a to output the amplitude spectrum UtR as the amplitude spectrum Qt to the IFFT 214a. Thereafter, the control unit 109 controls the post-correction unit 255b to output the amplitude spectrum UtR as the amplitude spectrum Qt to the IFFT 214b without outputting the amplitude spectrum UtL to the IFFT 214b (S1915). After the process of S1915 is performed, the control unit 109 ends the post-process. When it is determined that the amplitude spectrum UtL is larger than the amplitude spectrum UtR (No in S1914), the control unit 109 controls the post-correction unit 255b to output the amplitude spectrum UtL as the amplitude spectrum Qt to the IFFT 214b. Thereafter, the control unit 109 controls the post-correction unit 255a to output the amplitude spectrum UtL as the amplitude spectrum Qt to the IFFT 214a without outputting the amplitude spectrum UtR to the IFFT 214a (S1916). After the processing of S1916 is performed, the control unit 109 ends the post processing.

  When both the amplitude spectrum UtL and the amplitude spectrum UtR are larger than the fourth value Qmin (No in S1910), the control unit 109 determines whether or not the amplitude spectrum UtL is equal to or smaller than the amplitude spectrum UtR (S1911). ).

  When it is determined that the amplitude spectrum UtL is equal to or smaller than the amplitude spectrum UtR (Yes in S1911), the control unit 109 controls the post-correction unit 255b to output the amplitude spectrum UtL as the amplitude spectrum Qt to the IFFT 214b. Thereafter, the control unit 109 controls the post-correction unit 255a to output the amplitude spectrum UtL as the amplitude spectrum Qt to the IFFT 214a without outputting the amplitude spectrum UtR to the IFFT 214a. After the process of S1912 is performed, the control unit 109 ends the post-process.

  When it is determined that the amplitude spectrum UtL is larger than the amplitude spectrum UtR (No in S1911), the control unit 109 controls the post-correction unit 255a to output the amplitude spectrum UtR as the amplitude spectrum Qt to the IFFT 214a. Thereafter, the control unit 109 controls the post-correction unit 255b to output the amplitude spectrum UtR as the amplitude spectrum Qt to the IFFT 214b without outputting the amplitude spectrum UtL to the IFFT 214b (S1913). After the process of S1913 is performed, the control unit 109 ends the post-process.

  When post-processing in FIG. 19B is performed, a case will be described in which both the amplitude spectrum UtL and the amplitude spectrum UtR are larger than the fourth value Qmin. In this case, as shown in FIG. 18B, the smaller one of the amplitude spectrum UtL and the amplitude spectrum UtR is input to the IFFT 214a and the IFFT 214b as the amplitude spectrum Qt.

  Next, a case where any one of the amplitude spectrum UtL and the amplitude spectrum UtR is equal to or smaller than the fourth value Qmin when the post-processing of FIG. 19B is performed will be described. In this case, as shown in FIG. 18B, the larger one of the amplitude spectrum UtL and the amplitude spectrum UtR is input to the IFFT 214a and the IFFT 214b as the amplitude spectrum Qt.

Next, post-processing performed by the control unit 109 will be described with reference to FIGS. 19C and 18C. When the amplitude spectrum UtR is output from the amplitude spectrum subtraction unit 254a and the amplitude spectrum UtL is output from the amplitude spectrum subtraction unit 254b, the control unit 109 performs the process of S1921. In S1921, the control unit 109 calculates ΔtL and ΔtR, and calculates | ΔtL−ΔtR |. ΔtL is a difference between the amplitude spectrum ItL and the amplitude spectrum UtL, and ΔtR is a difference between the amplitude spectrum ItR and the amplitude spectrum UtR. Furthermore, the control unit 109 determines whether or not the following conditions are satisfied.
| ΔtL−ΔtR | ≦ | ΔtL−ΔtR | max
Note that | ΔtL−ΔtR | max is a predetermined threshold value, and is used to prevent a sense of incongruity between the left and right environmental sounds due to the difference between ΔtL and ΔtR.

  When it is determined that the condition | ΔtL−ΔtR | ≦ | ΔtL−ΔtR | max is satisfied (Yes in S1921), the control unit 109 determines whether or not the amplitude spectrum UtL is equal to or smaller than the amplitude spectrum UtR. (S1922). When it is determined that the amplitude spectrum UtL is equal to or smaller than the amplitude spectrum UtR (Yes in S1922), the control unit 109 controls the post-correction unit 255b to output the amplitude spectrum UtL as the amplitude spectrum Qt to the IFFT 214b. Thereafter, the control unit 109 controls the post-correction unit 255a to output the amplitude spectrum UtL as the amplitude spectrum Qt to the IFFT 214a without outputting the amplitude spectrum UtR to the IFFT 214a (S1923).

  After the process of S1923 is performed, the control unit 109 ends the post-process. When it is determined that the amplitude spectrum UtL is larger than the amplitude spectrum UtR (No in S1922), the control unit 109 controls the post-correction unit 255a to output the amplitude spectrum UtR as the amplitude spectrum Qt to the IFFT 214a. Thereafter, the control unit 109 controls the post-correction unit 255b to output the amplitude spectrum UtR as the amplitude spectrum Qt to the IFFT 214b without outputting the amplitude spectrum UtL to the IFFT 214b (S1924). After the processing of S1924 is performed, the control unit 109 ends the post processing.

  When the condition | ΔtL−ΔtR | ≦ | ΔtL−ΔtR | max is not satisfied, | ΔtL−ΔtR |> | ΔtL−ΔtR | max. When it is determined that the condition | ΔtL−ΔtR | ≦ | ΔtL−ΔtR | max is not satisfied (No in S1921), the control unit 109 determines whether or not the amplitude spectrum UtL is equal to or smaller than the amplitude spectrum UtR. (S1925).

  When it is determined that the amplitude spectrum UtL is equal to or smaller than the amplitude spectrum UtR (Yes in S1925), the control unit 109 controls the post-correction unit 255a so as to output the amplitude spectrum UtR to the IFFT 214a as the amplitude spectrum Qt. Thereafter, the control unit 109 controls the post-correction unit 255b to output the amplitude spectrum UtR as the amplitude spectrum Qt to the IFFT 214b without outputting the amplitude spectrum UtL to the IFFT 214b (S1926). After the process of S1926 is performed, the control unit 109 ends the post-process. When it is determined that the amplitude spectrum UtL is larger than the amplitude spectrum UtR (No in S1925), the control unit 109 controls the post-correction unit 255b to output the amplitude spectrum UtL as the amplitude spectrum Qt to the IFFT 214b. Thereafter, the control unit 109 controls the post-correction unit 255a to output the amplitude spectrum UtL as the amplitude spectrum Qt to the IFFT 214a without outputting the amplitude spectrum UtR to the IFFT 214a (S1927). After the process of S1927 is performed, the control unit 109 ends the post-process.

A case where | ΔtL−ΔtR | ≦ | ΔtL−ΔtR | max is satisfied when the post-processing in FIG. 19C is performed will be described. In this case, as shown in FIG. 18C, the smaller one of the amplitude spectrum UtL and the amplitude spectrum UtR is input to the IFFT 214a and the IFFT 214b as the amplitude spectrum Qt.
Next, a case where | ΔtL−ΔtR | ≦ | ΔtL−ΔtR | max does not hold when post-processing in FIG. 19C is performed will be described. In this case, as shown in FIG. 18C, the larger one of the amplitude spectrum UtL and the amplitude spectrum UtR is input to the IFFT 214a and the IFFT 214b as the amplitude spectrum Qt.

  The post-processing has been described with reference to FIGS. 19A, 19B, and 19C. The post-processing of any one of FIGS. 19A, 19B, and 19C is performed. Is assumed to be performed by the control unit 109.

  After the post-processing of any one of FIGS. 19A, 19B, and 19C is performed, the IFFT 214a uses the phase information supplied from the FFT 207a to the amplitude spectrum Qt. By performing inverse fast Fourier transform, the original time-series audio data is generated. After the post-processing of any one of FIGS. 19A, 19B, and 19C is performed, the IFFT 214b uses the phase information supplied from the FFT 207b to the amplitude spectrum Qt. By performing inverse fast Fourier transform, the original time-series audio data is generated.

  In this way, the control unit 109 performs processing for correcting the Rch sound and the Lch sound so that the levels match. As a result, the imaging apparatus 100 can prevent a sense of incongruity due to the difference between the left and right environmental sounds.

In the present embodiment, the imaging apparatus 100 has been described as having a configuration in which two channels of Rch and Lch are input. However, the imaging device 100 has a configuration in which audio having two or more channels is input. There may be. In addition, the imaging apparatus 100 may be configured to receive one system of audio.
(Other examples)
The above-described embodiments can be realized by causing a computer (or CPU, MPU, etc.) to execute software such as a program. In this case, the software is supplied to a computer (or CPU, MPU, etc.) via a network or a storage medium.

Claims (5)

A first microphone,
A second microphone,
Input means for inputting a drive instruction for driving the drive means ;
First conversion means for acquiring sound spectrum data by performing Fourier transform on sound data obtained from the first microphone;
Second conversion means for acquiring sound spectrum data by performing Fourier transform on the sound data obtained from the second microphone;
A first average value is an average value of the voice spectrum data obtained by converting by said first of said first converting means the audio data obtained from the microphone before the driving instruction is input The voice data obtained from the first microphone is converted by the first conversion means after a predetermined stabilization period until the voice spectrum data is stabilized after the drive instruction is input. a first generation means for generating a first noise spectrum data representing the driving noise of the difference in basis, the driving means and the second average value is an average value of the voice spectrum data obtained by,
A third average value is an average value of the voice spectrum data obtained by converting by said second of said second converting means the audio data obtained from the microphone before the driving instruction is input The voice data obtained from the second microphone is converted by the second conversion means between the input of the driving instruction and the elapse of a predetermined stabilization period until the voice spectrum data is stabilized. a second generation means for generating a second noise spectrum data representing the driving noise of the difference in basis, the driving means and the fourth average value is an average value of the voice spectrum data obtained by,
The voice spectrum data obtained from the first conversion means and the second voice spectrum data obtained from the first conversion means and the sum of the voice spectrum data obtained from the second conversion means. When the value indicating the ratio of the difference from the audio spectrum data obtained from the conversion means is equal to or less than a predetermined value, control is performed so that the first noise spectrum data is used to reduce the driving noise, and the value is If the not less than a predetermined value, the electronic device characterized by chromatic and control means for controlling so as not using the first noise spectrum data in order to reduce the driving noise. The control means includes voice spectrum data obtained from the first conversion means for a sum of the voice spectrum data obtained from the first conversion means and the voice spectrum data obtained from the second conversion means; When the value indicating the ratio of the difference from the audio spectrum data obtained from the second conversion means is equal to or less than a predetermined value, control is performed to use the second noise spectrum data in order to reduce the driving noise. If the value is not less than the predetermined value, the electronic device according to claim 1, wherein the controller controls so as not using the second noise spectrum data in order to reduce the driving noise. It said drive means, an electronic device according to claim 1 or 2, characterized by controlling so as to move the lens. The electronic apparatus according to any one of claims 1 to 3 , further comprising an imaging unit for imaging the image data. A control method for controlling an electronic apparatus having a first microphone, a second microphone, and input means for inputting a drive instruction for driving the drive means,
A first conversion step of acquiring audio spectrum data by performing Fourier transform on the audio data obtained from the first microphone;
A second transforming step of acquiring speech spectrum data by performing Fourier transform on the speech data obtained from the second microphone;
A first average value is an average value of the voice spectrum data obtained by converting by said first of said first converting step the audio data obtained from the microphone before the driving instruction is input The voice data obtained from the first microphone is converted by the first conversion step after a predetermined stabilization period until the voice spectrum data is stabilized after the driving instruction is input. a first generation step of generating a first noise spectrum data representing the driving noise of the difference in basis, the driving means and the second average value is an average value of the voice spectrum data obtained by,
A third average value is an average value of the voice spectrum data obtained by converting by said second of said second conversion step the audio data obtained from the microphone before the driving instruction is input The voice data obtained from the second microphone is converted by the second conversion step after a predetermined stabilization period until the voice spectrum data is stabilized after the driving instruction is input. a second generation step of generating a second noise spectrum data representing the driving noise of the difference in basis, the driving means and the fourth average value is an average value of the voice spectrum data obtained by,
The voice spectrum data obtained from the first conversion step and the second voice spectrum data obtained from the first conversion step and the sum of the voice spectrum data obtained from the second conversion step. When the value indicating the ratio of the difference from the audio spectrum data obtained from the conversion step is equal to or less than a predetermined value , control is performed to use the first noise spectrum data in order to reduce the driving noise, and the value is And a control step for controlling not to use the first noise spectrum data in order to reduce the driving noise when the predetermined value is not less than the predetermined value .

Images