JP4639902B2 - Imaging apparatus, audio recording method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To properly eliminate a mechanism noise sound caused at photographing from a sound signal and recording the resulting sound signal. <P>SOLUTION: A storage section 54 stores a plurality of spectral patterns of motor sound picked up in advance through a sound input section 51 (main microphone) as first noise spectral patterns. Further, a storage section 64 stores a plurality of spectral patterns of motor sound picked up in advance through a reference input section 61 (reference microphone) as second noise spectral patterns. In photographing, the second noise spectral in accordance with the motor sound obtained from the reference input section 61 is selected from the storage section 64 and the first noise spectral corresponding to the second noise spectral is selected from the storage section 54 and both are given to a subtract section 55. Thus, the noise component can properly be eliminated by subtracting the noise spectral with the same input characteristic as that of the spectral of the sound signal from the spectral of the sound signal obtained through the sound input section 51. <P>COPYRIGHT: (C)2007,JPO&amp;INPIT

Description

  The present invention relates to an imaging apparatus such as a digital camera, and more particularly to an imaging apparatus having a function capable of recording an audio signal input during imaging together with a captured image, and an audio recording method and program used for the imaging apparatus.

  Conventionally, a spectral subtraction method is known as a method for removing noise superimposed on an audio signal. This spectral subtraction method (hereinafter referred to as the SS method) estimates the spectrum in the silent section as a noise spectrum, and subtracts a signal obtained by multiplying the noise spectrum by a predetermined coefficient (subtract coefficient) from the input speech spectrum. This is a method for removing components.

  Here, in Patent Document 1, in the noise removal system using the SS method, the subtract coefficient is changed for each frame depending on the frame power of the audio signal, thereby reducing the spectrum distortion due to the excessive pulling of the estimated noise spectrum. Is disclosed. That is, input speech due to excessive estimation noise spectrum by multiplying normal subtract coefficients such as vowels and reducing the subtract coefficient in parts with low speech power such as burst consonants. The distortion of the spectrum is suppressed.

Further, Patent Document 2 proposes a method of estimating a noise spectrum from a spectrum input from a separately provided reference input unit, instead of a noise spectrum.
JP-A-8-2221092 JP-A-5-165492

  As described above, a method for removing a noise component from input speech using the SS method is known. However, a digital camera equipped with a moving image recording function with sound has a problem that a mechanism sound such as a zoom sound or a focus sound is generated and enters the input sound regardless of the sound input during the shooting.

  In this case, with the method of estimating the noise spectrum from the speech spectrum signal in the silent section as in Patent Document 1, the mechanical sound generated regardless of the speech input cannot be removed as noise.

  In addition, the configuration in which a reference input unit is newly provided as in Patent Document 2 has the following problems.

  Consider the application of the SS method to the reduction of mechanical sounds, such as zoom motor sound, during moving image shooting with a digital camera. That is, a main microphone that is installed on the housing of the camera device and mainly collects audio signals and a reference microphone that is installed near the motor in the device housing and mainly collects motor sound are prepared. Then, after generating the spectrum of the audio signal input from the main microphone and the spectrum of the motor sound input from the reference microphone, it is included in the audio signal from the main microphone by subtracting the motor sound spectrum signal from the audio spectrum signal. The zoom sound will be reduced.

  However, the transfer function from the motor to each microphone has considerably different characteristics due to the difference in installation location of the main microphone and the reference microphone. In other words, since the reference microphone mainly inputs only the motor sound inside the device without inputting external sound, it includes multiple reflected sounds inside the device case, vibrations transmitted through the electronic board and case, etc. become.

  Therefore, if the spectrum of the motor sound input through the reference microphone is subtracted from the spectrum of the audio signal input through the main microphone, the motor sound cannot be removed correctly because the input characteristics of the two differ, and the spectrum is excessively pulled. Therefore, there is a problem that the input voice itself is distorted.

  The present invention has been made in view of the above points, and an object of the present invention is to provide an imaging apparatus, an audio recording method, and a program capable of appropriately removing and recording a mechanical sound generated during imaging from an audio signal. To do.

  The imaging apparatus according to claim 1 of the present invention is an imaging apparatus having an audio recording function for removing and recording mechanical sound generated as a result of a shooting operation from an audio signal as noise when performing moving image shooting with audio. A first input means for inputting an audio signal; a first storage means for storing a plurality of patterns of the spectrum of the mechanism sound previously collected through the first input means as a first noise spectrum; and a mechanism sound. A second input means provided in the vicinity of the source of the noise and a spectrum of the mechanical sound collected in advance through the second input means as a second noise spectrum in association with each pattern of the first noise spectrum Based on the second storage means storing a plurality of patterns and the spectrum of the mechanical sound obtained from the second input means at the time of shooting, the characteristics of the mechanical sound are selected from the second storage means. The second noise spectrum is selected, and the selection means for selecting the first noise spectrum corresponding to the second noise spectrum from the first storage means and the first input means are obtained. Noise removal means for removing a noise component by subtracting a signal obtained by multiplying a first noise spectrum selected by the selection means by a predetermined coefficient from the spectrum of the audio signal, and noise removal obtained by the noise removal means The present invention is characterized by comprising inverse conversion means for inversely converting a later audio spectrum into the original audio signal, and recording means for recording the audio signal obtained by the inverse conversion means together with the photographed image.

  According to such a configuration, a signal of only the mechanical sound is input through the second input means provided in the vicinity of the mechanical sound generation source separately from the first input means, and the characteristic of the mechanical sound is dealt with. The noise spectrum is subtracted from the input speech spectrum. At that time, by subtracting using a noise spectrum having the same input characteristics as the voice signal, the mechanical sound included in the voice signal can be appropriately removed and recorded as a noise component.

  According to a second aspect of the present invention, in the imaging apparatus according to the first aspect, the spectrum of the mechanical sound obtained from the second input means during photographing and the noise spectrum stored in the second storage means Similarity calculation means for calculating the similarity to each pattern is provided, and the selection means is configured to select a second noise having the highest similarity from the second storage means based on a calculation result of the similarity calculation means. A spectrum is selected, and a first noise spectrum corresponding to the second noise spectrum is selected from the first storage means.

  According to such a configuration, it is possible to appropriately remove and record the mechanical sound included in the audio signal as a noise component using the noise spectrum closest to the characteristic of the mechanical sound.

  According to a third aspect of the present invention, in the imaging apparatus according to the first aspect, the mechanical sound includes a driving sound of a specific motor related to a photographing operation, and the first and second storage units include Each pattern of the noise spectrum corresponding to at least three periods of when the motor starts to be driven, during steady rotation, and when the drive is stopped is stored.

  According to such a configuration, when the mechanism sound includes a driving sound of a specific motor related to the photographing operation, each pattern of the noise spectrum is selectively selected according to each period when the motor is driven. By using it, the mechanical sound contained in the audio signal can be appropriately removed as a noise component and recorded.

  According to a fourth aspect of the present invention, in the imaging apparatus according to the first aspect, the mechanical sound includes a driving sound of a specific motor related to a photographing operation, and the first and second storage units include Each pattern of the noise spectrum when the motor is driven in a predetermined direction and when the motor is driven in a direction opposite to the predetermined direction is stored.

  According to such a configuration, when the mechanism sound includes a driving sound of a specific motor related to the shooting operation, each noise spectrum pattern is selectively used according to the driving direction of the motor, The mechanical sound included in the signal can be appropriately removed as a noise component and recorded.

  According to a fifth aspect of the present invention, in the imaging apparatus according to the first aspect, the mechanical sound includes a driving sound of a specific motor related to a photographing operation, and the first and second storage units include , When the motor is driven in a predetermined direction and when it is driven in a direction opposite to the predetermined direction, the motor corresponds to at least three periods at the start of driving, at the time of steady rotation, and at the time of stopping driving. Each pattern of the noise spectrum is stored.

  According to such a configuration, when the mechanism sound includes a driving sound of a specific motor related to the photographing operation, each pattern of the noise spectrum is selectively used according to the rotation direction and the driving period of the motor. Thus, the mechanical sound included in the audio signal can be appropriately removed as a noise component and recorded.

  According to a sixth aspect of the present invention, in the imaging apparatus according to any one of the first to fifth aspects, a power calculating unit that calculates the power of the mechanical sound obtained from the second input unit; And control means for controlling a noise removal operation by the noise removal means based on the power of the mechanical sound calculated by the power calculation means.

  According to such a configuration, noise removal can be performed using the noise spectrum only when a mechanical sound is actually generated at the time of shooting.

  According to a seventh aspect of the present invention, in the imaging apparatus according to the sixth aspect, the control means is configured to reduce noise generated by the noise removal means when the power of the mechanical sound calculated by the power calculation means is smaller than a predetermined value. A removal operation is prohibited, and a noise removal operation by the noise removal unit is permitted when the power of the mechanical sound is equal to or higher than a predetermined value.

  According to such a configuration, the noise removal operation is prohibited when the mechanical sound power is lower than the predetermined value, so that the noise removal is performed when no mechanical sound is generated at the time of shooting, and the spectrum is excessively drawn. It is possible to prevent the audio signal from being distorted by such as

  According to an eighth aspect of the present invention, in the imaging apparatus according to any one of the first to fifth aspects, an amplification unit that amplifies an audio signal obtained from the first input unit; An amplification factor adjustment unit that calculates power and adjusts the amplification factor of the amplification unit based on the power value, and the first noise spectrum according to the amplification factor of the amplification unit adjusted by the amplification factor adjustment unit Coefficient variable means for changing the value of the coefficient to be multiplied by is provided.

  According to such a configuration, when the recording level is controlled by adjusting the amplification factor of the input sound, the value of the coefficient multiplied by the first noise spectrum is changed according to the amplification factor, thereby subtracting the spectrum. It can prevent over and over.

  According to a ninth aspect of the present invention, there is provided an audio recording method for use in an image pickup apparatus provided with a second input unit in the vicinity of a mechanical sound source, in addition to the first input unit for inputting an audio signal. A method of storing a plurality of patterns of mechanical sound previously collected through the first input unit as a first noise spectrum in a first storage unit, and collecting in advance through the second input unit A plurality of patterns stored in the second storage unit in association with each pattern of the first noise spectrum as a second noise spectrum, and a mechanism obtained from the second input unit during photographing Based on the sound spectrum, a second noise spectrum corresponding to the characteristic of the mechanical sound is selected from the second storage unit, and the second noise spectrum is selected from the first storage unit. Selecting a first noise spectrum corresponding to the tone, and subtracting a signal obtained by multiplying the selected first noise spectrum by a predetermined coefficient from the spectrum of the audio signal obtained from the first input unit. Removing the noise component in step, inversely transforming the speech spectrum after the noise removal into the original speech signal, recording the speech signal obtained by the inverse transforming means together with the captured image in a predetermined memory, and It is provided with.

  According to such a sound recording method, the same effects as those of the first aspect of the invention can be achieved by executing the processing according to the steps.

  The program according to claim 10 of the present invention is executed by a computer mounted on an imaging device provided with a second input unit in the vicinity of the mechanical sound source, in addition to the first input unit that inputs an audio signal. A program for storing a plurality of patterns of mechanical sounds previously collected through the first input unit in the computer as a first noise spectrum in the first storage unit; The mechanism sound spectrum obtained in advance through the input unit is stored as a second noise spectrum in association with each pattern of the first noise spectrum, and a plurality of patterns are stored in the second storage unit. Based on the spectrum of the mechanical sound obtained from the input unit, the second noise spectrum corresponding to the characteristic of the mechanical sound is selected from the second storage unit, A function of selecting a first noise spectrum corresponding to the second noise spectrum from the first storage unit, and the first noise selected from the spectrum of the audio signal obtained from the first input unit A function for removing noise components by subtracting a signal obtained by multiplying the spectrum by a predetermined coefficient, a function for inversely transforming the speech spectrum after noise removal into the original speech signal, and a speech obtained by the inverse transform means And a function of recording a signal in a predetermined memory together with a photographed image.

  Therefore, when the computer executes the program for realizing each function, the same effects as those of the first aspect of the invention can be achieved.

  According to the present invention, a noise spectrum having the same input characteristics can be subtracted from the spectrum of an audio signal when shooting a moving image with audio. As a result, the mechanical sound included in the audio signal at the time of shooting can be appropriately removed as a noise component and recorded together with the shot image with high quality.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
1A and 1B are diagrams showing an external configuration when a digital camera is taken as an example of the imaging apparatus of the present invention. FIG. 1A mainly shows a front configuration, and FIG. 1B mainly shows a rear configuration. It is a perspective view.

  The digital camera 1 has a photographing lens 3, a self-timer lamp 4, an optical finder window 5, a strobe light emitting unit 6, a microphone unit 7 and the like on the front surface of a substantially rectangular thin plate-like body 2 on the upper surface (for the user). On the right end side, a power key 8 and a shutter key 9 are provided.

  The power key 8 is a key operated every time the power is turned on / off, and the shutter key 9 is a key for instructing a photographing timing at the time of photographing.

  Also, on the back of the digital camera 1, a shooting mode (R) key 10, a playback mode (P) key 11, an optical viewfinder 12, a speaker unit 13, a macro key 14, a strobe key 15, a menu (MENU) key 16, a ring A key 17, a set (SET) key 18, a display unit 19, and the like are provided.

  The shooting mode key 10 is operated automatically from the power-off state to automatically turn on the power and shift to the still image shooting mode. On the other hand, by repeatedly operating from the power-on state, the still image mode and the moving image mode are switched. Set cyclically. The still image mode is a mode for photographing a still image. The moving image mode is a mode for shooting a moving image. In particular, in this embodiment, it is assumed that moving image shooting with sound is possible.

  The shutter key 9 is commonly used for these photographing modes. That is, in the still image mode, a still image is taken at the timing when the shutter key 9 is pressed. In the moving image mode, shooting of a moving image is started at a timing when the shutter key 9 is pressed, and shooting of the moving image is ended when the shutter key 9 is pressed again.

  When the playback mode key 11 is operated from the power-off state, the playback mode key 11 is automatically turned on to enter the playback mode.

  The macro key 14 is operated when switching between normal shooting and macro shooting in the still image shooting mode. The strobe key 15 is operated when switching the light emission mode of the strobe light emitting unit 6. The menu key 16 is operated when selecting various menu items. The ring key 17 is integrally formed with item selection keys in the up, down, left, and right directions, and the set key 18 located in the center of the ring key 17 sets the item selected at that time. To operate.

  The display unit 19 is composed of a color liquid crystal panel with a backlight, and displays a through image on the monitor as an electronic viewfinder in the photographing mode, and reproduces and displays the selected image and the like in the reproduction mode.

  Further, the digital camera 1 is provided with an optical zoom function, and the enlargement ratio of the image can be changed by physically changing the focal length by operating the zoom keys 20a and 20b. Of the zoom keys 20a and 20b, one zoom key 20a is for the telephoto end and is used when the zoom magnification is changed to the telephoto side. The other zoom key 20b is for the wide end and is used when the zoom magnification is changed to the wide angle side.

  Although not shown, the digital camera 1 has a memory card slot for attaching / detaching a memory card used as a recording medium, a serial interface connector for connecting to an external personal computer, etc., for example, USB (Universal). Serial Bus) connector and the like are provided.

  FIG. 2 is a block diagram showing an electronic circuit configuration of the digital camera 1.

  The digital camera 1 is provided with a lens optical system 22 including a focus lens and a zoom lens (not shown) constituting the photographing lens 3 so as to be movable within a predetermined range in the optical axis direction. The lens optical system 22 is moved by a motor 21 that is rotationally driven by a motor drive unit 21a.

  The motor 21 includes a plurality of different motors such as a zoom magnification adjustment motor (zoom motor) and a focus adjustment motor (focus motor), and a motor driving unit 21a corresponding to each of them is provided. And

  A CCD (charge coupled device) 23 that is an image pickup device is disposed behind the optical axis of the motor 21. The CCD 23 receives light from each part of the subject input through the photographing lens 3 and outputs an electrical signal corresponding to the intensity of the light.

  In the recording mode, which is the basic mode, the CCD 23 is scanned and driven by a timing generator (TG) 24 and a driver 25, and outputs a photoelectric conversion output corresponding to a light image formed at regular intervals for one screen. The photoelectric conversion output of the CCD 23 is appropriately gain-adjusted for each primary color component of RGB in the state of an analog value signal, sampled and held by the sample hold circuit 26, and converted into digital data by the A / D converter 27. The

  Then, the image processing circuit 28 performs image processing including pixel interpolation processing and γ correction processing to generate a digital luminance signal Y and color difference signals U and V (Cb, Cr), and DMA (Direct Memory Access). ) Output to the controller 29.

  The DMA controller 29 once uses the luminance signal Y and the color difference signals U and V output from the image processing circuit 28 by using the composite synchronization signal, the memory write enable signal, and the clock signal from the image processing circuit 28 once. And the DMA transfer to the DRAM 31 used as the buffer memory via the DRAM interface (I / F) 30.

  The control unit 32 controls the entire digital camera 1 and is constituted by a microcomputer including a CPU, a ROM storing an operation program executed by the CPU, a RAM used as a work memory, and the like. . After the DMA transfer of the luminance and color difference signals to the DRAM 31, the control unit 32 reads the luminance and color difference signals from the DRAM 31 via the DRAM interface 30 and writes them to the VRAM 34 via the VRAM controller 33.

  The digital video encoder 35 periodically reads the luminance and color difference signals from the VRAM 34 via the VRAM controller 33, generates a video signal based on these data, and outputs the video signal to the display unit 19.

  As described above, the display unit 19 functions as a monitor display unit (electronic finder) at the time of shooting. By performing display based on the video signal from the digital video encoder 35, the display unit 19 captures from the VRAM controller 33 at that time. An image based on the image information is displayed in real time.

  As described above, when the image at that time is displayed in real time as the monitor image on the display unit 19, for example, when the shutter key 9 is pressed at a timing at which still image shooting is desired, a trigger signal is generated.

  In response to the trigger signal, the control unit 32 immediately stops the path from the CCD 23 to the DRAM 31 immediately after the DMA transfer of the luminance and color difference signals for one screen captured from the CCD 23 to the DRAM 31 is completed. , Transition to the record storage state.

  In this recording and storage state, the control unit 32 outputs the luminance and color difference signals for one frame written in the DRAM 31 to 8 pixels × 8 pixels for each of Y, Cb, and Cr components via the DRAM interface 30. The data is read out in units called basic blocks, written in a JPEG (Joint Photographic Coding Experts Group) circuit 37, and this JPEG circuit 37 uses an ADCT (Adaptive Discrete Cosine Transform), which is an entropy coding system. Data compression is performed by processing such as conversion.

  The obtained code data is read out from the JPEG circuit 37 as a data file of one image and written in the recording memory 38. The memory 38 includes a memory card that is detachably mounted as a recording medium in addition to an internal memory such as a flash memory built in the main body in advance. With the compression processing of the luminance and color difference signals for one frame and the completion of writing all the compressed data to the memory 38, the control unit 32 activates the path from the CCD 23 to the DRAM 31 again.

  The control unit 32 is further connected with an audio processing unit 39, a USB interface (I / F) 40, and a strobe driving unit 41.

  The sound processing unit 39 includes a sound source circuit such as a PCM sound source, digitizes the sound signal input from the microphone unit (MIC) 7 when recording sound, and performs a predetermined data file format such as MP3 (MPEG-1 audio layer). 3) Data compression is performed in accordance with the standard to create an audio data file and send it to the memory 38. On the other hand, at the time of audio reproduction, the audio data file read from the memory 38 is uncompressed and converted into an analog signal. The sound is output through a speaker unit (SP) 13 provided on the back side.

  Note that, as will be described later, a reference microphone 7 a installed near the motor 21 is connected to the audio processing unit 39 separately from the microphone unit (MIC) 7. This reference microphone 7a is mainly used as an input means for inputting motor sound for noise removal.

  The USB interface 40 performs communication control when image data and other information are transmitted / received to / from another information terminal device such as a personal computer connected by wire via a USB connector. The strobe drive unit 41 charges a strobe capacitor (not shown) at the time of shooting, and then drives the strobe light emitting unit 6 to flash based on control from the control unit 32.

  In addition to the shutter key 9 described above, the key input unit 36 includes a power key 8, a shooting mode key 10, a playback mode key 11, a macro key 14, a strobe key 15, a menu key 16, a ring key 17, and a set key. 18, zoom keys 20 a and 20 b and the like, and signals accompanying these key operations are sent directly to the control unit 32.

  Further, when shooting a moving image instead of a still image, when the shutter key 9 is pressed, the above-described JPEG circuit 37 compresses the captured moving image using a technique such as motion-JPEG (Joint Photographic Experts Group). To the memory 38. In this case, in the case of moving image shooting with audio, the audio signal input from the microphone unit (MIC) 7 during the shooting is recorded in the memory 38 together with the moving image data. When the shutter key 9 is operated again, the recording of the moving image data is finished.

  On the other hand, in the playback mode which is the basic mode, the control unit 32 selectively reads out the image data recorded in the memory 38 and is compressed by a procedure completely opposite to the procedure of data compression in the recording mode by the JPEG circuit 37. Decompress image data. The decompressed image data is held in the DRAM 31 via the DRAM interface 30, and then the content held in the DRAM 31 is stored in the VRAM 34 via the VRAM controller 33. The image data is periodically read out from the VRAM 34. A video signal is generated and reproduced and output by the display unit 19.

  If the selected image data is not a still image but a moving image, the multiple frames of still image data that make up the moving image data are played back and displayed sequentially in chronological order, and all the still image data is played back. For example, the top still image data is displayed until the next playback instruction is given. At this time, if the moving image data includes audio data, the audio data is output through the speaker unit (SP) 13.

  Next, an audio recording apparatus having a noise removal function used in the digital camera 1 will be described.

  FIG. 3 is a block diagram showing a configuration of an audio recording apparatus having a noise removal function used in the digital camera 1 according to the first embodiment of the present invention.

  This sound recording apparatus is mainly used for moving image shooting with sound of the digital camera 1 and has a function of removing mechanical sounds such as zoom sound and focus sound mixed in sound signals during the shooting as noise. .

  In the first embodiment, the voice recording apparatus includes a motor 21, a motor drive unit 21a, a control unit 32, a key input unit 36, a voice input unit 51, a frame division unit 52, a Fourier transform unit 53, and a first spectrum storage. Unit 54, subtracting unit 55, spectrum switching unit 56, inverse Fourier transform unit 57, and waveform synthesis unit 58. Furthermore, as another system, a reference input unit 61, a frame division unit 62, a Fourier transform unit 63, a second spectrum storage unit 64, a similarity calculation unit 65, and a spectrum selection unit 66 are provided.

  Of the components, 51 to 58 and 61 to 66 are included in the audio processing unit 39 of the digital camera 1 shown in FIG.

  The motor 21 is a motor for moving the lens optical system 22 such as a zoom lens in the optical axis direction, and the motor drive unit 21a is a drive mechanism for driving the motor 21 to rotate.

  The control unit 32 receives an operation signal from the zoom keys 20a and 20b included in the key input unit 36 and outputs a motor drive control signal to the motor drive unit 21a. Here, the drive of the motor 21 is performed during video recording with sound. A function of controlling the spectrum switching unit 56 based on the timing is provided.

  On the other hand, the audio input unit 51 includes a microphone unit 7 installed on the device housing of the digital camera 1 shown in FIG. 1 as a main microphone, and a frame dividing unit 52 using an audio signal input through the main microphone as a main signal. To give. In this case, for example, when a zoom operation is performed during moving image recording with audio, a motor sound (zoom sound) generated along with the zoom operation enters through the audio input unit 51 together with the audio signal.

  The frame dividing unit 52 divides the audio signal (main signal) input by the audio input unit 51 in units of frames for a predetermined time. The Fourier transform unit 53 performs Fourier transform on the audio signal divided by the frame unit by the frame division unit 52 and converts it into a spectrum signal (Ia) indicating the power for each frequency.

  In the first spectrum storage unit 54, a plurality of patterns of motor sound spectrum signals collected in advance through the voice input unit 51 (main microphone) are stored as first noise spectrum signals (see FIG. 4).

  Based on the input speech spectrum signal (Ia) obtained by the Fourier transform unit 53 and the noise spectrum signal (Xv) selected from the first spectrum storage unit 54, the subtracting unit 55 performs SS (spectral subtraction). The noise removal process by the method is performed.

  Specifically, a process of removing a noise component contained in the audio signal is performed by subtracting a signal obtained by multiplying the noise spectrum signal (Xv) by a predetermined subtract coefficient α from the input audio spectrum signal (Ia). The spectrum switching unit 56 selects the input speech spectrum signal (Ia) obtained by the Fourier transform unit 53 and the speech spectrum signal (Ib) after noise removal obtained by the subtracting unit 55 from the control unit 32. And are given to the inverse Fourier transform unit 57.

  The inverse Fourier transform unit 57 performs inverse Fourier transform on the input speech spectrum signal (Ia) input through the spectrum switching unit 56 or the speech spectrum signal (Ib) after noise removal, and returns the original speech signal for each frame unit.

  The waveform synthesizing unit 58 synthesizes the audio signal for each frame obtained by the inverse Fourier transform unit 57 to restore the audio signal continuous in time. This audio signal is used as an audio signal for final recording, and is recorded in the memory 38 shown in FIG. 2 together with moving image data obtained from the imaging system of the digital camera 1.

  Further, the reference input unit 61 mainly includes a reference microphone 7a for collecting motor sound, and gives a signal of only motor sound input through the reference microphone 7a to the frame dividing unit 52 as a reference signal. The reference microphone 7a is installed in the vicinity of the motor 21 in the device casing separately from the microphone unit (MIC) 7 as the main microphone, and inputs only the motor sound generated when the motor is driven.

  The frame dividing unit 62 divides the motor sound only signal (reference signal) input by the reference input unit 61 in units of frames for a predetermined time. The Fourier transform unit 63 performs a Fourier transform on the motor sound signal divided by the frame unit by the frame division unit 62 and converts the signal into a spectrum signal (Rv) indicating the power for each frequency.

  The second spectrum storage unit 64 stores a plurality of patterns of motor sound spectrum signals collected in advance through a reference input unit 61 (reference microphone) as second noise spectrum signals (see FIG. 5).

  The difference between the first noise spectrum signal stored in the first spectrum storage unit 54 and the second noise spectrum signal stored in the second spectrum storage unit 64 is that the first noise spectrum signal is By having the input characteristic of the motor sound obtained through the voice input unit 51, that is, the main microphone (microphone unit 7), the second noise spectrum has the input characteristic of the motor sound obtained through the reference input unit 61, that is, the reference microphone 7a. is there.

  The similarity calculation unit 65 is similar to the input motor sound spectrum signal (Rv) obtained through the reference input unit 61 and each pattern of the second noise spectrum signal stored in the second spectrum storage unit 64 in advance. Calculate the degree.

  The spectrum selection unit 66 selects the second noise spectrum signal having the highest similarity in the second spectrum storage unit 64 based on the similarity calculation result by the similarity calculation unit 65, and also stores the first spectrum storage. A first noise spectrum signal corresponding to the noise spectrum signal is selected from the unit 54. The noise spectrum signal selected at this time is given to the subtractor 55 as an optimum noise spectrum signal (Xv) having the same input characteristics as the voice signal, and is used for noise removal processing by the SS method.

  Next, the operation of the first embodiment will be described.

  Now, assume that, for example, the user operates the zoom keys 20a and 20b included in the key input unit 36 while shooting a moving image with sound.

  When the control unit 32 that controls the operation of the entire digital camera inputs zoom operation signals of the zoom keys 20a and 20b included in the key input unit 36, it sends a drive start signal to the motor drive unit 21a. The motor drive unit 21a receives the drive start signal and rotationally drives the motor 21. As the motor 21 rotates, a zoom lens (not shown) included in the lens optical system 22 shown in FIG. 2 moves on the optical axis and the zoom magnification changes.

  When the user finishes the zoom operation, the control unit 32 sends a drive stop signal to the motor drive unit 21a. Thereby, the rotational drive of the motor 21 is stopped and the zoom operation is finished.

  Here, during shooting of a moving image with sound, the sound input function by the main microphone (microphone unit 7) is always ON. For this reason, there is a problem that the motor sound generated by the zoom operation is mixed as noise in the input voice. In order to remove such motor noise from the audio signal and record it, the following processing is performed.

  That is, first, a spectrum signal of a motor sound (mechanism sound) to be subjected to noise removal is collected in advance and stored in the first spectrum storage unit 54 and the second spectrum storage unit 64. In the following, a motor sound generated during zoom operation, that is, a zoom sound will be described as a noise removal target.

  The zoom sound is collected by performing a zoom operation in a silent state, and only the zoom sound generated at that time is transmitted from the audio input unit 51 as the main microphone (microphone unit 7) and the reference input unit 61 as the reference microphone 7a. Do this by typing.

  In this case, the zoom sound has different characteristics depending on the drive timing of the motor 21 at the drive start point, the drive intermediate point, and the drive stop point. Also, the zoom sound characteristics differ between when the motor 21 is rotated in the wide-angle direction and when it is driven in the telephoto direction.

  Therefore, at least the zoom sound “ZO1” at the drive start point when the motor 21 is rotationally driven in the wide-angle direction, the zoom sound “ZO2” at the drive intermediate point, the zoom sound “ZO3” at the drive stop point, and the motor A total of six patterns are collected: zoom sound “ZI1” at the drive start point, zoom sound “ZI2” at the drive intermediate point, and zoom sound “ZI3” at the drive stop point when the 21 is rotated in the telephoto direction. It shall be.

  The first spectrum storage unit 54 stores a zoom sound only signal from the audio input unit 51, cuts out a frame section of about several tens of ms by the frame division unit 52, and converts this into a spectrum signal by the Fourier transform unit 53. Convert. The spectrum signal is calculated for the three periods of the drive start point, drive intermediate point, and drive stop point described above for the case where the motor 21 is rotated in the wide angle direction and the case where it is rotated in the telephoto direction. The average value of the level is obtained for each period.

  In this way, the patterns of the first noise spectrum signal obtained when the motor 21 is rotationally driven in the wide-angle direction and when it is rotationally driven in the telephoto direction are represented as MZO1, MZO, MZO3, MZI1, MZI2, and MZOI3. As shown in FIG. 4, it is stored in the first spectrum storage unit 41.

  Similarly, the second spectrum storage unit 64 stores a zoom sound-only signal from the reference input unit 61, cuts it out into a frame interval of about several tens of ms by the frame division unit 62, and converts this into a Fourier transform unit 53. Is converted into a spectrum signal. The spectrum signal is calculated for the three periods of the drive start point, drive intermediate point, and drive stop point described above for the case where the motor 21 is rotated in the wide angle direction and the case where it is rotated in the telephoto direction. The average value of the level is obtained for each period.

  In this way, the patterns of the second noise spectrum signals obtained when the motor 21 is rotationally driven in the wide-angle direction and when it is rotationally driven in the telephoto direction are RZO1, RZO2, RZO3, RZI1, RZI2, and RZI3. As shown in FIG. 5, it is stored in the second spectrum storage unit 64.

  In this case, in the control unit 32, each pattern of MZO1 to MZO3 and MZI1 to MZOI3 stored in the first spectrum storage unit 54 and RZO1 to RZO3 and RZI1 to RZI3 stored in the second spectrum storage unit 64 are stored. Each pattern is managed in association with a management table (not shown).

  Here, when the zoom operation is not performed, the control unit 32 switches and controls the spectrum switching unit 56 so as to select the input audio spectrum signal (Ia) obtained from the Fourier transform unit 53. As a result, the input audio signal is output as it is through the inverse Fourier transform unit 57 and the waveform synthesis unit 58.

  On the other hand, when the control unit 32 determines that the zoom operation is started based on the zoom operation signal from the key input unit 36, the noise removal obtained from the subtracting unit 55 simultaneously with the start of driving of the motor 21 (here, the zoom motor). The spectrum switching unit 56 is controlled so as to select a later audio spectrum signal (Ib).

  As described above, during the zoom operation, in addition to the audio signal, the audio sound generated at that time is input to the audio input unit 51 through the main microphone (microphone unit 7). Therefore, the Fourier transform unit 53 outputs an input sound spectrum signal (Ia) in which the spectrum of the input sound and the spectrum of the motor sound are mixed.

  On the other hand, only the motor sound is basically input to the reference input unit 61 because the reference microphone 7a is installed in the device casing. Therefore, the output of the Fourier transform unit 63 is a spectrum signal (Rv) of only motor sound.

  Here, in the similarity calculation unit 65, the spectrum signal (Rv) of the motor sound obtained by the Fourier transform unit 63 and each pattern RZO <b> 1 of the second noise spectrum stored in advance in the second spectrum storage unit 64. Similarities with RZO3, RZI1-RZI3 are calculated.

  As an example of the similarity, the absolute value of the input motor sound spectrum signal (Rv) is | RC (ω) |, and the absolute value of the second noise spectrum signal stored in the second spectrum storage unit 64 is | RM ( ω) |, where the root mean square error is MSE, the reciprocal of the MSE is the similarity.

MSE = Σ [{| RC (ω) | − | RM (ω) |}
* {| RC (ω) |-| RM (ω) |}] (1)
Σ is the integrated value from angular frequency ω = 0 to π
π = 1/2 of sampling frequency
Similarity = 1 / MSE (2)
According to the above formulas (1) and (2), if RC (ω) and RM (ω) are similar, the value of MSE decreases and the degree of similarity increases. Conversely, if RC (ω) and RM (ω) are not similar, MSE increases and the similarity decreases.

  Here, for example, assuming that the pattern having the highest similarity among RZO1 to RZO3, RZO1 to RZOI3 is RZO1, the spectrum selecting unit 66 selects the MZO1 corresponding to RZO1 from the first spectrum storage unit 54 as the optimum noise spectrum. Select as signal (Xv). RZO1 and MZO1 have motor characteristics obtained in advance at the drive start point when the motor 21 is rotationally driven in the wide-angle direction.

  That is, for example, when a zoom operation that rotates the motor 21 in the wide-angle direction is performed, RZO1 → MZO1 during the period until the driving start of the motor 21 reaches steady rotation, and RZO2 → MZO2 during the steady rotation period, In the period from when the drive is stopped until the rotation of the motor 21 is actually stopped, the first noise spectrum signal corresponding to the second noise spectrum signal is present from the first spectrum storage unit 54, such as RZO3 → MZO3. The optimum noise spectrum corresponding to the motor characteristics is sequentially selected and provided to the subtractor 55.

  The subtractor 55 performs noise removal processing by the SS method based on the input speech spectrum signal (Ia) obtained by the Fourier transform unit 53 and the noise spectrum signal (Xv) selected from the first spectrum storage unit 54. Do.

  This noise removal processing will be described in detail with reference to FIG.

  FIG. 6 is a diagram for explaining noise removal processing using the SS method (spectral subtraction method). 6A shows the waveform data of the input speech, and FIG. 6B shows the speech spectrum signal (Ia) obtained by Fourier transforming the input speech in units of frames.

  FIG. 6C shows the spectrum of the motor sound collected for noise removal, that is, the noise spectrum signal (Xv), and FIG. 8D shows the signal obtained by multiplying the noise spectrum signal (Xv) by a predetermined subtract coefficient α. is there. FIG. 4E shows a spectrum signal obtained by subtracting the noise spectrum signal (Xv) after coefficient multiplication from the input voice spectrum signal (Ia), that is, the voice spectrum signal (Ib) after noise removal. Fig. 8 (f) shows an audio signal obtained by inverse Fourier transform of the audio spectrum signal (Ib) after the noise removal, and Fig. 10 (g) shows an audio signal divided in units of frames synthesized in time series. It shows a state in which the original sound waveform is restored.

  Assume that an audio signal having a waveform as shown in FIG. 6A is input to the audio input unit 51 (main microphone). In this audio signal, for example, a motor sound generated by a zoom operation, that is, a zoom sound is mixed as noise.

  First, in the frame dividing unit 52, an audio signal is cut out in a frame section of, for example, about 10 ms, and an input audio spectrum signal (Ia) representing the power for each frequency in the Fourier transform unit 53 as shown in FIG. Is generated.

  Here, based on the spectrum signal (Rv) of only the motor sound input from the reference input unit 61 (reference microphone), the second noise spectrum signal having the highest similarity is selected from the second spectrum storage unit 64. Further, the first noise spectrum signal corresponding to the selected noise spectrum signal is selected from the first spectrum storage unit 54 as the optimum noise spectrum signal (Xv) corresponding to the motor characteristics.

  Then, as shown in FIGS. 5C to 5E, the subtracting unit 55 subtracts a signal obtained by multiplying the optimum noise spectrum signal (Xv) by a predetermined subtract coefficient α from the input speech spectrum signal (Ia). Thus, the speech spectrum signal (Ib) after noise removal is obtained.

  The subtract coefficient α is determined in advance according to the level of the first noise spectrum signal stored in the first spectrum storage unit 54, and is usually a value of “1” or more.

  In the control unit 32, during the zoom operation, that is, during the drive period of the motor 21 that is a zoom motor (a period from the start of motor drive to the stop of drive), after noise removal obtained from the subtractor 55 is performed. The spectrum switching unit 56 is controlled to select the audio spectrum signal (Ib).

  As shown in FIG. 6 (f), the speech spectrum signal (Ib) after this noise removal is subjected to inverse Fourier transform by an inverse Fourier transform unit 57. Then, as shown in FIG. 5G, the waveform synthesizing unit 58 synthesizes the audio signal for each frame in time series and restores the original analog waveform signal as the audio signal. This sound signal is recorded in the memory 38 together with image data during moving image shooting as a sound signal after noise removal.

  In the noise removal process as described above, before the input speech is actually divided into frames by the frame dividing unit 52 and subjected to Fourier transform, the speech signal is subjected to a window function such as a “Hanning window”. In addition, when the audio signal after inverse Fourier transform is synthesized for each frame by the waveform synthesis unit 58 in the subsequent stage, the audio signal for each frame is somewhat overlapped in order to prevent a discontinuous waveform at the frame boundary. And then synthesize.

  For example, the analysis point is shifted by 128 samples with a frame length of 256 samples. The Hanning window in this case can be expressed as shown in Equation (3).

w (n) = 0.5−cos {2 * PI * n / (L−1)} (3)
L: number of samples in one frame n = 0, 1,..., L−1
In this way, when the signals are overlapped with a shift of ½ frame, a speech waveform having a constant amplitude and no discontinuity can be obtained.

  As described above, according to the first embodiment of the present invention, after the second noise spectrum signal is selected by the motor sound input from the reference input unit 61 at the time of shooting, the first noise spectrum signal corresponding thereto is selected. Is used for noise removal, a noise spectrum having the same input characteristics can be subtracted from the spectrum of the speech signal. As a result, the mechanical sound included in the audio signal at the time of shooting can be appropriately removed as a noise component and recorded together with the shot image with high quality.

  When calculating the similarity between the input motor sound spectrum signal (Rv) and each pattern of the second noise spectrum stored in the second spectrum storage unit 64, all patterns (here, six patterns) are calculated. Even if the degree of similarity of the pattern) is not calculated, the degree of similarity may be calculated for the wide-angle side spectrums RZO1 to RZO3 as long as the motor 21 rotates in the wide-angle direction, for example. By doing so, the amount of similarity calculation is halved.

  In the first embodiment, motor sounds are collected in advance for three periods of the motor 21 drive start point, drive intermediate point, and drive stop point, and a spectrum signal corresponding to each period is used for noise removal. However, it is also possible to classify the motor periods more finely, store the spectrum signals corresponding to the motor periods, and selectively use them appropriately to remove noise. In this way, appropriate noise removal processing can be performed for each drive period of the motor 21.

  FIG. 7 is a flowchart when the audio recording process in the first embodiment is realized by software. Note that the processing shown in this flowchart is recorded in advance in a recording medium such as a ROM in the form of a program readable by the control unit 32 which is a computer.

  At the time of moving image recording with sound, the control unit 32 repeatedly executes the following processing until the end of moving image shooting is explicitly instructed by operating the shutter key 9, for example (steps S11 to S23).

  That is, first, the control unit 32 generates frame data from an audio signal input as a main signal through the microphone unit 7 that is a main microphone, and applies a window function to the frame data (step S11). Subsequently, an input speech spectrum signal (Ia) indicating power for each frequency is generated by performing Fourier transform on the frame data (step S12).

  Here, the control unit 32 determines whether or not the motor 21 is driven by a zoom operation (step S13). If the motor is not being driven (No in step S13), the control unit 32 performs inverse free conversion processing on the input speech spectrum signal (Ia) to return it to the original speech waveform data (step S20). While being synthesized in units of frames so as to be continuous with the waveform data (step S21), it is recorded in the memory 38 in synchronization with the photographed image (moving image data) (step S22).

On the other hand, when the motor is being driven, that is, when the motor 21 is rotationally driven in the wide-angle direction or the telephoto direction by the zoom operation (Yes in step S13), the control unit 32 transmits the reference signal through the reference microphone 7a. Frame data is generated from the signal of only the motor sound input as, and a window function is applied to the frame data (step S14). Subsequently, an input motor sound spectrum signal (Rv) indicating the power for each frequency is generated by Fourier transforming the frame data (step S15).
Here, the control unit 32 calculates the similarity between the input motor sound spectrum signal (Rv) and each pattern of the second noise spectrum signal stored in the second spectrum storage unit 64 (steps S16 and S17). . Then, the second noise spectrum signal having the highest similarity is selected from the second spectrum storage unit 64, and the first noise spectrum signal corresponding to the noise spectrum signal is selected from the first spectrum storage unit 54. A selection is made from among them (step S18).

  The control unit 32 uses the first noise spectrum signal selected from the first spectrum storage unit 54 as an optimum noise spectrum signal (Xv) having the same input characteristics as the voice signal, and uses the input noise spectrum signal (Ia). A noise component is removed by subtracting a signal obtained by multiplying the noise spectrum signal (Xv) by a predetermined subtract coefficient α (step S19).

  Then, the speech spectrum signal (Ib) after the noise removal is subjected to inverse free conversion processing to return to the original speech waveform data (step S20), and this is synthesized in units of frames so as to be continuous with the previous speech waveform data. (Step S21), and recorded in the memory 38 in synchronization with the photographed image (moving image data) (Step S22).

  Thus, even when the present apparatus is realized by software, the same effect as that of the configuration shown in FIG. 3 can be obtained.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.

  FIG. 8 is a block diagram showing a configuration of an audio recording apparatus having a noise removal function used in the digital camera 1 according to the second embodiment of the present invention. In addition, the same code | symbol is attached | subjected to the part same as the structure of FIG. 3 (1st Embodiment), and the description shall be abbreviate | omitted.

  8 is different from the configuration of FIG. 3 in that a short-time power calculation unit 71 is provided. The short-time power calculation unit 71 calculates the power (short-time power) in units of frames of the motor sound signal input as a reference signal through the reference input unit 61 (reference microphone). In this case, if the signal in the frame is r (i) and the power value is P, it can be expressed by the following equation (4).

P = Σ {r (i) * r (i)} (4)
The control unit 32 determines the generation period of the motor sound based on the power of the motor sound signal calculated by the short-time power calculation unit 71 and controls switching by the spectrum switching unit 56.

  The operation as the second embodiment will be described below.

  For example, assume that the user operates the zoom keys 20 a and 20 b included in the key input unit 36. When the control unit 32 that controls the operation of the entire digital camera inputs zoom operation signals of the zoom keys 20a and 20b included in the key input unit 36, it sends a drive start signal to the motor drive unit 21a. The motor drive unit 21a receives the drive start signal and rotationally drives the motor 21. Along with the rotation of the motor 21, a zoom lens (not shown) included in the lens optical system 22 in FIG. 2 moves on the optical axis to start a zoom operation, and a captured image is enlarged or reduced and recorded in the memory 38. The

  When the user finishes the zoom operation, the control unit 32 sends a drive end signal to the motor drive unit 21a. Thereby, the rotational drive of the motor 21 is stopped and the zoom operation is finished.

  Here, there is a slight delay from when the control unit 32 outputs a drive start signal to the motor 21 until the motor 21 actually starts to rotate. Therefore, if noise removal processing (subtract processing) is started at the same time as the drive start signal is output, the input speech spectrum is input to the subtractor 55 even though the motor sound (here, zoom sound) has not yet been generated. The spectrum for the motor sound is subtracted from the signal (Ia), and the audio signal output from the waveform synthesis unit 58 may be distorted.

  In order to prevent such a problem, in the second embodiment, paying attention to the fact that the input signal from the reference input unit 61 (reference microphone) is a signal of only substantially motor sound, the short-time power calculation unit 71. Thus, the short-time power of the motor sound signal obtained from the reference input unit 61 is calculated. Then, the calculated power of the motor sound signal is compared with a preset reference value, and when the power of the motor sound signal is smaller than the reference value, the input speech spectrum signal output from the Fourier transform unit 53 The spectrum switching unit 56 is controlled to select (Ia).

  On the other hand, if the power of the motor sound signal is equal to or higher than the reference value, it is determined that the motor sound is actually generated, and the speech spectrum signal (Ib) after noise removal output from the subtracting unit 55 is selected. The spectrum switching unit 56 is controlled.

  The same applies when stopping the motor 21, and after confirming that the rotation of the motor 21 has actually stopped by comparing the power of the motor sound signal obtained from the short-time power calculation unit 71 with the reference value. The spectrum switching unit 56 is controlled to select the input speech spectrum signal (Ia).

  As a result, even if there is a discrepancy between the motor drive operation and the timing at which the motor sound actually occurs, spectrum subtraction, that is, noise removal processing can be performed in accordance with the motor sound generation timing. No audio signal can be obtained.

  Note that the configuration of FIG. 8 can also be realized in software as the control unit 32 performs a series of processes according to a program in the same manner as described above.

(Third embodiment)
Next, a third embodiment of the present invention will be described.
In the third embodiment, an audio recording apparatus provided with an automatic recording level control system (ALS) is assumed.

  FIG. 9 is a block diagram showing a configuration of an audio recording apparatus having a noise removal function used in the digital camera 1 according to the third embodiment of the present invention. In addition, the same code | symbol is attached | subjected to the part same as the structure of FIG. 3 (1st Embodiment), and the description shall be abbreviate | omitted.

  9 is different from the configuration of FIG. 3 in that a recording level control unit 72, an amplifier 73, and a multiplication unit 74 are provided. The recording level control unit 72 performs control for keeping the level of the input sound substantially constant. That is, the recording level control unit 72 calculates the power of the audio signal input from the audio input unit 51 (main microphone), and if the power value is smaller than a predetermined value, the recording level (amplification factor) given to the amplifier 73 If the power value is larger than the predetermined value, the recording level (amplification factor) given to the amplifier 73 is controlled to be lowered.

  The amplifier 73 performs amplification adjustment (gain adjustment) on the input sound in accordance with an instruction from the recording level control unit 72 and supplies the input sound to the frame division unit 52. Further, the multiplication unit 74 changes the value of the subtract coefficient α to be multiplied by the noise spectrum signal (Xv) in accordance with the instruction from the recording level control unit 72.

  The operation as the third embodiment will be described below.

  Similarly to the first embodiment, the first spectrum storage unit 54 and the second spectrum storage unit 64 are silent in advance through the voice input unit 51 (main microphone) and the reference input unit 61 (reference microphone), respectively. A spectrum signal of motor sound (here, zoom sound) collected in the state is stored as a noise spectrum signal. The recording level in this case is fixed to the reference value “1”.

  Here, at the time of video recording with sound, the power (volume) of the audio signal input as the main signal from the audio input unit 51 (main microphone) is calculated by the recording level control unit 72, and the recording level based on the power value is calculated. Is determined.

  Assuming that the recording level at this time is “k”, the audio signal input as the main signal from the audio input unit 51 (main microphone) is multiplied by k by the amplifier 73 and then is several tens of ms by the frame dividing unit 52. The signal is divided into frames, and then converted into a spectrum signal (Ia) by a Fourier transform unit 53. The spectrum of the motor sound included as noise in the input sound spectrum signal (Ia) is k times that when the recording level is “1”.

  On the other hand, since the noise spectrum signal (Xv) stored in the first spectrum storage unit 54 is a spectrum of a motor sound collected at a normal recording level (k = 1), the noise removal processing of the subtractor unit 55 is performed as it is. When k is greater than 1, spectral leftover occurs, and as a result, the zoom sound remains in the audio signal output from the waveform synthesizer 58. On the other hand, when k is smaller than 1, an excessive spectrum is subtracted from the input voice spectrum signal (Ia). For this reason, the noise component contained in the input speech is removed, but the speech signal itself is distorted due to excessive spectrum drawing.

  In order to prevent this, the noise spectrum signal (Xv) selected from the first spectrum storage unit 54 is not supplied to the subtracting unit 55 as it is, but is supplied after being multiplied by k like the input speech. . Specifically, the subtract coefficient α to be multiplied by the noise spectrum signal (Xv) is changed by the multiplier 74 in accordance with the amplification factor of the amplifier 73. Thereby, the noise spectrum signal (Xv) multiplied by k together with the input voice signal is given to the subtractor 55, and a voice signal without distortion can be obtained from the waveform synthesizer 58.

  Note that the configuration of FIG. 6 can also be realized in software as the control unit 32 performs a series of processing according to a program in the same manner as described above.

  In each of the embodiments described above, the zoom sound has been described as a noise removal target. However, the present invention is not limited to the zoom sound, and the same applies to, for example, a focus sound and further a shutter sound. Can be applied to the case of removing from the input voice.

  Further, in each of the above embodiments, a digital camera capable of shooting a moving image with sound has been described as an example. However, the present invention is not limited to a digital camera, and for example, a captured image can be recorded together with a sound signal such as a mobile phone with a camera. Any electronic device having a function can be applied to all of them.

  In short, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying constituent elements without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the respective embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  In addition, the method described in the above-described embodiment is a program that can be executed by a computer, such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD-ROM, etc.), a semiconductor memory, etc. The program can be written on a medium and applied to various apparatuses, or the program itself can be transmitted through a transmission medium such as a network and applied to various apparatuses. A computer that implements this apparatus reads a program recorded on a recording medium or a program provided via a transmission medium, and performs the above-described processing by controlling operations by this program.

1A and 1B are diagrams showing an external configuration when a digital camera is taken as an example of the imaging apparatus of the present invention. FIG. 1A mainly shows a front configuration, and FIG. 1B mainly shows a rear configuration. It is a perspective view. FIG. 2 is a block diagram showing an electronic circuit configuration of the digital camera. FIG. 3 is a block diagram showing the configuration of an audio recording apparatus having a noise removal function used in the digital camera according to the first embodiment of the present invention. FIG. 4 is a diagram showing the configuration of the first spectrum storage unit in the first embodiment. FIG. 5 is a diagram showing a configuration of the second spectrum storage unit in the first embodiment. FIG. 6 is a diagram for explaining noise removal processing using the SS method (spectral subtraction method). FIG. 7 is a flowchart when the audio recording process in the first embodiment is realized by software. FIG. 8 is a block diagram showing the configuration of an audio recording apparatus having a noise removal function used in a digital camera according to the second embodiment of the present invention. FIG. 9 is a block diagram showing the configuration of an audio recording apparatus having a noise removal function used in a digital camera according to the third embodiment of the present invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Digital camera, 2 ... Body, 3 ... Shooting lens, 7 ... Microphone part (main microphone), 7a ... Reference microphone, 9 ... Shutter key, 20a, 20b ... Zoom key, 21 ... Motor, 21a ... Motor drive part, 32 ... Control part 36 ... Key input part 51 ... Audio input part 52 ... Frame division part 53 ... Fourier transform part 54 ... First spectrum storage part 55 ... Subtract part 56 ... Spectrum switching part 57 ... Inverse Fourier transform unit, 58... Waveform synthesis unit, 61... Reference input unit, 62... Frame division unit, 63. 71 ... Short-time power calculation unit, 72 ... Recording level control unit, 73 ... Amplifier, 74 ... Multiplication unit, Ia ... Input speech spectrum signal, Ib ... Speech spectrum after noise removal Torque signal, Xv ... noise spectrum signal, Rv ... input motor noise spectrum signal.

Claims (10)

An imaging device having an audio recording function for removing mechanical sound generated as a result of a shooting operation from an audio signal as noise when recording video with audio,
First input means for inputting an audio signal;
A first storage means for storing a plurality of patterns of the spectrum of the mechanical sound collected in advance through the first input means as a first noise spectrum;
Second input means provided in the vicinity of the mechanical sound source;
A second storage means for storing a plurality of patterns in association with each pattern of the first noise spectrum as a second noise spectrum, the spectrum of the mechanical sound collected in advance through the second input means;
Based on the spectrum of the mechanical sound obtained from the second input means at the time of shooting, a second noise spectrum corresponding to the characteristic of the mechanical sound is selected from the second storage means, and the first storage is performed. Selecting means for selecting a first noise spectrum corresponding to the second noise spectrum from the means;
Noise removal means for removing a noise component by subtracting a signal obtained by multiplying a first noise spectrum selected by the selection means by a predetermined coefficient from the spectrum of the audio signal obtained from the first input means;
Inverse conversion means for inversely converting the speech spectrum after noise removal obtained by the noise removal means into the original voice signal;
An imaging apparatus comprising: a recording unit that records an audio signal obtained by the inverse conversion unit together with a captured image. A degree-of-similarity calculation means for calculating the degree of similarity between the spectrum of the mechanical sound obtained from the second input means at the time of shooting and each pattern of the noise spectrum stored in the second storage means;
The selection means selects the second noise spectrum having the highest similarity from the second storage means based on the calculation result of the similarity calculation means, and selects the second noise spectrum from the first storage means. The imaging device according to claim 1, wherein a first noise spectrum corresponding to the second noise spectrum is selected. The mechanism sound includes a driving sound of a specific motor related to a photographing operation,
The first and second storage means store at least patterns of noise spectra corresponding to at least three periods of when the motor starts driving, during steady rotation, and when driving stops. Item 2. The imaging device according to Item 1. The mechanism sound includes a driving sound of a specific motor related to a photographing operation,
The first and second storage means store noise spectrum patterns when the motor is driven in a predetermined direction and when the motor is driven in a direction opposite to the predetermined direction. The imaging apparatus according to claim 1. The mechanism sound includes a driving sound of a specific motor related to a photographing operation,
In the first and second storage means, the motor is driven in a predetermined direction and when the motor is driven in a direction opposite to the predetermined direction, at least at the start of driving of the motor and during normal rotation The image pickup apparatus according to claim 1, wherein each pattern of the noise spectrum corresponding to three periods when the drive is stopped is stored. Power calculating means for calculating the power of the mechanical sound obtained from the second input means;
6. The control unit according to claim 1, further comprising: a control unit that controls a noise removal operation by the noise removal unit based on the power of the mechanical sound calculated by the power calculation unit. Imaging device.   The control means prohibits a noise removing operation by the noise removing means when the power of the mechanical sound calculated by the power calculating means is smaller than a predetermined value, and the noise when the power of the mechanical sound is equal to or higher than a predetermined value. The imaging apparatus according to claim 6, wherein a noise removing operation by the removing unit is permitted. Amplifying means for amplifying an audio signal obtained from the first input means;
An amplification factor adjusting unit that calculates the power of the audio signal and adjusts the amplification factor of the amplification unit based on the power value;
6. Coefficient variable means for changing a value of a coefficient to be multiplied by the first noise spectrum in accordance with the amplification factor of the amplification means adjusted by the amplification factor adjustment means. The imaging device according to any one of the above. In addition to the first input unit for inputting an audio signal, the audio recording method is used for an imaging apparatus in which a second input unit is provided in the vicinity of the mechanical sound source,
Storing a plurality of patterns in the first storage unit with the spectrum of the mechanical sound collected in advance through the first input unit as a first noise spectrum;
Storing a plurality of patterns in a second storage unit in association with each pattern of the first noise spectrum, as a second noise spectrum, the spectrum of the mechanical sound collected in advance through the second input unit;
Based on the spectrum of the mechanical sound obtained from the second input unit at the time of shooting, a second noise spectrum corresponding to the characteristic of the mechanical sound is selected from the second storage unit, and the first storage is performed. Selecting a first noise spectrum corresponding to the second noise spectrum from the part;
Removing a noise component by subtracting a signal obtained by multiplying the selected first noise spectrum by a predetermined coefficient from the spectrum of an audio signal obtained from the first input unit;
Inversely converting the speech spectrum after the noise removal into the original speech signal;
A voice recording method comprising: recording a voice signal obtained by the inverse conversion unit in a predetermined memory together with a photographed image. A program executed by a computer mounted on an imaging apparatus provided with a second input unit in the vicinity of a mechanism sound generation source separately from a first input unit that inputs an audio signal,
In the computer,
A function of storing a plurality of patterns of a mechanical sound collected in advance through the first input unit as a first noise spectrum in a first storage unit;
A function of storing a plurality of patterns in the second storage unit in association with each pattern of the first noise spectrum as a second noise spectrum, the spectrum of the mechanical sound collected in advance through the second input unit;
Based on the spectrum of the mechanical sound obtained from the second input unit at the time of shooting, a second noise spectrum corresponding to the characteristic of the mechanical sound is selected from the second storage unit, and the first storage is performed. A function of selecting a first noise spectrum corresponding to the second noise spectrum from the unit;
A function of removing a noise component by subtracting a signal obtained by multiplying the selected first noise spectrum by a predetermined coefficient from a spectrum of an audio signal obtained from the first input unit;
A function of inversely converting the speech spectrum after the noise removal into the original speech signal;
A program for realizing a function of recording an audio signal obtained by the inverse conversion means in a predetermined memory together with a photographed image.

Images