JP2011097419A - Photographing device, optical device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a photographing device capable of reducing noise of a sound signal satisfactorily, and to provide an optical device and a program. <P>SOLUTION: The photographing device 100 includes: a photographing portion 110 for photographing an image by an optical system 111; a microphone 230 for converting a sound wave into an electric signal; a detecting portion 191 for detecting at least either a sensor signal detected by a sensor for detecting a photographing state or a control signal for controlling photographing; a determining portion 251 for using at least either the sensor signal or the control signal to determine whether to find a change in a photographing operation; and a reduction processing portion 250 for differing processing for reducing noise of an electric signal between when the determining portion 251 determines that there is a change in the photographing operation and when the determining portion 251 does not determine that a change in the photographing operation. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a photographing apparatus, an optical apparatus, and a program for subtracting noise from a sound signal in a frequency domain.

  Japanese Patent Application Laid-Open No. 2004-151867 discloses a photographing apparatus that reduces noise generated in a sound signal by subtracting noise due to motor driving from the sound signal in the frequency domain.

JP 2006-203376 A

  By the way, noise includes noise that has a large amplitude at the start of driving of the motor and the like, and noise that occurs steadily during the period when the motor is driven, following the noise. The imaging apparatus disclosed in Patent Document 1 executes noise reduction processing without distinguishing between noise that has a large amplitude and that is generated instantaneously and noise that is constantly generated during a period in which the motor is driven. Therefore, there is a problem that noise that has a large amplitude and cannot be instantaneously reduced cannot be reduced.

  It is an object of the present invention to provide an imaging device, an optical device, and a program that can reduce noise in a sound signal.

  The present invention has been made to solve the above-described problem, and is a sensor detected by a photographing unit that photographs an image by an optical system, a microphone that converts sound waves into an electrical signal, and a sensor that detects a photographing state. A signal detection unit that detects at least one of a signal and a control signal for controlling shooting and at least one of the sensor signal and the control signal is used to determine whether or not there is a change in shooting operation And a process for reducing noise of the electrical signal is different between when the determination unit determines that there is a change in the shooting operation and when the determination unit does not determine that there is a change in the shooting operation. An imaging apparatus including a noise processing unit.

  The present invention also provides an optical system that forms an image by an optical system, a microphone that converts sound waves into an electrical signal, a sensor signal that is detected by a sensor that detects a shooting state, and a control signal for controlling shooting. A signal detection unit that detects at least one of the sensor signal, a determination unit that determines whether there is a change in the shooting operation using at least one of the sensor signal and the control signal, and the determination unit that performs the shooting operation. An optical system comprising: a noise processing unit that makes a process for reducing noise of the electrical signal different when it is determined that there is a change and when the determination unit does not determine that there is a change in a shooting operation. Device.

  Further, the present invention provides an input procedure for receiving at least one of an acoustic signal acquired at the time of shooting, a sensor signal detected by a sensor that detects a shooting state, and a control signal for controlling shooting in a computer, Using at least one of the sensor signal and the control signal, a determination procedure for determining whether or not there is a change in the shooting operation, and when the determination unit determines that there is a change in the shooting operation, the determination unit changes to the shooting operation This is a program for executing a noise processing procedure for different processing for reducing noise of the acoustic signal when it is not determined that there has been.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device, optical apparatus, and program which can reduce suitably the noise of a sound signal can be provided.

1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment of the present invention. It is a figure which shows an example of the output of the AF encoder in the 1st Embodiment of this invention, and an example of the impact sound noise and the drive sound noise which generate | occur | produced in the microphone output signal. It is a figure which shows an example of the waveform of the control signal in the 1st Embodiment of this invention. It is a figure which shows an example of the impact sound noise and drive sound noise which generate | occur | produced in the microphone output signal (periodic sound) in the 1st Embodiment of this invention. It is a figure which shows an example which removed the microphone output signal of the area A in the 1st Embodiment of this invention in the time domain. It is a figure which shows an example which interpolated the microphone output signal before the said area to the area A in the 1st Embodiment of this invention. It is a figure which shows an example of the noise reduction process signal from which the impact sound noise and the drive sound noise were reduced from the microphone output signal in the 1st Embodiment of this invention. It is a figure which shows an example of the positional relationship of the time tc which is the end point of the area A in the 1st Embodiment of this invention, and a Hanning window function. It is a figure which shows an example of the positional relationship of the time tc which is the end point of the area A in the 1st Embodiment of this invention, and a Hanning window function. It is a figure which shows an example of the positional relationship of the time tc which is the end point of the area A in the 1st Embodiment of this invention, and a Hanning window function. It is a figure which shows an example of the subtraction coefficient in the 1st Embodiment of this invention. It is a figure which shows an example of the impact sound noise and drive sound noise which generate | occur | produced in the microphone output signal (non-periodic sound) in the 2nd Embodiment of this invention. It is a figure which shows an example of the microphone output signal after removing the area Tc in the time domain in the 2nd Embodiment of this invention. It is a figure which shows an example of the copied microphone output signal in the 2nd Embodiment of this invention. It is a figure which shows an example of the weighting in the cross fade in the 2nd Embodiment of this invention. It is a figure which shows an example of the noise reduction process signal from which the impact sound noise and the drive sound noise were reduced from the microphone output signal in the 2nd Embodiment of this invention.

[First embodiment]
A first embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the imaging apparatus. The imaging device (imaging device) 100 captures (captures) an image by an optical system and stores the obtained image data in the storage medium 200. Further, the imaging apparatus 100 reduces noise from the collected sound and stores the obtained sound signal in the storage medium 200.

  The imaging apparatus 100 includes an imaging unit 110, an image processing unit 140, a display unit 150, a buffer memory 130, an operation unit 180, a storage unit 160, a CPU (Central Processing Unit) 190, a microphone 230, and a sound. A signal processing unit 240, a reduction processing unit 250, and a communication unit 170 are provided.

  The imaging unit 110 includes an optical system 111, an imaging element 119, and an A / D (Analog / Digital) conversion unit 120, and is controlled by the CPU 190 according to set imaging conditions (for example, an aperture value, an exposure value, etc.). The In addition, the imaging unit 110 forms an optical image by the optical system 111 on the imaging element 119 and generates image data based on the optical image converted into a digital signal by the A / D conversion unit 120.

  The optical system 111 includes a zoom lens 114, a focus adjustment lens (hereinafter referred to as “AF (Auto Focus) lens”) 112, a camera shake correction lens (hereinafter referred to as “VR (Vibration Reduction) lens”) 113, and a lens drive. A unit 116, a zoom encoder 115, an AF encoder 117, and an image blur correction unit 118. The optical system 111 guides incident light that has passed through the zoom lens 114, the AF lens 112, and the VR lens 113 to the light receiving surface of the image sensor 119, thereby forming an optical image. Note that the optical system 111 may be attached to and integrated with the imaging apparatus 100, or may be detachably attached to the imaging apparatus 100.

  The zoom encoder 115 detects the driving direction of the zoom lens 114 as a photographing state, and outputs a signal corresponding to the driving direction of the zoom lens 114 to the CPU 190 as a sensor signal. Here, the signal according to the driving direction of the zoom lens 114 means that the zoom lens 114 is stopped in the optical system 111 or is driven in the infinite direction (for driving the zoom lens 114). The motor, cam, etc. are rotated clockwise (CW), for example, or driven in the direction of the closest end (the motor, cam, etc. for driving the zoom lens 114 are, for example, counterclockwise (CCW) ) May be a signal indicating that it is in any state of rotation). That is, the detection of the driving direction of the zoom lens 114 may be detection of the rotation direction of a motor, a cam or the like for driving the zoom lens 114. Note that a motor, a cam, and the like for driving the zoom lens 114 may be provided in the lens driving unit 116. Further, the zoom encoder 115 detects a zoom position representing the position of the zoom lens 114 based on the detected driving direction and driving amount of the zoom lens 114, and outputs it to the CPU 190 as a sensor signal.

  The AF encoder 117 detects the driving direction of the AF lens 112 as a photographing state, and outputs a signal corresponding to the driving direction of the AF lens 112 to the CPU 190 as a sensor signal. Here, the signal corresponding to the driving direction of the AF lens 112 is a state in which the AF lens 112 is stopped in the optical system 111 or is driven in an infinite direction (for driving the AF lens 112). The motor, cam, etc. are rotated clockwise (CW), for example, or are driven in the direction of the closest end (the motor, cam, etc. for driving the AF lens 112 are, for example, counterclockwise (CCW) ) May be a signal indicating that it is in any state of rotation). That is, the detection of the driving direction of the AF lens 112 may be detection of the rotation direction of a motor, a cam, or the like for driving the AF lens 112. Note that a motor, a cam, and the like for driving the AF lens 112 may be provided in the lens driving unit 116. The AF encoder 117 detects a focus position representing the position of the AF lens 112 based on the detected driving direction and driving amount of the AF lens 112, and outputs it to the CPU 190 as a sensor signal.

  The image blur correction unit 118 detects image blur due to the optical system 111 as a photographing state, and outputs it to the CPU 190 as a sensor signal. Then, the image blur correction unit 118 drives the VR lens 113 in a direction to cancel the blur based on a control signal input from a CPU 190 described later. Note that the image blur correction unit 118 may detect the position of the VR lens 113 and output it to the CPU 190 as a sensor signal.

  The lens driving unit 116 controls the positions of the AF lens 112 and the zoom lens 114 based on a control signal input from a CPU 190 described later. The lens driving unit 116 includes an actuator that is driven in photographing. Here, the actuator driven in photographing may be, for example, a motor that drives the AF lens 112, the zoom lens 114, and the like. The actuator may be provided in either the imaging device 100 or an optical system (lens barrel) that is detachable from the imaging device 100.

  The image sensor 119 includes, for example, a photoelectric conversion surface, converts an optical image formed on the light receiving surface into an electrical signal, and outputs the electrical signal to the A / D conversion unit 120.

  Further, the image sensor 119 stores the image data obtained when a shooting instruction is received via the operation unit 180 in the storage medium 200 via the A / D conversion unit 120 as still image data or moving image data. . On the other hand, the image sensor 119 receives continuously acquired image data as through image data in the state in which an imaging instruction is not received via the operation unit 180, and sends it to the CPU 190 and the display unit 150 via the A / D converter 120. Output.

  The A / D converter 120 digitizes the electrical signal converted by the image sensor 119 and outputs image data that is a digital signal to the buffer memory unit 130.

  The operation unit 180 includes, for example, a power switch, a shutter button, a multi-selector (cross key), or other operation keys. The operation unit 180 receives a user operation input when operated by the user, and outputs a signal corresponding to the operation input to the CPU 190. Output to.

  The image processing unit 140 performs image processing on the image data temporarily stored in the buffer memory unit 130 with reference to the image processing conditions stored in the storage unit 160. The image data subjected to the image processing is stored in the storage medium 200 via the communication unit 170. Note that the image processing unit 140 may perform image processing on the image data stored in the storage medium 200.

  The display unit 150 is, for example, a liquid crystal display, and displays image data obtained by the imaging unit 100, an operation screen, and the like.

  The buffer memory unit 130 temporarily stores image data captured by the imaging unit 110. The buffer memory unit 130 temporarily stores a sound signal corresponding to the sound collected by the microphone 230.

  The microphone 230 picks up sound, converts the sound wave into an electric signal (analog signal), and outputs it. That is, the microphone 230 outputs a sound signal (hereinafter referred to as “microphone output signal”) corresponding to the collected sound to the buffer memory unit 130 via the sound signal processing unit 240.

  Then, there is a possibility that at least one of the impact sound noise and the drive sound noise is superimposed on the microphone output signal due to the sound generated by the operation unit. An example of the operation unit referred to here is the zoom lens 114, the VR lens 113, the AF lens 112, or the operation unit 180 described above. By operating in the configuration of the imaging apparatus 100, Or it is the structure which a sound produces (or a sound may produce) by being operated.

  In addition, the operation unit refers to a sound generated by the operation or a sound generated by the operation of the configuration of the imaging apparatus 100 collected by the microphone 230 (or collected). It may be sounded).

  The driving sound noise here is noise that is constantly generated in the microphone output signal during the period (period) in which the operating unit is operating. In addition, the impact sound noise has an amplitude larger than that of the drive sound noise when there is a change in the shooting operation, that is, when the operation unit starts, ends, reverses the operation direction, or the operation unit is operated. It is a large and instantaneous noise generated in the microphone output signal.

  For example, the impact sound noise may be noise generated when a gear (not shown) included in the optical system 111 is engaged. Further, the impact noise noise may be, for example, noise generated when the mechanism of the zoom lens 114 hits an end that limits the movement range of the zoom lens 114.

  The storage unit 160 stores determination conditions referred to when the CPU 190 determines a scene, imaging conditions associated with each scene determined by the scene determination, and the like.

  In addition, the storage unit 160 stores a signal including specific sound information (hereinafter referred to as “estimated noise signal”) and an operation unit in association with each other. Here, the specific sound information may be, for example, drive sound noise waveform information recorded in advance. The estimated noise signal may be a Fourier-transformed signal.

  The sound signal processing unit 240 digitizes the microphone output signal output from the microphone 230 and stores the digitized signal in the buffer memory unit 130.

  The CPU 190 controls the entire imaging apparatus. As an example, the CPU 190 controls the imaging unit 110 according to set imaging conditions (for example, an aperture value, an exposure value, and the like). Further, the CPU 190 determines whether the zoom encoder 115 and the AF encoder 117 are based on the zoom position input from the photographic zoom encoder 115, the focus position input from the AF encoder 117, and the operation input input from the operation unit 180. A control signal for controlling the position is generated. The CPU 190 controls the positions of the zoom encoder 115 and the AF encoder 117 via the lens driving unit 116 based on this control signal.

  The CPU 190 includes a detection unit 191. The detection unit 191 detects the timing at which the operation unit included in the imaging apparatus 100 operates. And the detection part 191 may detect the timing which an operation | movement part operate | moves based on the control signal which operates an operation | movement part. This control signal is a control signal for operating the operating unit with respect to the driving unit for operating the operating unit, or a control signal for driving the driving unit.

For example, the detection unit 191 is generated by the CPU 190 based on a control signal input to the lens driving unit 116 or the image blur correction unit 118 in order to drive the zoom lens 114, the VR lens 113, or the AF lens 112. The timing at which the operating unit operates may be detected based on the control signal.
Further, when the CPU 190 generates a control signal, the detection unit 191 may detect the timing at which the operation unit operates based on processing and commands executed in the CPU 190.
The detection unit 191 may detect the timing at which the operation unit operates based on a signal indicating that the zoom lens 114 or the AF lens 112 is input from the operation unit 180.

  The detection unit 191 may detect the timing at which the operating unit operates based on a signal indicating that the operating unit has been operated.

For example, the detection unit 191 may detect the timing at which the operation unit operates by detecting that the zoom lens 114 or the AF lens 112 is driven based on the output of the zoom encoder 115 or the AF encoder 117. .
The detection unit 191 may detect the timing at which the operation unit operates by detecting that the VR lens 113 is driven based on the output from the image blur correction unit 118.
The detection unit 191 may detect the timing at which the operation unit operates by detecting that the operation unit 180 has been operated based on an input from the operation unit 180.

  Then, the detection unit 191 detects the timing at which the operation unit included in the imaging apparatus 100 operates, and outputs a signal indicating this timing to the determination unit 251 described later of the reduction processing unit 250.

Next, details of the reduction processing unit 250 will be described.
The reduction processing unit 250 includes a determination unit 251. The determination unit 251 determines whether there is a change in the shooting operation based on a signal indicating timing. The reduction processing unit 250 reduces the impact noise and the driving sound noise when the determination unit 251 determines that there is a change in the shooting operation and when the determination unit 251 does not determine that there is a change in the shooting operation. Different processing. For example, the determination unit 251 may determine that there is a change in the shooting operation when it can be determined that an impact sound is generated based on a signal indicating timing. Further, the determination unit 251 may determine that there is no change in the photographing operation when it can be determined that a steady sound is generated based on a signal indicating timing. Further, when the determination unit 251 can determine that the photographing operation is not performed based on the information indicating the timing, or when it can be determined that neither the impact sound nor the steady sound is generated, no noise is generated. You may judge.

 Specifically, the reduction processing unit 250 reduces the impact noise by performing noise reduction processing in the time domain for the impact sound noise as a process that varies depending on the determination. On the other hand, for drive sound noise, the drive sound noise is reduced by executing noise reduction processing in the frequency domain, for example, by spectral subtraction. Then, the reduction processing unit 250 outputs the microphone output signal with reduced noise to the communication unit 170 as a noise subtraction processing signal (sound data).

  The communication unit 170 is connected to a removable storage medium 200 such as a card memory, and writes, reads, or deletes information (image data, noise subtraction processing signal, etc.) to the storage medium 200.

  The storage medium 200 is a storage unit that is detachably connected to the imaging apparatus 100, and stores, for example, image data generated by the imaging unit 100, a noise subtraction processing signal, and the like. Note that the storage medium 200 may be integrated with the imaging device 100.

  Next, as an example of the timing at which the operating unit operates, the relationship between the timing at which the AF lens 112 operates and the impact sound noise and drive sound noise generated in the microphone output signal will be described.

  FIG. 2 shows an example of the output of the AF encoder and an example of the impact sound noise and the drive sound noise generated in the microphone output signal. The horizontal axes in FIGS. 2A and 2B indicate time. The vertical axis in FIG. 2A indicates the driving direction of the AF lens 112 as the output of the AF encoder. That is, the AF lens 112 as the operation unit stops in the optical system 111 and is driven in an infinite direction (a motor, a cam or the like for driving the AF lens 112 is rotated clockwise (CW), for example), or This indicates that it is in any state of driving in the direction of the closest end (a motor, a cam or the like for driving the AF lens 112 rotates counterclockwise (CCW), for example). The vertical axis in FIG. 2B indicates a microphone output signal output from the microphone 230.

  For example, it is assumed that a motor, a cam, and the like for driving the AF lens 112 are changed from a stop to a clockwise rotation (drive start) at time t1 (see FIG. 2A). FIG. 2B shows that, due to this, impact sound noise is generated in the microphone output signal, and subsequently drive sound noise is generated. Then, it is assumed that a motor, a cam, and the like for driving the AF lens 112 continue to rotate (drive) and stop (drive end) at time t2. Thereby, it is shown in FIG. 2B that the impact sound noise is generated at time t2.

  Furthermore, for example, at time t3, it is assumed that the motor, cam, and the like for driving the AF lens 112 change (start driving) from the stop to the clockwise rotation (see FIG. 2A). As a result, at time t3, it is shown in FIG. 2B that impact sound noise is generated in the microphone output signal and subsequently drive noise is generated. Then, it is assumed that the motor, cam, and the like for driving the AF lens 112 continue to rotate (drive), and the rotation direction (drive direction) is reversed at time t4. As a result, it is shown in FIG. 2B that impact noise is generated at time t4. Furthermore, it is assumed that a motor, a cam, and the like for driving the AF lens 112 continue to rotate (drive) and stop (drive end) at time t5. Thereby, it is shown in FIG. 2B that the impact sound noise is generated at time t5.

  As in the example shown in FIG. 2, the impact noise noise and the driving noise noise generated by the AF lens 112 are generated in synchronization with the timing (t1, t2, t3, t4, t5) at which the AF lens 112 operates. The detection unit 191 detects the timing (t1, t2, t3, t4, t5) at which the AF lens 112 operates, and outputs a signal indicating this timing to the determination unit 251. Further, the determination unit 251 determines that the photographing operation has changed at the timing (t1, t2, t3, t4, t5) based on the signal indicating this timing. Similarly, the determination unit 251 determines that drive sound noise (steady sound) is generated at times t1 to t2, t3 to t4, and t4 to t5 based on a signal indicating timing, and the photographing operation does not change. To do. Similarly, based on the signal indicating the timing, the determination unit 251 determines that the drive is stopped at times t2 to t3 (no shooting operation is performed) and noise is not generated.

Next, details of the noise reduction processing will be described.
Hereinafter, in the first embodiment, it is assumed that the sound (microphone output signal) collected by the microphone 230 has a basic period. The case where the sound collected by the microphone 230 has no basic period will be described later in the second embodiment. Hereinafter, a case where the timing detected by the detection unit 191 is the timing when the operation unit starts operation will be described.

  FIG. 3 shows an example of the waveform of the control signal. Here, the level of the control signal has increased since time t1. In such a case, the detection unit 191 sets, for example, the time t1 when the level of the control signal has risen as the operation start time t1 of the operation unit. FIG. 4 shows an example of impact sound noise and drive sound noise generated in the microphone output signal (periodic sound). As in FIG. 3, impact noise is generated in the microphone output signal at time t1. Further, after the time t1, driving noise is generated in the microphone output signal in a section in which the operating unit is operating.

  The reduction processing unit 250 is a microphone output signal (referred to as “section A” hereinafter, timings t1, t2, t3, t4, and t5 in FIG. 2A) including the timing detected by the detection unit 191. The signal defined with respect to the time axis is removed in the time domain. For example, the reduction processing unit 250 sets all the values of the microphone output signal in the section A to “0” (silence) in order to remove the microphone output signal in the section A in the time domain.

  FIG. 5 shows an example in which the microphone output signal in section A is removed in the time domain. Hereinafter, the end point of the section A is defined as time tc. Here, the reduction processing unit 250 may set the length of the section to be removed as an integer multiple of the basic period of the microphone output signal. In this way, in the signal interpolation processing described later with reference to FIG. 6, the length of the section to be removed is an integral multiple of the wavelength of the signal interpolated in that section, thus facilitating the signal interpolation processing. Can do.

  In the present embodiment, for the sake of simplicity of explanation, the time from the time when the amplitude of the microphone output signal is “0” to the time when the amplitude is again “0” is expressed as the microphone output signal. The basic period method was used. Then, the reduction processing unit 250 selects a section from the time when the amplitude of the microphone output signal is “0” to the time when the amplitude is “0” again, or a section that is an integral multiple of the section. It may be removed. In this way, in the signal interpolation processing described later with reference to FIG. 6, the interpolated signal and the signals before and after the signal are continuous with an amplitude of “0”, so that no signal discontinuity is created. be able to.

  As a method for obtaining the fundamental period of sound, there are an autocorrelation function method, a cepstrum method, and the like. This autocorrelation function method is a method for obtaining a fundamental period by detecting a peak of an autocorrelation function of a sound signal. The cepstrum method is a method for obtaining a fundamental period by detecting a peak of a cepstrum obtained by inverse Fourier transform of a logarithm of a spectrum of a sound signal.

  FIG. 6 shows an example in which the microphone output signal before the section is interpolated in the section A. The reduction processing unit 250 interpolates the microphone output signal before the section in the section A (signal interpolation process). Here, the section of the microphone output signal interpolated in section A (hereinafter referred to as “section B”) is a section having the same length as section A and positioned before section A. For example, the reduction processing unit 250 may set the section located immediately before the section A as the section B. Then, the reduction processing unit 250 interpolates the microphone output signal in the section B into the section A. Thereby, the reduction process part 250 removes the noise of the area A in a time domain.

  Next, the reduction processing unit 250 is a section in which driving noise is generated (a section in which the operation unit is operating after the time tc. Timings t1 to t2, t3 to t4, t4 to t4 in FIG. 2A). (See t5.) For example, by using a spectral subtraction method, the microphone output signal is a signal defined with respect to frequency, and noise reduction processing is performed in the frequency domain, thereby reducing drive noise. FIG. 7 shows an example of a noise reduction processing signal in which impact sound noise and driving sound noise are reduced from the microphone output signal. Note that the reduction processing unit 250 may perform noise reduction processing for the section A in the frequency domain, for example, using a spectral subtraction method.

  Then, the reduction processing unit 250 outputs the microphone output signal in which the impact sound noise and the driving sound noise are reduced from the microphone output signal to the communication unit 170 as a noise subtraction processing signal (sound data). As a result, the noise subtraction processing signal is stored in the storage medium 200.

  When the timing detected by the detection unit 191 is the timing when the operation unit ends the operation, the reduction processing unit 250 uses the microphone output signal of the section including the timing when the operation unit ends the operation as the operation unit operates. As in the case of the section including the timing at which start is started, removal is performed in the time domain. And the reduction process part 250 interpolates the microphone output signal of the area after the said removed area in the removed area. Further, the reduction processing unit 250 reduces the drive sound noise by performing noise reduction processing in the frequency domain, for example, using a spectral subtraction method. Note that the reduction processing unit 250 does not have to perform the noise reduction processing for the times t2 to t3 when the determination unit 251 determines that no noise is generated.

<About subtraction coefficient>
The reduction processing unit 250 performs noise subtraction processing using a predetermined section as a unit of signal processing. Here, the predetermined section is a unit (frame) of signal processing, and may be a section repeated at a constant interval. And these predetermined sections may overlap with other predetermined sections in order of half.

  Then, the reduction processing unit 250 weights the microphone output signal with a window function for each predetermined section. Further, the reduction processing unit 250 reduces noise generated in the microphone output signal by subtracting the estimated noise in the frequency domain from the microphone output signal weighted by the window function.

  Here, when noise is generated only in a part of the section, if the estimated noise is directly subtracted from the section, the noise may be excessively reduced. For this reason, the reduction processing unit 250 may weight the estimated noise with a subtraction coefficient, and subtract the weighted estimated noise in the frequency domain from the microphone output signal weighted with the window function. Then, the subtraction coefficient corresponding to the predetermined section is determined as follows according to the relative position between the window function corresponding to the section and the time tc which is the end point of the section A.

  Hereinafter, the window function will be described as an example of a Hanning window function. 8 to 10 show an example of the positional relationship between the time tc, which is the end point of the section A, and the window function. This time tc is the time tc shown in FIGS. 8 to 10, window functions corresponding to predetermined sections T1 to T4 are defined as window functions W1 to W4, respectively.

  For example, in the example shown in FIG. 8, when the section T1 does not include the time tc and is positioned before the time tc, the subtraction coefficient corresponding to the section T1 is determined according to this relative position. May be. When the section T2 includes the time tc, the subtraction coefficient corresponding to the section T2 is a ratio between the area of the window function W2 in the section T2 and the area of the window function W2 after the time tc (shaded portion in FIG. 8). May be determined based on The same applies to the subtraction coefficient corresponding to the section T3 (see FIG. 9). In FIG. 10, when the section T4 does not include the time tc and is located after the time tc, the subtraction coefficient corresponding to the section T4 may be determined according to the relative position.

  Note that the subtraction coefficient is the relative position between the window function and the time tc when the detection unit 191 detects the operation end timing of the operation unit, as in the case where the detection unit 191 detects the operation start timing of the operation unit. May be determined according to

  FIG. 11 shows an example of the subtraction coefficient thus determined. In the example shown in FIGS. 8 to 10, for example, the subtraction coefficient corresponding to the section T1 is the value “0”, the subtraction coefficient corresponding to the section T2 is the value “0.3”, and the section T3 The corresponding subtraction coefficient is the value “0.9”, and the subtraction coefficient corresponding to the section T4 is the value “1”. Note that these values are examples.

[Second Embodiment]
A second embodiment of the present invention will be described in detail with reference to the drawings. The second embodiment is different from the first embodiment in that the sound (microphone output signal) collected by the microphone 230 has no basic period. Only differences from the first embodiment will be described below.

  FIG. 12 shows an example of impact sound noise and drive sound noise generated in the microphone output signal (non-periodic sound). In FIG. 12, it is assumed that the detection unit 191 detects timing indicating the operation start of the operation unit due to the operation unit starting the operation. FIG. 12 shows that impact noise is generated in the microphone output signal in the section Tc including this timing. Further, it is shown that driving sound noise is generated in the microphone output signal after the impact sound noise is generated and in the section in which the operation unit is operating. Further, a section T5 immediately before the section Tc and having a length of “section Tc + (section Tf) × 2” is shown. Here, the lengths of the sections Tc and Tf may be determined in advance.

  First, the reduction processing unit 250 removes the section Tc from the microphone output signal. FIG. 13 shows an example of the microphone output signal after the section Tc is removed in the time domain. Further, the reduction processing unit 250 makes the midpoint of the section T5, which is the section immediately before the section Tc and whose length is “section Tc + (section Tf) × 2”, coincide with the midpoint of the section Tc. The microphone output signal of section T5 is duplicated. FIG. 14 shows an example of the microphone output signal duplicated in this way.

  Then, the reduction processing unit 250 crossfades the microphone output signal (see FIG. 13) after removing the section Tc in the time domain, and the duplicated microphone output signal (see FIG. 14). Connect signals continuously.

  FIG. 15 shows an example of weighting in crossfading. The solid line shown in FIG. 15 shows the weighting for the microphone output signal (see FIG. 13) after removing the section Tc in the time domain. On the other hand, the broken line indicates the weighting for the duplicated microphone output signal (see FIG. 14). The reduction processing unit 250 may continuously connect both signals by crossfading with such weighting.

  Next, the reduction processing unit 250 performs, for example, spectral subtraction for the section in which the driving sound noise is generated (section after the section Tc and in which the operating section is operating), as in the first embodiment. By using this method, noise reduction processing is executed in the frequency domain, thereby reducing drive sound noise. FIG. 16 shows an example of a noise reduction processing signal in which impact sound noise and driving sound noise are reduced from the microphone output signal.

  In this way, the imaging device (imaging device) reduces the impact sound noise in the time domain and crossfades the microphone output signal in which the impact sound noise is reduced as a process to be varied depending on the judgment. Even when the signal has no fundamental period, not only driving noise but also impact noise can be reduced.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  For example, the detection unit 191 may estimate the timing at which the zoom lens 114 hits the end that limits the movement range of the zoom lens 114 as the zoom lens 114 moves in the optical system 111 based on the amount of movement of the zoom lens 114. . Thereby, the detection unit 191 can estimate the generation timing of the impact noise generated by hitting the end that limits the movement range of the zoom lens 114.

  For example, the detection unit 191 may detect the timing at which the operation unit operates by using a signal indicating timing and a microphone output signal in combination. For example, since the impact noise includes a high frequency component, the detection unit 191 passes the microphone output signal through a high-pass filter, and determines the timing at which the operation unit operates based on the strength of the power of the microphone output signal that has passed. It may be detected.

  Further, for example, when the optical system 111 is detachable from the imaging apparatus 100, any one of the microphone 230, the sound signal processing unit 240, and the reduction processing unit 250 forms an image by the optical system (such as a lens). It may be provided in an optical device (for example, a lens barrel) provided with 111.

  For example, the reduction processing unit 250 may omit the process of setting all the values of the microphone output signals in the section A to “0” and replace the microphone output signals in the section A with the microphone output signals in the section B.

  In addition, for example, the reduction processing unit 250 may perform SNG (Spectral Noise Gating) processing on the microphone output signal.

  For example, the subtraction coefficient may be determined according to the position of the time tc in the section, for example, without depending on the area ratio in the window function.

  Further, for example, the reduction processing unit 250 adds a predetermined delay time to the timing detected by the detection unit 191, and removes the microphone output signal of the section including the timing obtained thereby in the time domain. May be.

  In the above description, the case where the reduction processing unit 250 performs signal processing on the sound signal collected by the microphone 230 has been described. However, the processing of the above-described reduction processing unit 250 according to the present embodiment is not limited to this. Thus, the present invention is not applied only to a sound signal collected in real time.

  For example, for a sound signal that has already been recorded, the timing indicating that the operation unit of the device that has recorded the sound signal is operated in association with the sound signal. Even when recorded in the storage unit, the reduction processing unit 250 according to the present embodiment can similarly perform the signal processing described above.

  Further, the procedure by the detection unit 191, the determination unit 251, or the reduction processing unit 250, which is a program for realizing the procedure described above with reference to FIGS. An execution process may be performed by causing a computer system to read and execute a program recorded on a recording medium. Here, the “computer system” may include hardware such as an OS (Operating System) and peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 100 ... Imaging device, 110 ... Imaging part, 190 ... CPU, 191 ... Detection part, 170 ... Communication part, 200 ... Storage medium, 230 ... Microphone, 240 ... Sound signal processing part, 250 ... Reduction processing part, 251 ... Determination part

Claims (11)

A photographing unit for photographing an image by an optical system;
A microphone that converts sound waves into electrical signals;
A signal detection unit for detecting at least one of a sensor signal detected by a sensor for detecting a shooting state and a control signal for controlling shooting;
A determination unit that determines whether or not there is a change in a shooting operation using at least one of the sensor signal and the control signal;
A noise processing unit for differentiating processing for reducing noise of the electrical signal when the determination unit determines that there is a change in the shooting operation and when the determination unit does not determine that there is a change in the shooting operation; An imaging device comprising: An imaging apparatus according to claim 1,
The noise processing unit performs noise reduction processing on the electrical signal in a time domain when the determination unit determines that there is a change in shooting operation, and when the determination unit does not determine that there is a change in shooting operation, An imaging apparatus, wherein an electrical signal is subjected to noise reduction processing in a frequency domain. An imaging apparatus according to claim 1,
When the determination unit determines that there is a change in the shooting operation, the noise processing unit performs noise reduction processing using the electrical signal as a signal defined with respect to a time axis, and the determination unit changes the shooting operation. When it is not determined that there is an image pickup apparatus, noise reduction processing is performed using the electrical signal as a signal defined with respect to frequency. It is an imaging device given in any 1 paragraph of Claims 1-3,
The noise processing unit performs noise reduction processing for a time region including a time at which it is determined that there is a change in the shooting operation of the electrical signal,
An imaging apparatus, wherein noise reduction processing is performed using the electrical signal as a signal defined with respect to frequency. It is an imaging device given in any 1 paragraph of Claims 1-4,
The determination unit is configured to determine whether or not there is a change in a photographing operation that causes an impact sound. An imaging apparatus according to any one of claims 1 to 5, wherein
The imaging device change determined by the determination unit is at least one of an operation change of an actuator driven in imaging and an operation change of an operation unit operable by a photographer. It is an imaging device given in any 1 paragraph of Claims 1-6,
The determination unit determines whether there is a change in a shooting operation using the electrical signal. It is an imaging device given in any 1 paragraph of Claims 1-7,
The determination unit determines at least one of the start of the shooting operation and the end of the shooting operation,
When the determination unit determines that the photographing operation is started, the noise processing unit performs noise reduction processing using the electrical signal before the start of the photographing operation, and the determination unit determines that the photographing operation is finished. When the determination is made, a noise reduction process is performed using the electrical signal after the end of the shooting operation. An optical system for forming an image by the optical system;
A microphone that converts sound waves into electrical signals;
A signal detection unit for detecting at least one of a sensor signal detected by a sensor for detecting a shooting state and a control signal for controlling shooting;
A determination unit that determines whether or not there is a change in a shooting operation using at least one of the sensor signal and the control signal;
A noise processing unit for differentiating processing for reducing noise of the electrical signal when the determination unit determines that there is a change in the shooting operation and when the determination unit does not determine that there is a change in the shooting operation; An optical device comprising: The optical device according to claim 9, comprising:
When the determination unit determines that there is a change in the shooting operation, the noise processing unit performs noise reduction processing using the electrical signal as a signal defined with respect to a time axis, and the determination unit changes the shooting operation. When it is not determined that there is an optical device, noise reduction processing is performed using the electrical signal as a signal defined with respect to frequency. On the computer,
An input procedure for receiving at least one of an acoustic signal acquired at the time of shooting, a sensor signal detected by a sensor for detecting a shooting state, and a control signal for controlling shooting;
A determination procedure for determining whether or not there has been a change in photographing operation using at least one of the sensor signal and the control signal;
A noise processing procedure for performing different processing for reducing noise of the acoustic signal between when it is determined that there is a change in the shooting operation and when the determination unit does not determine that there is a change in the shooting operation program.

Images