JP2019036867A - Voice processing apparatus and control method of the same - Google Patents

Abstract

To make it possible to remove or reduce noise from a drive unit without newly adding a dedicated microphone for noise detection, without generating processing for audio processing with a two-channel microphone configuration.SOLUTION: A voice processing apparatus includes: a first microphone that sets a main acquisition target as voice from outside the voice processing apparatus; a second microphone whose main acquisition target is drive noise from a drive unit as compared with the first microphone; and a noise removing unit for generating two-channel voice data in which driving noise generated by the drive unit included in the apparatus is reduced on the basis of a difference between time-series voice data obtained from each of the first and second microphones. The noise removing unit has two adaptive filter units for generating stereo two-channel audio data by filtering each piece of time-series voice data from the first and second microphones.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a voice processing technique.

  In recent years, higher functions of photographing apparatuses such as cameras have been advanced. There are many cameras that can shoot both video and still images as part of improving functionality. In moving image shooting, these cameras acquire sound at the same time as the moving image is acquired, and perform recording in synchronization with the moving image. Not a few cameras have a problem that the drive sound generated by the drive unit of the optical system (the drive sound of the focusing lens and zoom lens) is recorded as noise.

  Patent Document 1 discloses a noise removal technique for removing or reducing the drive sound generated when driving such focus and zoom.

JP 2011-114465 A

  Patent Document 1 has a noise recording microphone for detecting noise in the drive unit, and subtracts the audio signal acquired by the noise recording microphone from the audio signal acquired by the normal audio recording microphone. Drive noise is reduced.

  However, miniaturization and integration are progressing in photographing apparatuses such as digital cameras. Naturally, a sound collection unit such as a microphone, a display unit for confirming an image, an operation member, and the like are arranged at positions close to each other. For this reason, adding a new noise recording microphone causes an increase in cost and area.

  In general, the noise of the drive unit is removed by converting a time-series audio signal into the frequency domain once by FFT or the like, discriminating and removing the noise of the drive unit, and converting it again into a signal in the time domain (inverse FFT). Take the configuration. Since the conversion to the frequency domain is performed based on time-series data, there is a problem that a delay occurs in the recorded sound when the noise removal process is executed.

  The present invention provides a technology for removing or reducing noise from a drive unit without adding a new dedicated microphone for noise detection, without generating processing for audio processing with a two-channel microphone configuration. It is something to be offered.

In order to solve this problem, for example, the speech processing apparatus of the present invention has the following configuration. That is,
A voice processing device,
A drive unit;
A first microphone whose main acquisition target is audio from outside the device;
Compared with the first microphone, a second microphone whose main acquisition target is drive noise from the drive unit,
A noise removing unit that generates 2-channel audio data in which driving noise generated by the driving unit is reduced based on a difference between time-series audio data obtained from each of the first microphone and the second microphone; Have
The noise removing unit
A determination unit for determining occurrence of the driving noise from a difference between time-series audio data obtained from the first and second microphones;
A correlation processing unit for obtaining a correlation value of phases of time-series audio data obtained from the first and second microphones when the determination unit determines that a driving noise is generated;
Based on the correlation value, out of time-series audio data from each of the first and second microphones, an error in the angle of incidence of sound from the outside to the first and second microphones is set in advance. A generation unit that generates time-series audio data determined to exceed the threshold value;
The time series audio data obtained from the first microphone and the time series audio data corresponding to the first microphone generated by the generation unit are input to perform adaptive filter processing, and one of the stereo A first adaptive filter for generating audio data of a plurality of channels;
The time-series audio data obtained from the second microphone and the time-series audio data corresponding to the second microphone generated by the generation unit are input to perform adaptive filter processing, and the other stereo And a second adaptive filter for generating audio data of channels.

  According to the present invention, noise is removed or reduced from the drive unit without adding a dedicated microphone for noise detection, with a 2-channel microphone configuration, and without processing for audio processing. It becomes possible.

The block diagram which shows the adaptive filter in embodiment. 1 is a block diagram showing a system configuration of a digital camera in an embodiment. The block diagram which shows the noise removal system in embodiment. The operation | movement timing chart of the MS calculating part in embodiment. The mechanical block diagram which shows the structure of the microphone unit in embodiment. The conceptual diagram of the phase difference detection in embodiment.

  DESCRIPTION OF EMBODIMENTS Hereinafter, an audio processing device according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In the following description, an image pickup apparatus typified by a digital camera will be described as an example of an apparatus equipped with an audio processing apparatus. However, any apparatus having a drive unit that generates drive noise can be applied, and thus the present invention is limited. Is not to be done. It should be recognized that this is to facilitate understanding by showing specific examples.

  FIG. 2 is a block diagram of the photographing apparatus 201 to which the embodiment is applied. The shutter 202 has an aperture function. The image sensor 204 converts the optical image into an electrical signal and outputs an analog signal representing the intensity of light. The A / D converter 205 converts the analog signal output from the image sensor 204 into a digital signal. The timing generation circuit 206 is controlled by the memory control circuit 208 and the system control circuit 218, and supplies a clock signal and a control signal to the image sensor 204, A / D converter 205, and D / A converter 210. The image processing circuit 207 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 205 or the data from the memory control circuit 208.

  The image processing circuit 207 performs predetermined arithmetic processing using the captured image data. The system control circuit 218 performs AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash) light emission processing (not shown) based on the calculation result obtained from the image processing circuit 207. Further, the image processing circuit 207 performs predetermined arithmetic processing using the captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained arithmetic result.

  The memory control circuit 208 controls the A / D converter 205, timing generation circuit 206, image processing circuit 207, image display memory 209, D / A converter 210, memory 213, and compression / decompression circuit 214. The data of the A / D converter 205 is written to the image display memory 209 or the memory 213 via the image processing circuit 207 and the memory control circuit 208 or directly from the data of the A / D converter 205 via the memory control circuit 208. It is.

  The external output connector 211 outputs the output of the D / A converter 210 to an external monitor. The system control circuit 218 can detect whether or not a connector is inserted into the external output connector 211 based on a signal from the external output connection detection unit 235. The external output connector 211 is a composite interface, for example. However, an HDMI (registered trademark) connector may be used.

  The image display unit 212 includes a TFT LCD or the like, and receives display image data written in the image display memory 209 via the D / A converter 210 and displays it. A live view function can be realized by sequentially displaying captured image data on the image display unit 212. Further, the image display unit 212 can arbitrarily turn on / off the display according to an instruction from the system control circuit 218. When the display is turned off, the power consumption of the photographing apparatus 201 can be significantly reduced. I can do it.

  The memory 213 is a memory for temporarily storing captured still images and moving images, and has a sufficient storage capacity for storing a predetermined number of still images and moving images for a predetermined time. This makes it possible to write a large amount of images to the memory 213 at high speed even in continuous shooting or panoramic shooting in which a plurality of still images are continuously shot. The memory 213 can also be used as a work area for the system control circuit 218. Further, the memory 213 is also used as a write buffer for the recording medium 229.

  The compression / decompression circuit 214 is a circuit that compresses / decompresses image data by adaptive discrete cosine transform, etc., reads an image stored in the memory 213, performs compression processing or decompression processing, and stores the processed data in the memory 213. Write to.

  The shutter 202 having a diaphragm function has a drive unit such as a motor for driving the diaphragm and the shutter. The exposure control unit 215 controls the shutter 202 having an aperture function by controlling the operation of the driving unit. The taking lens 203 has a drive unit such as a motor for driving the lens. The distance measurement control unit 216 controls the driving unit of the taking lens 203 to control focusing. Further, the zoom control unit 217 controls the zooming by controlling the drive unit of the taking lens 203.

  The exposure control unit 215 and the distance measurement control unit 216 perform control using the TTL method. These controls are performed by the system control circuit 218. That is, the system control unit 218 controls the exposure control unit 215 and the distance measurement control unit 216 based on the calculation result calculated by the image processing circuit 207 for the image data obtained by imaging.

  The system control circuit 218 is a circuit that controls the entire photographing apparatus 201. The system control circuit 218 implements the processing of each embodiment described later by executing a program recorded in the nonvolatile memory 220.

  The memory 219 is a memory that develops constants and variables for the operation of the system control circuit 218, a program read from the nonvolatile memory 220, and the like, and has a faster access speed than the memory 213. Typically, the memory 213 is a DRAM and the memory 219 is an SRAM. The nonvolatile memory 220 is an electrically erasable / recordable memory. The nonvolatile memory 220 stores constants, programs, and the like for operating the system control circuit 218. Here, the program is a program for executing various flowcharts in each embodiment described later.

  The shutter switches SW221, SW222, and the operation unit 223 are operation units for inputting various operation instructions of the system control circuit 218. The shutter switches SW221, SW222, and the operation unit 223 are configured by a single or a combination of a switch, a dial, a touch panel, a voice recognition device, and the like. Is done. Here, a specific description of these operation units will be given. The shutter switch SW221 is turned on during the operation of the shutter button, and instructs to start operations such as AF (autofocus) processing, AE (automatic exposure) processing, and AWB (auto white balance) processing. The shutter switch SW222 is turned on when the operation of the shutter button is completed. When the shutter switch SW222 is turned on, the system control unit 218 converts the video signal from the image sensor 204 into digital image data by the A / D converter 205, and the image data is stored in the memory via the memory control circuit 208. An exposure process for writing image data to 213 is performed. At the same time, the system control unit 218 instructs the start of an unillustrated EF (flash emission) process as necessary. In addition, the system control unit 218 causes development processing using calculation in the image processing circuit 207 and the memory control circuit 208 to be performed. Further, the system control unit 218 performs a series of processes such as a recording process of reading image data from the memory 213, compressing the image data by the compression / decompression circuit 214, and writing the image data to the recording medium 229. In the case of moving image shooting, the system control unit 218 instructs various circuits to start / stop moving image shooting.

  The operation unit 223 includes various buttons and a touch panel. Button types include menu button, set button, macro button, multi-screen playback pagination button, flash setting button, single / continuous / self-timer switching button, menu move + (plus) button, menu move-(minus) ) Button is included. Also, playback image move + (plus) button, playback image-(minus) button, shooting image quality selection button, exposure compensation button, date / time setting button. A selection / switch button for setting selection and switching of various functions and a determination button for setting determination and execution of various functions are included. Further, a display button for setting ON / OFF of the image display unit 212 is also included. Also included is a quick review ON / OFF switch for setting a quick review function for automatically reproducing image data taken immediately after shooting. The operation unit 223 also includes a zoom operation unit that adjusts zoom and wide angle during shooting, adjusts image enlargement / reduction during playback, and switches between single-screen display / multi-screen display. Further, a compression mode switch for selecting a compression rate for JPEG compression or for selecting a CCD RAW mode in which a signal from an image sensor is directly digitized and recorded on a recording medium is also included.

  The power supply control unit 225 detects the presence / absence of a battery, the type of battery, and the remaining battery level, and based on the detection result and the instruction of the system control circuit 218, each unit including a recording medium for a necessary period of time To supply.

  The power supply unit 228 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like. The power supply control unit 225 and the power supply unit 228 are connected to each other via respective electrodes 226 and 227.

  The interface 224 is an interface with a recording medium such as a memory card or a hard disk. The interface 224 may be configured using an SD card, a compact flash (registered trademark) card, or the like that conforms to a standard. Furthermore, by connecting various communication cards to the interface 224, image data and management information attached to the image data can be transferred to and from other devices.

  The protection unit 231 functions as a barrier that prevents contamination and damage of the imaging unit by covering the imaging unit including the lens 203 of the imaging apparatus 201 when the power is turned off in conjunction with the power source of the apparatus.

  The microphone unit 232 is an audio data acquisition unit from a microphone. The audio processing circuit 233 performs A / D conversion so that the audio data obtained by the microphone unit 232 is acquired by the system control circuit 218. The stereo microphone unit 232 is a microphone unit of 2ch or more, but in the embodiment, for the sake of simplicity, the stereo microphone unit 232 will be described as a 2ch (stereo) microphone.

  The speaker unit 234 is an audio data output unit that outputs audio data from the speaker. The system control circuit 218 causes the audio processing circuit 233 to D / A convert the processed audio data and output the audio data to the speaker unit 234 to reproduce the audio.

  The recording medium 229 is a recording medium such as a memory card or a hard disk. In addition, when the recording medium 229 is a PCMCIA card, a compact flash (registered trademark) card, or the like, an information storage circuit in which performance is described may be incorporated.

  The posture detection unit 236 detects the tilt and rotation of the photographing device 201 and outputs posture information indicating the device posture. The acceleration detection unit 237 derives the acceleration with respect to the movement amount of the apparatus in the three axis directions, and outputs the acceleration information.

  The structure and processing / function of the imaging apparatus 201 in the embodiment have been described above.

  Next, the drive sound removal process in the embodiment will be described in detail with reference to FIGS. The driving sound referred to here indicates noise generated by the driving unit when the zoom control unit 217 performs zooming control of the photographing lens 203.

  First, the configuration of an adaptive filter will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the adaptive filter. This adaptive filter is also a series of arithmetic processing executed by a program (not shown) recorded in advance in the memory 219 in FIG. The system control circuit 218 reads a program (not shown) from the memory 219 and sequentially executes the audio data input via the audio processing circuit 233. The configuration and arithmetic processing of this adaptive filter will be described in detail.

  The adaptive filter has two inputs A and B, a transversal filter circuit 101 that performs a product-sum operation on data from the input B, and coefficients used by the transversal filter circuit 101 based on an evaluation function based on an adaptive algorithm. An updating unit 103 for updating and an adder 102 for adding the output of the transversal filter circuit 101 and the input A are provided.

  In general, the input A side is called a desired signal, the input B side is called a reference signal, and the output side is called a desired signal. When an adaptive filter is used as a noise removing unit, an audio signal generated from a noise source to be removed is applied to a desired signal, and an audio signal in which the noise is added to an audio signal to be observed is applied to a reference signal and output. A speech signal from which noise is removed is obtained as a signal.

  The transversal filter circuit 101 includes a plurality of delay elements (not shown) that delay the reference signal x (t) obtained from the input B, x (t), and delayed signals x (t−1), x (t -2), a plurality of multipliers that multiply the coefficients h0 (t), h1 (t), and h2 (t) set by the evaluation unit 103 according to the evaluation function, and the output of the multipliers are added to obtain an estimated signal and a plurality of adders for outputting y (t). At this time, t is a unit representing time, and x (t) represents the t-th sample in the time-series audio digital data x.

  The estimated signal y (t) is given by the following equation. m is the number of coefficients, N is a natural number, and m = 2 and N = 3 when the adaptive filter has h0 (t), h1 (t), and h2 (t) as coefficients.

  Further, a subtractor 102 that subtracts the estimated signal y (t) from the desired signal d (t) is provided, and an error signal e that is a difference between the estimated signal y (t) that is the output of the subtracter and the desired signal d (t). The coefficient of the transversal filter circuit 101 is updated by the evaluation function 103 so that (t) approaches 0.

  As a coefficient updating algorithm, a least mean square (LMS) algorithm has been widely used. In this algorithm, the coefficient is updated so as to minimize the mean square error E [e (t) 2] of the error signal e (t). The coefficients h0 (t), h1 (t) and h2 (t) set in advance are updated, and h0 (t + 1), h1 (t + 1) and h2 (t + 1) are derived.

  The following formula shows an LMS algorithm as an example of coefficient update.

  Μ in this equation is called a step size and has a role of determining the magnitude of coefficient update. Usually, a constant value is used, and a value of about 0.05 to 0.10 is used. It is desirable to determine in advance according to the configuration of the photographing apparatus 201, and accurate estimation is possible if it is small, but if it is too large, the filter output diverges.

  A noise component to be removed is input to the reference signal x (t), and an audio signal including the noise component is input to the desired signal d (t). By repeating the above series of processing, the error signal e (t) can be brought close to 0, that is, noise can be removed.

  Further, since processing can be performed for each sample of audio data without using time-series audio data which is different from that of FFT or the like, there is no delay due to the processing.

  Based on the above, the noise removal system in the embodiment will be described with reference to the block diagram of FIG.

  This noise removal system includes a MAIN microphone 301, a SUB microphone 302, an A / D conversion unit 303, and a noise removal unit 309. The MAIN microphone 301 and the SUB microphone 302 are microphones constituting the 2ch microphone unit 232. As will be described in detail later, the MAIN microphone 301 is a microphone whose main acquisition target is audio from outside the apparatus. Further, the SUB microphone 302 has a main acquisition target as drive noise from the drive unit as compared with the MAIN microphone 301. The A / D conversion unit 303 is a circuit included in the audio processing circuit 233. The noise removing unit 309 is a series of arithmetic processing in which the system control circuit 218 executes a program (not shown) recorded in advance in the memory 219 in FIG. This program is stored in the nonvolatile memory 220, and is read by the system control circuit 218 into the memory 219 and executed. The system control circuit 218 sequentially executes this program and processes the audio data input from the audio processing circuit 233.

  Here, the mechanical configuration that constitutes the microphone unit 232 for 2ch according to the present embodiment will be described in detail with reference to FIGS.

  FIG. 4A is an external view of the imaging apparatus according to the present embodiment. The MAIN microphone 301 is on the right side and the SUB microphone 302 is on the left side when viewed from the photographer side when the image pickup apparatus is held facing the subject. The MAIN microphone 301 and the SUB microphone 302 are symmetrical with respect to the center position of the viewpoint of the imaging unit in order to finally function as a stereo microphone.

  The enlarged view in FIG. 4A shows the mechanical components of the MAIN microphone 301 and the SUB microphone 302 that are part of the microphone unit 232, and FIG. 4B shows the broken line а-а ′ portion of the mechanical configuration. It is an enlarged view of the section shown.

  An exterior portion 401 that forms an opening (hereinafter referred to as a microphone hole) for passing acoustic vibrations that propagate air, a MAIN microphone bush 403 that holds a MAIN microphone 301, a SUB microphone bush 402 that holds a SUB microphone 302, respectively. The pressing portion 406 presses and holds the microphone bush against the exterior portion 401. The exterior portion 401 and the pressing portion 406 are made of a mold member such as a PC material, but there is no problem even if it is a metal member such as aluminum or stainless steel. Further, the MAIN microphone bush 403 and the SUB microphone bush 402 are made of a rubber material such as ethylene propylene diene rubber.

  Here, the hole diameter (area) of the microphone hole in the exterior part 401 will be described. The diameter of the microphone hole 401b to the SUB microphone 302 is smaller than the diameter of the microphone hole 401a to the MAIN microphone 301, and is reduced at a predetermined magnification. The microphone hole shape is preferably circular or elliptical, but may be rectangular. Moreover, about each hole shape, the same shape or another shape may be sufficient. This configuration is intended to make it difficult for drive noise transmitted by air propagation to the microphone inside the imaging apparatus to leak from the microphone hole side of the SUB microphone 302 to the outside.

  Next, the space in front of the microphone constituted by the exterior portion 401 and the microphone bush will be described. The space in front of the SUB microphone 302 composed of the exterior part 401 and the SUB microphone bush 402 has a larger space volume than the space in front of the MAIN microphone 301 composed of the exterior part 401 and the MAIN microphone bush 403 and has a predetermined magnification. The structure which secures the volume of is taken. With this configuration, in the space in front of the SUB microphone 302, the atmospheric pressure change in the space becomes large, and the drive noise from the drive unit (the drive sound of the zoom lens in the embodiment) is emphasized.

  As described above, the input of the SUB microphone 302 in the microphone input mechanical configuration has a configuration in which the amplitude of the driving noise is greatly emphasized with respect to the input of the MAIN microphone 301. Regarding the relationship between the sound levels at which driving noise is input to each microphone, the SUB microphone 302 is larger than the MAIN microphone 301. Conversely, the level relationship of the sound (sound in the surrounding environment, which is the original purpose of sound collection) input from the front of the microphone hole to each microphone by air propagation is larger in the MAIN microphone 301 than in the SUB microphone 302. Become.

  As described above, of the two-channel microphones constituting the microphone unit 232, one of the microphones (MAIN microphone 301) has a structure that makes it easy to pick up external sounds and does not easily pick up internal sounds, and acquires environmental sounds. Have a role to do. The other microphone (SUB microphone 302) has a holding structure that makes it easy to pick up internal sound and difficult to pick up external sound, and has a role of acquiring drive sound information. In such a configuration, the driving sound is recorded more loudly in the SUB microphone than in the MAIN microphone. However, since the sound around the subject is far away from both microphones, both microphones have almost the same volume. Is output.

  Next, processing related to the MAIN microphone 301 and the SUB microphone 302 will be described in detail with reference to FIGS. The driving noise in the embodiment is a sound generated during zoom driving, and the generation source is the imaging apparatus itself. Therefore, the distance between the source of the driving noise and each microphone is much shorter than the distance between the subject and the imaging device at the time of imaging. Therefore, it can be said that the phase difference between the drive noises detected by the MAIN microphone 301 and the SUB microphone 302 is negligibly small. On the other hand, it should be noted that the sound propagating from outside the apparatus detected by the MAIN microphone 301 and the SUB microphone 302 naturally has a phase difference.

  In FIG. 3, an A / D conversion unit 303 converts audio signals from the MAIN microphone 301 and the SUB microphone 302 into digital signals at a preset sampling period (for example, 44.1 KHz). The MS calculation unit 304 functions as a determination unit for determining the presence or absence of drive noise from the audio signals obtained from the MAIN microphone 301 and the SUB microphone 302.

  The operation of the MS calculation unit 304 is shown in the operation timing chart of FIG.

  In FIG. 5, MAIN [t] and SUB [t] are audio signals of the t samples of the MAIN microphone 301 and the SUB microphone 302, and MAIN [t] −SUB [t] are audio signals of the MAIN microphone 301 from the SUB microphone 302. The subtraction amount obtained by subtracting the signal is shown. Further, t1-t2 indicates a driving period of the zoom lens.

  As described above, the MAIN microphone 301 and the SUB microphone 302 are detected by superimposing the sound from the outside of the apparatus and the driving noise from the driving source in the apparatus. However, as compared with the sub microphone 302, the MAIN microphone 301 is mainly targeted for sound from outside the apparatus. On the contrary, the SUB microphone 302 has a driving noise as a main target as compared with the MAIN microphone 301. Therefore, during the period in which the zoom lens before the timing t1 is in a non-driving state, no driving noise is generated, so MAIN [t] -SUB [t] generally has a positive value as illustrated.

  In the period from timing t1 to t2, which is the driving period of the zoom lens, SUB [t] greatly increases with respect to MAIN [t], and the subtraction amount MAIN [t] −SUB [t] is a negative value. Thus, it can be seen that the value falls below the zoom detection threshold value 501 (threshold value having a negative value). That is, it can be said that the period from timing t1 to t2 is a period indicating a noise generation state.

  The MS calculation unit 304 obtains a subtraction amount MAIN [t] -SUB [t] from the input MAIN [t] and SUB [t] signals, and the MAIN during the period when the subtraction amount falls below the zoom detection threshold. The signals [t] and SUB [t] are output as M_x [t] and S_x [t]. Here, M_x [t] corresponds to MAIN [t], and S_x [t] corresponds to SUB [t].

  As shown in the timing chart of FIG. 5, the output from the period tl to t2 of MAIN [t] is M_x [t], and the output from the period tl to t2 of SUB [t] is S_x [t]. Note that in a period in which the subtraction amount MAIN [t] −SUB [t] is 0 or has a positive value, M_x [t] and S_x [t] have a value of zero.

  At this time, t is a unit representing time, and x [t] represents the t-th sample in the time-series audio digital data x.

In the embodiment, when the MS calculation unit 304 has a negative threshold value Th,
MAIN [t] -SUB [t] <Th
It was determined that there was drive noise in a state satisfying the conditions, and the values of MAIN [t] and SUB [t] at that time were output. Then, the MS calculation unit 304
MAIN [t] −SUB [t] ≧ Th
When satisfying the condition, MAIN [t] = SUB [t] = 0 was output.

  However, the determination method with the threshold is not limited to the above. For example, an appropriate positive threshold Th may be defined, and it may be determined that there is drive noise when SUB [t] −MAIN [t]> Th. In short, it is only necessary to determine that there is drive noise on the condition that the value of the audio data is sufficiently larger than the value of the audio data obtained by the MAIN microphone 301 by the SUB microphone 302.

  Next, the MS calculation unit 304 sequentially supplies the output data M_x [t] and S_x [t] to the cross correlation processing unit 305 and the phase difference detection processing 306. The cross-correlation processing unit 305 and the phase difference detection processing 306 perform processing aimed at accurate discrimination / extraction of drive sound. The cross-correlation processing unit 305 and the phase difference detection processing 306 are for accurately extracting only the drive sound generated by zooming control from the digital data M_x [t] and S_x [t] output from the MS calculation unit 304. Process.

  First, the cross-correlation processing unit 305 examines the cross-correlation between M_x [t], which is the MAIN microphone output signal from the MS arithmetic unit 304, and S_x [t], which is the SUB microphone output signal.

  The output of the MS calculation unit 304 is MAIN [t], SUB [t] during a period in which the difference between the microphone outputs is large and the subtraction amount exceeds the zoom detection threshold value 501 (below the zoom detection threshold value 501). Signal. For this reason, these data may include environmental sounds generated during the zoom period.

  The cross-correlation processing unit 305 is performed to extract only the driving sound from these data, and selects and outputs signals having a high correlation with each other. The two inputs M_x [t] and S_x [t], which are these data, have a difference in level, but the rise time and fall time are generally the same, the waveforms tend to overlap, and the cross-correlation value tends to increase. It is in. The contents of this processing are shown in the following equation (3).

  The two inputs are shifted by M samples and summed. M is a preset value and is stored in the recording unit 230 in FIG. Although M is influenced by the main body configuration of the product, it is desirable that M be as small as possible (about 1 to 5).

  In Expression (3), when φms takes a positive value, it can be determined that the cross-correlation is high. Therefore, the cross-correlation processing unit 305 obtains M when φms is positive and maximum. The cross-correlation processing unit 305 outputs the obtained M and the shifted result as M_x ′ [t] and S_x ′ [t].

  Further, since the threshold value of the sum is affected by the main body configuration of the product and the microphone arrangement, there may be a configuration in which a correction term or the like is added for each product. In this case, the correction term is held in a nonvolatile recording unit such as the nonvolatile memory 220.

  The outputs M_x ′ [t] and S_x ′ [t] of the cross correlation processing unit 305 are then input to the phase difference detection processing unit 306. Then, the phase difference detection processing unit 306 performs phase difference detection processing, and generates time-series audio data for MAIN and SUB. The contents of the process will be described with reference to FIG.

  In FIG. 6, when the distance between the MAIN microphone 301 and the SUB microphone is defined as Lmic and the incident angle at which sound enters is defined as θ, the phase difference is derived by the following equation.

  C is the speed of sound, and j is the number of samples shifted to obtain the correlation in equation (3). G is a threshold value of Δθ, and a value studied in accordance with the configuration of the photographing apparatus is recorded in advance in the nonvolatile memory 220 in FIG.

  When the expressions (4) and (5) are calculated, θ = 0 when the phases of the audio data are aligned.

  Since the driving sound propagates through the photographing apparatus, it is recorded as audio data through various members holding the microphone unit. Since there are various propagation paths, when the phase detection as shown in FIG. 6 is performed, the value of the incident angle θ into which the sound enters is not constant, and the temporal change becomes severe.

  Using this characteristic, in this embodiment, the phase detection processing unit 306 passes the audio data when the phase difference Δθ per unit time is larger than the threshold G in order to select the noise that has propagated inside the housing. In the case where Δθ is equal to or smaller than the threshold value G, zero is output. The phase detection processing unit 306 outputs M_x ′ [t] and S_x ′ [t] corresponding to Expression (6) as M_x ″ [t] and S_x ″ [t] to the subsequent processing, thereby allowing the interior of the casing to be output. It is possible to distinguish and extract the driving sound transmitted.

  The MAIN channel adaptive filter 307 receives the signal M_x ″ [t] that has passed through the phase detection and the signal MAIN [t] that has passed through the A / D conversion unit 303, performs filter processing, and reduces drive noise. It is output as an audio signal of one stereo channel (stereo R channel audio signal according to FIG. 4), which is removed or reduced.

  On the other hand, the adaptive filter 308 for the SUB channel receives the signal S_x ″ [t] that has passed through the phase detection and the signal SUB [t] that has passed through the A / D conversion unit 303, performs filter processing, and is driven. The audio signal is output as the other stereo channel audio signal (stereo L channel audio signal according to FIG. 4) with noise removed or reduced.

  Then, the system control circuit 218 performs gain adjustment processing and stereo enhancement processing for each channel, and then combines with the moving image data, converts it to a file such as MOV and MPEG, and sends the audio to the recording unit 230 in FIG. It will be saved as an attached moving image file.

  As described above, an audio signal that cannot be cross-correlated and cannot be phase-detected is separated as noise in which driving sound has propagated inside the housing, and noise removal is performed by adaptive filter processing using the signal as a reference signal. In this way, it is possible to remove drive sound.

  In the embodiment, each component of the noise removing unit 309 illustrated in FIG. 3 has been described as a functional unit that executes a program by the system control circuit 218. In this case, each component is implemented as a function, and the processing sequence between them is as shown in FIG. Therefore, FIG. 3 can also be viewed as a flowchart executed by the system control circuit 218. Note that some or all of the elements in FIG. 3 may be realized by hardware.

  As described above, according to the embodiment, the amount of processing delay related to driving noise removal is eliminated because there is no longer a period for storing a large number of audio data for FFT conversion, compared to the conventional case using FFT (Fast Fourier Transform). Can be less. As a result, for example, when the noise removal unit 309 shown in the embodiment is installed in an imaging apparatus such as a digital video camera, the recorded image can be recorded while confirming the actual noise-removed sound using headphones or the like. .

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 201 ... Imaging device, 218 ... System control circuit, 232 ... Microphone unit, 233 ... Audio processing circuit, 301 ... MAIN microphone, 302 ... SUB microphone, 303 ... A / D converter, 304 ... MS operation part, 305 ... Cross-correlation processing unit, 306... Phase detection processing unit, 307, 308.

Claims (7)

A voice processing device,
A drive unit;
A first microphone whose main acquisition target is audio from outside the device;
Compared with the first microphone, a second microphone whose main acquisition target is drive noise from the drive unit,
A noise removing unit that generates 2-channel audio data in which driving noise generated by the driving unit is reduced based on a difference between time-series audio data obtained from each of the first microphone and the second microphone; Have
The noise removing unit
A determination unit for determining occurrence of the driving noise from a difference between time-series audio data obtained from the first and second microphones;
A correlation processing unit for obtaining a correlation value of phases of time-series audio data obtained from the first and second microphones when the determination unit determines that a driving noise is generated;
Based on the correlation value, out of time-series audio data from each of the first and second microphones, an error in the angle of incidence of sound from the outside to the first and second microphones is set in advance. A generation unit that generates time-series audio data determined to exceed the threshold value;
The time series audio data obtained from the first microphone and the time series audio data corresponding to the first microphone generated by the generation unit are input to perform adaptive filter processing, and one of the stereo A first adaptive filter for generating audio data of a plurality of channels;
The time-series audio data obtained from the second microphone and the time-series audio data corresponding to the second microphone generated by the generation unit are input to perform adaptive filter processing, and the other stereo And a second adaptive filter for generating voice data of the channels. The determination unit has drive noise when time-series audio data obtained from the second microphone is greater than a preset threshold with respect to time-series audio data obtained from the first microphone. The sound processing apparatus according to claim 1, wherein The correlation processing unit
When time-series audio data from the first and second microphones obtained from the determination unit is M_x [t] and S_x [t],

The speech processing apparatus according to claim 1, wherein M having a positive φms and a maximum value is determined as a correlation value. The first microphone is a microphone whose main target is sound from outside the device, which propagates through a first opening provided at a predetermined position of the casing of the sound processing device,
The second microphone is a microphone that converts sound input through the second opening having a smaller area than the first opening into an electric signal, and the driving unit included in the sound processing device. In order to propagate the driving noise from the second microphone to the second microphone, the volume of the space between the second microphone and the second opening is determined between the first microphone and the first opening. The audio processing apparatus according to claim 1, wherein the volume is larger than a volume between them. The audio processing apparatus according to claim 1, wherein an imaging unit is provided between the first and second microphones. A drive unit, a first microphone having a main acquisition target as a sound from outside the device, and a second microphone having a main acquisition target as drive noise from the drive unit as compared with the first microphone A method for controlling a speech processing apparatus, comprising:
A noise removing step of generating two-channel audio data in which the driving noise generated by the driving unit is reduced based on a difference between the time-series audio data obtained from the first microphone and the second microphone; And
The noise removal step is
A determination step of determining occurrence of the driving noise from a difference between time-series audio data obtained from the first and second microphones;
A correlation processing step for obtaining a correlation value of phases of time-series audio data obtained from the first and second microphones when it is determined in the determination step that a driving noise is generated;
Based on the correlation value, out of time-series audio data from each of the first and second microphones, an error in the angle of incidence of sound from the outside to the first and second microphones is set in advance. A generation step of generating time-series audio data determined to exceed the threshold value,
The time series audio data obtained from the first microphone and the time series audio data corresponding to the first microphone generated by the generation step are input to perform adaptive filter processing, and one of the stereo A first filtering step for generating audio data of a plurality of channels;
The time-series audio data obtained from the second microphone and the time-series audio data corresponding to the second microphone generated by the generation step are input to perform adaptive filter processing, and the other stereo And a second filter step for generating audio data of the channels. A method for controlling an audio processing device. A drive unit, a first microphone having a main acquisition target as a sound from outside the device, and a second microphone having a main acquisition target as drive noise from the drive unit as compared with the first microphone A program that is read and executed by a processor in a sound processing device,
The program for making the said processor perform each process of Claim 6.

Images