JP2017229086A - Sound processing device and sound processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound processing device and a sound processing program capable of suppressing a displacement of a sound image due to noise reduction processing.SOLUTION: The sound processing device includes: a calculation unit for calculating a reference relation which is a relation between a first sound collected by a first sound collecting unit and a second sound collected by a second sound collecting unit of sounds collected by a plurality of sound collecting units; and a processing unit for processing the sound collected by the plurality of sound collecting units so that the relation between the first sound and the second sound is included in a predetermined range including the reference relation calculated by the calculation unit.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a sound processing device and a sound processing program.

2. Description of the Related Art As an imaging apparatus including a plurality of sound collectors capable of stereo recording, there is an apparatus that performs noise reduction processing in accordance with the generation of driving sound such as autofocus (hereinafter abbreviated as “AF”) during moving image shooting.
In a noise suppression apparatus that suppresses noise of sound signals of a plurality of channels such as stereo, a technique for suppressing noise of a stereo component is known (see Patent Document 1 and the like).

JP 2008-283385 A

  By the way, during stereo recording, if noise reduction processing is performed in accordance with the generation of driving sound, the balance of the sound signal may change due to the noise reduction processing. As a result, the sound image is displaced and reproduced. There is a problem that it sometimes causes discomfort.

  The subject of this invention is providing the sound processing apparatus and sound processing program which can suppress the displacement of the sound image accompanying a noise reduction process.

The sound processing apparatus of the present invention includes a sound collecting unit having a first sound collecting unit that outputs first sound data and a second sound collecting unit that outputs second sound data, the first sound data, and the second sound data. A calculation unit that calculates a ratio of amplitudes in a plurality of frequency regions to sound data; and a removal unit that removes part of sound data from at least one of the first sound data and the second sound data; The ratio of the amplitude in the frequency domain of the first sound data and the second sound data from which the partial sound data has been removed from at least one of the first sound data and the second sound data before removal is determined. And a processing unit that approximates the amplitude ratio in the plurality of frequency regions.
The program of the present invention includes a plurality of processes of inputting the first sound data by the first sound collection unit and the second sound data by the second sound collection unit, and the first sound data and the second sound data. Calculating a ratio of amplitudes in the frequency domain, a process of removing part of sound data from at least one of the first sound data and the second sound data, and the part of sound from at least one of the first sound data and the second sound data The ratio of the amplitude in the frequency domain of the first sound data and the second sound data from which the data has been removed is the amplitude ratio in the frequency domain of the first sound data and the second sound data before the removal. The computer is configured to execute the process of approaching the ratio.

  ADVANTAGE OF THE INVENTION According to this invention, the sound processing apparatus and sound processing program which can suppress the displacement of the sound image accompanying a noise reduction process can be provided.

The camera which is one Embodiment of the sound processing apparatus in this invention is shown, (a) is the block block diagram, (b) is a conceptual front view. It is explanatory drawing of the noise reduction process in the sound information processing part, and its correction | amendment. It is a flowchart of the noise reduction process and correction | amendment in a sound information processing part. It is a figure explaining a sound image displacement. It is a flowchart of the noise reduction process and correction | amendment in the sound information processing part concerning 2nd Embodiment. It is a figure explaining the correction | amendment matched with the signal ratio change before and behind a noise reduction process part.

Embodiments of the present invention will be described below with reference to the drawings.
(First embodiment)
FIG. 1 shows a camera 1 which is an embodiment of a sound processing apparatus according to the present invention. FIG. 1 (a) is a block diagram of the camera 1 and FIG. 1 (b) is a conceptual front view of the camera 1.
As shown in FIG. 1A, the camera 1 includes a camera body 10 and a lens barrel 20. The camera 1 has an autofocus (hereinafter abbreviated as AF) function for automatically focusing. In addition, the camera 1 can shoot both still images and moving images, and can record sound in stereo at the same time as images during moving image shooting.

The camera body 10 includes an image sensor 11, an image processing unit 12, a stereo sound collecting device 13, a sound information processing unit 14, a storage unit 15, a control unit 16, an output unit 18, and an input unit 19. I have.
The image sensor 11 is composed of a photoelectric conversion element such as a CCD, and converts the subject image light imaged by the imaging optical system of the lens barrel 20 into an electrical signal.
The image processing unit 12 A / D converts analog image information output from the image sensor 11 and performs image processing to generate image data.

As shown in FIG. 1B, the stereo sound collecting device 13 includes a pair of left and right microphones (a left microphone 13L and a right microphone 13R). The left microphone 13L and the right microphone 13R are disposed at substantially symmetrical positions with a vertical line passing through the center of the lens barrel 20 in a state where the camera 1 is held in the horizontal position. The microphones 13L and 13R collect external sounds, detect them as analog signals, and output them to the sound information processing unit 14.
The sound information processing unit 14 A / D converts the sound signal input from the stereo sound collecting device 13 into a digital signal and performs noise reduction processing. The sound information processing unit 14 includes a noise reduction processing unit 14A and a correction unit 14B as functional units related to noise reduction processing. These will be described in detail later.

  The storage unit 15 stores the image data output from the image processing unit 12 and the sound data output from the sound information processing unit 14. The storage unit 15 may be a buffer or a memory built in the camera, or may be an external storage medium such as an SD card or HDD.

  The output unit 18 outputs image data and sound data stored in the storage unit 15. The output unit 18 is an interface for outputting sound information (electric signal) to an external device. Examples of the external device include, but are not limited to, a PC, an external speaker, and a mobile phone. However, the present invention is not limited to this, and the output unit 18 may be a rear liquid crystal and a speaker provided in the camera 1. When the output unit 18 is a speaker, the output unit 18 also includes a conversion unit that converts sound information (electric signal) into sound.

The input unit 19 is an interface for inputting data from an external device.
When exchanging (communication) data with an external device, the output unit 18 and the input unit 19 do not have to be separated, and the input unit 19 and the output unit 18 are integrated. There may be.
The external device is not limited to this, but is, for example, a PC, an external microphone, a mobile phone, or the like.

  The control unit 16 includes a CPU and the like, and each configuration of the camera 1 including each component to be described later of the lens barrel 20 according to the set imaging conditions (for example, an aperture value, an exposure value, etc.). Supervise and control elements. For example, the control unit 16 generates a drive control signal for driving an AF drive motor 22 in a lens barrel 20 described later, and outputs the drive control signal to the lens control unit 24.

The lens barrel 20 includes an imaging optical system (not shown) including a focusing lens, a camera shake correction lens, a zooming lens, and the like, an AF encoder 21, and an AF drive motor 22.
The AF encoder 21 detects the position of the focusing lens and outputs it to the lens control unit 24 and the control unit 16. The lens control unit 24 outputs the detected position information of the focusing lens to the control unit 16.
The AF drive motor 22 moves and drives the AF lens according to a drive control signal for controlling the position of the AF lens input from the lens control unit 24.

The camera 1 is controlled by the control unit 16 to perform a shooting operation when a shooting command is issued by a user pressing a shutter button (not shown).
That is, the subject image light is converted into an electrical signal by the image sensor 11 and the image data processed by the image processing unit 12 is stored (captured) in the storage unit 15. The control unit 16 performs AF control for moving and driving the AF lens via the lens control unit 24 and the AF driving motor 22 during photographing.

  At the time of moving image shooting, the image pickup device 11 converts subject image light into an electrical signal and sequentially captures it, and stores an image of a predetermined frame (number of frames) per second via the storage unit 15. Further, as described above, the sound data collected by the sound information processing unit 14 is stored (recorded) through the storage unit 15 together with the image data. During moving image shooting, AF control is performed throughout the shooting period.

  Here, the sound information collected by the stereo sound collecting device 13 is input to the sound information processing unit 14. The sound information processing unit 14 performs a reduction process on driving noise (AF driving sound) related to AF control included in the sound collected by the stereo sound collecting device 13. Then, the sound information processing unit 14 outputs the sound information on which the drive noise (AF drive sound) has been reduced to the storage unit 15.

  However, it is not limited to the above processing flow. For example, as a modification, 1) the control unit 16 temporarily stores the sound collected by the stereo sound collecting device 13 in the storage unit 15. 2) The control unit 16 performs noise reduction processing on the stored sound data. 3) The reduction processing unit 14A performs a reduction process on the sound data. 4) Next, the control unit 16 stores the reduced sound data in the storage unit 15 again. Processing flow may be used.

  Returning to the processing flow of the present embodiment, the sound information processing unit 14 will be described in detail with reference to FIGS. 2 to 4 in addition to FIG. 1 described above. FIG. 2 is an explanatory diagram of noise reduction processing and correction in the sound information processing unit 14. FIG. 3 is a flowchart of noise reduction processing and correction in the sound information processing unit 14. FIG. 4 is a diagram for explaining the sound image displacement.

As described above, the sound information processing unit 14 includes the noise reduction processing unit 14A and the correction unit 14B.
The noise reduction processing unit 14A uses the noise frequency spectrum SN and performs noise reduction processing on the AF driving sound by a spectral subtraction method. The noise frequency spectrum SN is estimated from previously stored operation noise information or sound information collected in the past, as shown in FIG. 2B as an example.

More specifically, the noise reduction processing unit 14A performs Fourier transform in units of frames obtained by dividing a digitized sound signal input from the stereo sound collector 13 (the left microphone 13L and the right microphone 13R) by a predetermined length. Perform frequency analysis by conversion.
Then, frequency spectra SIL and SIR divided into a plurality of frequency bands (f1 to f8) as shown in FIG. 2A are obtained.
The noise component is removed by subtracting the noise frequency spectrum SN shown in FIG. 2B from the frequency spectra SIL and SIR.
Further, flooring processing such as signal lower limit regulation is performed as necessary, and the frequency spectra SSL and SSR after the noise reduction processing shown in FIG. 2C are output to the correction unit 14B.

The noise reduction processing by the noise reduction processing unit 14A is performed for each frame with respect to the frame including the AF driving sound.
The detection of the frame including the AF driving sound is performed, for example, based on the output of the AF encoder 21 that detects the position of the AF lens (the output of the AF encoder 21 changes when the AF lens moves).
Note that the shaded portions for the frequency spectra SIL and SIR in FIG. 2A refer to the frequency spectrum of only the target sound that does not include the AF driving sound.

Here, the noise reduction processing by the noise reduction processing unit 14A is performed independently on the sound signals from the left and right microphones (the left microphone 13L and the right microphone 13R) in the stereo sound collecting device 13.
However, since the left microphone 13L and the right microphone 13R are arranged substantially symmetrically with respect to the lens barrel 20, the input AF noise (AF drive sound) is the same and the noise frequency spectrum SN is the same. Is used.
Note that the left microphone 13L and the right microphone 13R are not limited to a form in which the left microphone 13L and the right microphone 13R are arranged substantially symmetrically with respect to the lens barrel 20, and may be asymmetrical with respect to the optical axis.

The correction unit 14B
A left / right signal ratio (pre-processing ratio, reference ratio) of each frequency band (f1 to f8) of the frequency spectrum (pre-processing spectrum) SIL, SIR before the noise reduction processing by the noise reduction processing unit 14A;
The left / right signal ratio (post-processing ratio, first relationship) in each frequency band (f1 to f8) of the frequency spectrum (processed spectrum) SSL, SSR after the noise reduction processing by the noise reduction processing unit 14A,
Are compared.

Based on the comparison result, the correction unit 14B corrects the post-processing ratio RS so that it substantially coincides with the pre-processing ratio RI in each frequency band, thereby correcting the post-correction ratio RC (second relationship). The frequency spectrum (corrected spectrum) SCL, SCR is obtained.
Then, the correction unit 14B outputs the corrected spectra SCL and SCR to the storage unit 15.

Hereinafter, the correction by the correction unit 14B will be described in more detail with reference to FIG.
(Spectrum before processing)
As shown in FIG. 2A, each frequency band (f1) in the frequency spectrum (pre-processing spectrum (L)) of the sound (sound signal L) input from the left microphone 13L before the noise reduction processing by the noise reduction processing unit 14A. ˜f8) is assumed to be SIL1 to SIL8.
The amplitudes of the frequency bands (f1 to f8) in the frequency spectrum (pre-processing spectrum (R)) of the sound (sound signal R) input from the right microphone 13R are SIR1 to SIR8.
The left / right signal ratio of the amplitude for each frequency band (f1 to f8) of the spectrum before processing (hereinafter, this left / right signal ratio is referred to as the ratio before processing) is RI1 = SIL1 / SIR1,..., RI8 = SIL8 / SIR8.

(Processed spectrum)
Further, as shown in FIG. 2C, the amplitude of each frequency band (f1 to f8) in the frequency spectrum (processed spectrum (L)) of the sound signal L after the noise reduction processing by the noise reduction processing unit 14A is expressed as SSL1. ˜SSL8.
The amplitudes of the frequency bands (f1 to f8) in the frequency spectrum (processed spectrum (R)) of the sound signal R after the noise reduction processing by the noise reduction processing unit 14A are defined as SSR1 to SSR8.
The left / right signal ratio of the amplitude for each frequency band (f1 to f8) of the processed spectrum (hereinafter, this left / right signal ratio is referred to as the processed ratio) is RS1 = SSL1 / SSR1,..., RS8 = SSL8 / SSR8.

(Corrected spectrum)
The correction unit 14B includes a pre-processing ratio (RI1 to RI8), a post-processing ratio (RS1 to RS8),
Are compared in each frequency band (f1 to f8).
And the correction | amendment part 14B correct | amends so that post-process ratio (RS1-RS8) may become equal to pre-process ratio (RI1-RI8), respectively, as shown in FIG.2 (d). Then, a corrected spectrum (L) (SCL1 to SCL8) and a corrected spectrum (R) (SCR1 to SCR8) are obtained.

  Here, methods for obtaining a corrected spectrum include an increase correction, a decrease correction, and an average correction.

(Increase correction)
In the increase correction, the amplitude of either the post-processing spectrum (L) or the post-processing spectrum (R) is largely corrected to make the post-processing ratio RS coincide with the pre-processing ratio RI.
1. When the post-process ratio RSn is larger than the pre-process ratio RIn (1) Obtain the corrected spectrum (L) (fixed L)
The processed spectrum (L) SSLn is set as a corrected spectrum (L) SCLn (SCLn = SSLn).
(2) Obtaining the corrected spectrum (R) Then, the corrected spectrum (R) SCRn is obtained so that the ratio of the corrected spectrum (L) SCLn obtained in (1) is equal to the pre-processing ratio RIn.
At this time, since the post-processing ratio RSn is larger than the pre-processing ratio RIn, the post-processing spectrum (R) so as to satisfy the pre-processing ratio with respect to the corrected spectrum (L) SCL having the same value as the post-processing spectrum (L). ) When the SSR is corrected, the corrected spectrum (R) SCRn becomes larger than the processed spectrum (R) SSRn (SCRn> SSRn).

2. When the post-process ratio RSn is smaller than the pre-process ratio RIn (1) Obtain the corrected spectrum (R) (fixed R)
The post-processing spectrum (R) SSRn is used as the corrected spectrum (R) SCRn (SCRn = SSRn).
(2) Obtaining the corrected spectrum (L) Then, the corrected spectrum (L) SCLn is obtained so that the ratio to the corrected spectrum (R) SCRn obtained in (1) is equal to the pre-processing ratio RIn.
At this time, SCLn> SSLn.
Note that “n” is a number (1 to 8) indicating each frequency band.

  In the above increase correction, the amplitude of the corrected spectrum is equal to or smaller than the amplitude before the noise reduction processing in the present embodiment, but is not limited thereto. For example, when the spectrum after the noise reduction process is once amplified and the amplitude of the spectrum is corrected, the amplitude of the spectrum after the correction may be larger than the amplitude before the noise reduction process.

3. Specific Example As a specific example, as shown in FIGS. 2A to 2E, there is a difference between the left and right frequency bands f3 in the frequency spectrum, and the left and right (L, R) amplitude values in the frequency band f3 are noise reduced. It is assumed that before the processing (6, 3), the noise reduction processing is changed to (4, 1).
In this case, the pre-processing ratio RI3 is different from 6/3 = 2 and the post-processing ratio RS3 is different from 4/1 = 4. In order to make the right / left signal ratio (corrected ratio) RC3 after correction equal to the pre-processing ratio RI3, the right (R) amplitude value after noise reduction processing is corrected from 1 to 2.
As a result, the amplitude values of L and R after correction are (4, 2), which is equal to the pre-processing ratio 2.
Such increase correction can suppress the deterioration of the target sound, and is suitable when there is a human sound or when the target sound is large and noise is not a concern.

(Decrease correction)
In the reduction correction, the amplitude of either the processed spectrum (L) or the processed spectrum (R) is corrected to be small so that the processed ratio RS matches the preprocessed ratio RI.
1. When the post-process ratio RSn is larger than the pre-process ratio RIn (1) Obtain the corrected spectrum (R) (fixed R)
The post-processing spectrum (R) SSRn is used as the corrected spectrum (R) SCRn (SCRn = SSRn).
(2) Obtaining the corrected spectrum (L) Then, the corrected spectrum (L) SCLn is obtained so that the ratio to the corrected spectrum (R) SCRn obtained in (1) is equal to the pre-processing ratio RIn.
At this time, SCLn <SSLn.

2. When the post-processing ratio RSn is smaller than the pre-processing ratio RIn (1) Obtain the corrected spectrum (L) (fixed L)
The processed spectrum (L) SSLn is set as a corrected spectrum (L) SCLn (SCLn = SSLn).
(2) Obtaining the corrected spectrum (R) Then, the corrected spectrum (R) SCRn is obtained so that the ratio of the corrected spectrum (L) SCLn obtained in (1) is equal to the pre-processing ratio RIn.
At this time, SCRn <SSRn.
Such reduction correction is suitable for a case where the noise reduction effect is high and there is no human voice.

  In the above reduction correction, the amplitude of the corrected spectrum is equal to or smaller than the amplitude after the noise reduction processing in the present embodiment, but is not limited thereto. For example, when the spectrum amplitude after correction after the noise reduction processing is once amplified, the amplitude of the corrected spectrum may be larger than the amplitude after the noise reduction processing. Depending on the degree of amplification, the amplitude before the noise reduction process may be larger.

(Average correction)
The average correction is a compromise between the above-described increase correction and decrease correction. The sum of the amplitudes in the left and right frequency spectra after the noise reduction processing is distributed and corrected so that the post-processing ratio RSn = the pre-processing ratio RIn.

  Each of the above correction methods may be configured to be applied by switching the correction method according to the object to be corrected and the situation. Switching of the correction method can be performed by using face recognition or person recognition from sound recognition or imaging information, which are publicly known techniques. For example, it is configured to apply an increase correction when a person is photographed large, when a person's sound input is recognized and when the input is large, and to apply a decrease correction in other cases where the person is not recognized Just do it.

  In the present embodiment, the example in which the post-processing ratio RS (first relationship) matches the pre-processing ratio RI (reference relationship) has been described. However, the present embodiment is not limited to this. The corrected ratio RC does not necessarily have to be RC = pre-processing ratio RI, and RC may be within a predetermined range including the pre-processing ratio RI. The predetermined range of the post-correction ratio RC is a range that is closer to the pre-processing ratio RI than the post-processing ratio RS.

That is, if the sound of the post-processing ratio RS (first relation) can be heard, the localization of the sound of the post-correction ratio RC is more than the localization of the sound of the first relation (post-processing ratio RS). It is close to the sound localization of the pre-processing ratio RI.
The predetermined range of the corrected ratio RC may be determined as a range in which the corrected ratio RC is included within plus or minus 5% of the pre-processing ratio RI.

  Further, the predetermined range of the corrected ratio RC may be a range in which the position of the corrected sound image is included within ± 30 ° with respect to the position of the sound image before the noise reduction processing. As described above, the predetermined range of the corrected ratio RC may be determined as a range in which the position of the corrected sound image is included in the range of the predetermined angle. Further, the predetermined range of the corrected ratio RC may be a range in which the position of the corrected sound image is narrower than plus / minus 30 ° and included within plus / minus 15 °.

Next, the flow of noise reduction processing and correction control by the noise reduction processing unit 14A and the correction unit 14B will be described with reference to the flowchart shown in FIG. In FIG. 3 and the following description, step is also abbreviated as “S”.
The noise reduction processing by the noise reduction processing unit 14A and the correction by the correction unit 14B start based on AF drive information such as the output of the AF encoder 21 as described above. That is, it functions only during AF driving.

In the noise reduction processing and correction control, first, the correction unit 14B calculates a pre-processing ratio RI of the frame before the noise reduction processing by the noise reduction processing unit 14A (S301), and the noise reduction processing unit 14A performs the noise reduction processing. (S302).
Next, the correction unit 14B calculates the post-processing ratio RS after the noise reduction processing by the noise reduction processing unit 14A (S303), and compares the post-processing ratio RS with the pre-processing ratio RI (S304).
If it is determined in step 304 that they are not equal (No), the correction unit 14B corrects the signal after the noise reduction processing (S305). On the other hand, if it is determined in step 304 that both are equal (Yes), the control is terminated without correction.

As described above, the correction unit 14B corrects the post-processing ratio in each frequency band of the frequency spectrum so as to substantially match the pre-processing ratio.
Thereby, when the noise reduction process is performed on the stereo signal in accordance with the noise generation timing, it is possible to suppress the sound image displacement of the target sound due to the noise reduction process.

That is, as shown in the conceptual diagram of FIG. 4, even if the sound image of the processed sound that is not corrected only by the noise reduction process moves greatly relative to the sound image position of the target sound viewed from the person M, the sound image is moved by the correction. Can be kept small. As a result, it is possible to prevent an uncomfortable sound image displacement such that the image and the sound image suddenly deviate during noise reduction processing (AF driving).
In the present embodiment, the sound processing may not be performed in the entire frequency band, and the sound processing may be performed on a part of the frequency bands. Examples of some frequency bands include a frequency band in which noise is particularly detected, an audible frequency band, and a frequency band in which extreme high and low frequencies are cut.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.
FIG. 5 is a flowchart of noise reduction processing and correction in the sound information processing unit 14 according to the second embodiment. Similar to FIGS. 2 and 3, the frequency band f <b> 3 in the frequency spectrum will be described.
In the second embodiment, the left / right signal ratio (pre-processing ratio) as a reference for correction is acquired from a portion (frame) where no noise (AF drive sound) is generated. The mechanical configuration is exactly the same as that of the first embodiment described above, and a description thereof is omitted. Refer to FIG. 1 for the reference numerals of the constituent elements in the following description.
In the second embodiment, the left / right signal ratio as a reference for correction is obtained from a portion immediately before or after the noise reduction processing portion. Note that real-time processing (sequential processing) is possible when the immediately preceding signal ratio is used, but when the immediately following signal ratio is used, sequential processing is difficult and can be applied only to post-processing.

In this way, by obtaining the left / right signal ratio from the portion where noise (AF driving sound) is not mixed and using this as the reference for correction, the right / left ratio of the target sound can be obtained without being affected by noise. .
However, when the time variation of the target sound is large, the spectrum (right / left signal ratio) of the target sound may change greatly between the noise reduction processing part and the noise reduction processing part, which is actually occurring. It may not be possible to follow the movement of the target sound. In order to prevent this, the right / left signal ratio used as a reference for correction is determined as the right / left signal ratio obtained immediately before the noise reduction processing portion, and the left / right signal ratio of the noise reduction processing portion as in the first embodiment described above. It is preferable that any one of these can be selected.

The noise reduction processing and correction in the case of selecting and applying the right / left signal ratio as the correction reference will be described with reference to the flowchart shown in FIG.
First, the correction unit 14B calculates the left / right signal ratio RIb of the frame immediately before starting the noise reduction process based on the AF drive information such as the output of the AF encoder 21 (S501), and after entering the noise reduction process, the noise The left / right signal ratio RIa of each frame before the noise reduction processing by the reduction processing unit 14A is calculated (S502).

Then, the difference (absolute value) between the signal ratio RIb and the signal ratio RIa is compared with a predetermined threshold A (S503).
If it is determined in step 503 that the difference between the signal ratio RIb and the signal ratio RIa is equal to or less than the threshold A (Yes), the signal ratio RIb is set as the reference ratio RI (S504). On the other hand, if it is determined in step 503 that the difference between the signal ratio RIb and the signal ratio RIa exceeds the threshold A (No), the signal ratio RIa is set as the reference ratio RI (S505).

Thereafter, noise reduction processing is performed by the noise reduction processing unit 14A (S506), and then the correction unit 14B calculates a left / right signal ratio RS after the noise reduction processing by the noise reduction processing unit 14A (S507), and the noise reduction is performed. The processed right / left signal ratio RS is compared with the left / right signal ratio RI before noise reduction processing (S508).
If it is determined in step 508 that the two are not equal (No), the correction unit 14B corrects the signal after the noise reduction processing (S509). On the other hand, if it is determined in step 508 that both are equal (Yes), the control is terminated without correction.

In the above configuration, the left / right signal ratio RIb obtained immediately before the noise reduction processing portion is compared with the left / right signal ratio RIa of the noise reduction processing portion, and if the difference is small, it is determined that the movement of the sound image of the target sound is small. When the signal ratio RIb that is not affected by noise is adopted as the reference signal ratio RI and the difference is larger than a predetermined amount, it is determined that the movement of the sound image of the target sound is large, and the signal ratio RIa is determined as the reference signal ratio RI. Is to be adopted.
According to such a configuration, when the movement of the target sound image is small, the continuity of the sound image immediately before the processing portion and the processing portion can be maintained, and when the movement of the target sound image is large, the sense of discomfort is maintained. Reproducible smooth sound image movement.

  In addition, post processing (processing that is read after recording once instead of sequential processing) is performed, and the left and right signal ratios of the portion (frame) immediately before and after the noise reduction processing portion are respectively determined and corresponded to the rate of change. Thus, the left / right signal ratio may be changed. In other words, if the left / right signal ratio is significantly different between immediately before and after the noise reduction processing part, it is considered that the sound source has moved left and right, so that it corresponds to the change in the left / right signal ratio immediately before and after the noise reduction processing part. The process of moving the sound image is performed.

FIG. 6 is an explanatory diagram of such processing.
As shown in FIG. 6A, when the frames 4 to 10 are noise reduction processing frames, the frame 3 is the immediately preceding portion and the frame 11 is the immediately following portion.
In FIG. 6B, SFL3 is the left spectrum immediately before (frame 3), SFR3 is the right spectrum immediately before (frame 3), SFL11 is the left spectrum immediately after (frame 11), and SFR11 is the right spectrum immediately after (frame 11). It is.

Here, for example, when looking at the frequency band f3,
Left side: f3 (amplitude 1.5) of SFL11 is smaller than f3 (amplitude 3) of SFL3.
Right: f3 (amplitude 3) of SFR11 is greater than f3 (amplitude 1) of SFR3.
This indicates that the sound source is moving from the left side to the right side during the noise reduction processing frames 4 to 10.

Therefore, the left / right signal ratio (pre-processing ratio) in the noise reduction processing frames 4 to 10 is as shown in FIG. 6C from the right / left signal ratio (3/1 = 3) immediately before (frame 3). Then, the signal ratio that is the reference value for correction is obtained so as to continuously change to the right / left signal ratio (1.5 / 3 = 0.5) immediately after (frame 11).
Specifically, based on the immediately preceding and immediately following values (3 and 0.5) and the immediately preceding and immediately following frames (seven), the left and right signal ratio values in each frame are obtained. Specifically, correction is performed so as to decrease the left / right ratio by a value of 2.5 / 8 between frames 4-10.
The same processing is performed for frequency bands other than f3.
As a result, during the process from immediately before the process, the right / left signal ratio immediately after the process changes continuously, the movement of the sound image becomes smooth, and the uncomfortable feeling can be reduced.

As described above, this embodiment has the following effects.
(1) The correction unit 14B in the camera 1 substantially matches the left / right signal ratio in each frequency band of the frequency spectrum after the noise reduction processing with the left / right signal ratio in each frequency band of the frequency spectrum before the noise reduction processing. To correct. Thereby, when the noise reduction process is performed on the stereo signal in accordance with the noise generation timing, the sound image displacement of the target sound caused by the noise reduction process can be suppressed. As a result, it is possible to prevent a sense of incongruity due to sound image displacement during noise reduction processing (AF driving).

(Deformation)
The present invention is not limited to the above-described embodiment, and various modifications and changes as described below are possible, and these are also within the scope of the present invention.
(1) In the above embodiment, the present invention is applied to a camera as a sound processing apparatus. However, the present invention is not limited to this, and may be provided as a program that causes a computer to function as each of the above components.

(2) In the above embodiment, the present invention is configured to reduce noise caused by AF driving sound in a camera. However, the present invention is not limited to this, and can also be applied to the reduction of operating noise of zooming and blur correction devices. Furthermore, the present invention can be applied not only to cameras but also to optical equipment having a recording function.

(3) In the present embodiment, an example in which the sound information processing unit 14 is included in the camera body 10 has been described. However, the present invention is not limited to this, and after recording with a stereo microphone provided in the camera, Data may be transmitted and reduction processing may be performed by the sound processing device. That is, the part that collects the sound and the part that performs the sound reduction process may be separated.
In this case, as an example, processing is performed in the following flow.
Ambient sounds are recorded with a stereo microphone on the camera.
The sound recorded by the stereo microphone is converted into sound data and stored in the storage unit.
When an operation of a function such as AF is performed at the time of recording, sound data obtained by recording surrounding sounds and the timing at which the function operation (for example, AF operation) of the camera is performed are stored in association with each other. .
Next, the sound data and the operation timing stored in the storage unit are output to a separate sound processing device, such as a PC, via the output unit.
The sound processing apparatus includes a control unit, a storage unit, and a noise reduction processing unit (hereinafter referred to as an SP control unit, an SP storage unit, and an SP noise reduction processing unit).
The SP control unit stores the sound data, operation timing, and sound data input from the camera via the input unit in the SP storage unit.
The SP control unit outputs the sound data stored in the SP storage unit to the SP reduction processing unit, and the SP reduction processing unit reduces noise such as AF sound on the sound data.
Note that the sound reduction processing is performed based on the operation timing of the function stored together with the sound data. Thereafter, the SP control unit stores the reduced sound data in the SP storage unit. In this way, reduction processing may be performed on the sound data.

  In addition, although embodiment and a deformation | transformation form can also be used in combination as appropriate, detailed description is abbreviate | omitted. Further, the present invention is not limited to the embodiment described above.

  1: Camera, 13: Stereo sound collector, 13L: Left microphone, 13R: Right microphone, 14: Sound information processing unit, 14A: Noise reduction processing unit, 14B: Correction unit, 20: Lens barrel, 21: AF encoder 22: AF driving motor, SIL, SIR: frequency spectrum before noise reduction processing, RI: pre-processing ratio, SN: noise frequency spectrum, SSL, SSR: frequency spectrum after noise reduction processing, RS: post-processing ratio, SCL, SCR: Frequency spectrum after correction

Claims (4)

A sound collection unit having a first sound collection unit that outputs first sound data and a second sound collection unit that outputs second sound data;
A calculation unit that calculates a ratio of amplitudes in a plurality of frequency regions between the first sound data and the second sound data;
A removing unit that removes part of the sound data from at least one of the first sound data and the second sound data;
The ratio of the amplitude in the frequency domain of the first sound data and the second sound data from which the partial sound data has been removed from at least one of the first sound data and the second sound data before removal is determined. A sound processing apparatus comprising: a processing unit that approximates an amplitude ratio in the plurality of frequency regions. The sound processing device according to claim 1, wherein the sound data removed by the removing unit is sound data generated by driving a mechanism of a camera lens. The sound processing apparatus according to claim 2,
A detection unit that detects sound data generated by driving the mechanism of the camera lens;
The sound processing device, wherein the processing unit removes the sound data from at least one of the first sound data and the second sound data based on the sound data detected by the detection unit. A process of inputting the first sound data by the first sound collection unit and the second sound data by the second sound collection unit;
A process of calculating a ratio of amplitudes in a plurality of frequency regions between the first sound data and the second sound data;
A process of removing a part of sound data from at least one of the first sound data and the second sound data;
The ratio of the amplitude in the frequency domain of the first sound data and the second sound data from which the partial sound data has been removed from at least one of the first sound data and the second sound data before removal is determined. A program for causing a computer to execute a process of approaching an amplitude ratio in the plurality of frequency regions.

Images