JP2016018082A - Voice processing device and method, as well as imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain effective noise elimination as balancing both of a computation load and sound quality.SOLUTION: A voice processing device is configured to: acquire a voice signal of a plurality of channels including a first channel and a second channel; detect a phase difference between the channels; generate a first prediction voice signal for replacing the voice signal in a noise immixture segment, using the voice signal of the first channel in at least one of a segment in front of a noise immixture segment in the acquired voice signal and a segment at the back thereof, and replace the voice signal of the first channel in the noise immixture segment with the first prediction voice signal; and next, generate a second prediction voice signal by correcting the first prediction voice signal on the basis of the phase difference, and replace the voice signal of the second channel in the noise immixture segment with the second prediction voice signal.SELECTED DRAWING: Figure 8

Description

  The present invention relates to a voice processing technique.

  Some recent digital cameras have not only a still image shooting function but also a function of shooting a moving image with audio signal recording. At the time of moving image shooting, the drive unit of the imaging apparatus operates such as driving a focus lens according to a change in shooting state and driving a diaphragm mechanism according to a change in luminance of a subject. There is a problem that the operation sound of the drive unit is mixed as noise in the recorded sound. Conventionally, various methods have been disclosed for driving noise reduction processing.

  For example, Patent Document 1 proposes a technique for predicting a voice that does not include noise from voice signals before and after a driving noise generation section and replacing the data of the driving noise generation section with the predicted voice data.

JP 2008-0538802 A

  However, when a plurality of microphones that acquire audio signals are provided and a multi-channel audio recording function is provided, there are the following problems. For example, when noise is removed by prediction processing for stereo audio having Lch and Rch, prediction processing must be performed for both Lch and Rch audio signals. However, it is computationally intensive to perform prediction processing for audio signals of 2ch or more. As the number of microphones increases, the calculation load increases in proportion to the number of microphones.

  The present invention realizes effective noise removal while achieving both computational load and sound quality.

  According to one aspect of the present invention, an acquisition means for acquiring audio signals of a plurality of channels including the first channel and the second channel, and a position between the audio signal of the first channel and the audio signal of the second channel. Phase difference detection means for detecting a phase difference, and first predicted speech for replacing the speech signal in the noise-mixed section using the speech signal of the first channel in at least one section before and after the noise-mixed section in the speech signal A first replacement means for generating a signal and replacing the first channel speech signal in the noise-mixed section with the first predicted speech signal; and correcting the first predicted speech signal with the phase difference to generate a second signal. And a second replacement means for generating a predicted speech signal and replacing the second channel speech signal in the noisy section with the second predicted speech signal. Processing apparatus is provided.

  ADVANTAGE OF THE INVENTION According to this invention, effective noise removal can be performed, making calculation load and sound quality compatible.

1 is an overall view of an imaging apparatus according to a first embodiment. 1 is a block diagram of an imaging apparatus according to a first embodiment. FIG. 3 is a detailed view of the microphone according to the first embodiment. Explanatory drawing of the audio | voice prediction processing method. The schematic diagram of an audio | voice signal when object sound arrives from the front. The schematic diagram of an audio | voice signal when object sound arrives from the front. The flowchart of the audio | voice recording operation | movement in 1st Embodiment. The schematic diagram of the audio | voice signal and correlation value which showed the prediction process operation | movement in 1st Embodiment. The flowchart of the audio | voice recording operation | movement in 2nd Embodiment.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited to the following embodiment, It shows only the specific example advantageous for implementation of this invention. Moreover, not all combinations of features described in the following embodiments are indispensable for solving the problems of the present invention.

<First Embodiment>
FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus 100 to which a sound processing apparatus of the present invention is applied. The imaging apparatus 100 is, for example, a digital single-lens reflex camera, and includes a camera body 101 and a photographic lens 102 that can be attached to and detached from the camera body 101. The taking lens 102 has an imaging optical system 104 having an optical axis 105 in a lens barrel 103. The imaging optical system 104 includes a focus lens group, a camera shake correction lens unit, a lens driving unit 106 that drives a diaphragm mechanism, and a lens control unit 107 that controls the lens driving unit 106. The taking lens 102 is electrically connected to the camera body 101 via a lens mount contact 108. A subject optical image incident from the front of the photographing lens 102 enters the camera body 101 through the optical axis 105, is partially reflected by the main mirror 110 formed of a half mirror, and forms an image on the focusing screen 117. . The user can visually recognize the optical image formed on the focusing screen 117 from the eyepiece window 112 through the pentaprism 111.

  On the other hand, the photometric sensor 116 detects the brightness of the optical image formed on the focusing screen 117. The subject optical image transmitted through the main mirror 110 is reflected by the sub-mirror 113 and enters the focus detection unit 114, and is used for the focus detection calculation of the subject image. When a release button (not shown) in the camera body 101 is operated and an imaging start command is issued, the main mirror 110 and the sub mirror 113 are retracted from the imaging optical path so that the subject optical image is incident on the image sensor 118. Light rays incident on the focus detection unit 114, the photometric sensor 116, and the image sensor 118 are converted into electrical signals, which are sent to the camera control unit 119 to control the camera system.

  During moving image shooting, the sound of the subject is input to the microphone 115 and sent to the camera control unit 119, and recording processing is performed in synchronization with the subject optical image signal incident on the image sensor 118. The microphone 115 includes a configuration for acquiring audio signals of a plurality of channels. For example, as shown in FIG. 3, the microphone 115 configures a two-channel microphone array of an Lch 115a that is the first channel and an Rch 115b that is the second channel. An accelerometer 120 is installed on the inner surface of the camera body 101 near the microphone 115. The accelerometer 120 detects vibration that propagates through the photographing lens 102 and the camera body 101 when the lens driving unit 106 drives a driving mechanism such as a focus lens group, a camera shake correction lens unit, and a diaphragm mechanism. The camera control unit 119 analyzes the vibration detection result and calculates a noise mixed section.

  FIG. 2 is a block diagram for explaining electrical control of the imaging apparatus 100. The imaging apparatus 100 includes an imaging system, an image processing system, an audio processing system, a recording / reproducing system, and a control system. The imaging system includes a photographic lens 102 and an imaging element 118. The image processing system includes an A / D converter 131 and an image processing circuit 132. The audio processing system includes a microphone 115 and an audio signal processing circuit 137. The recording / reproducing system includes a recording processing circuit 133 and a memory 134. The control system includes a camera control unit 119, a focus detection unit 114, a photometric sensor 116, an operation detection unit 135, a lens control unit 107, and a lens driving unit 106. The lens driving unit 106 includes a focus lens driving unit 106a, a blur correction driving unit 106b, and an aperture driving unit 106c.

  The imaging system is an optical processing system that forms an image of light from an object on the imaging surface of the imaging element 118 via the imaging optical system 104. During a preliminary shooting operation such as aiming, a part of the light beam is also guided to the focus detection unit 114 via a mirror provided on the main mirror 110. Further, as will be described later, by appropriately adjusting the imaging optical system by the control system, an object light of an appropriate amount of light is exposed to the imaging element 118 and a subject image is formed in the vicinity of the imaging element 118.

  The image processing circuit 132 is a signal processing circuit that processes an image signal of the number of pixels of the image sensor received from the image sensor 118 via the A / D converter 131. Specifically, the image processing circuit 132 includes a white balance circuit, a gamma correction circuit, an interpolation calculation circuit that performs high resolution by interpolation calculation, and the like.

  The audio processing system performs appropriate processing on the signal from the microphone 115 by the audio signal processing circuit 137 to generate a recording audio signal. The generated recording audio signal is recorded by being linked to an image by a recording processing unit described later.

  The accelerometer 120 is connected to the camera control unit 119 via the accelerometer processing circuit 138. The acceleration signal of the vibration of the camera body 101 detected by the accelerometer 120 is subjected to amplification, high-pass filter processing, and low-pass filter processing in the accelerometer processing circuit 138 so as to extract a target frequency.

  The recording processing circuit 133 outputs an image signal to the memory 134 and generates and stores an image to be output to the display unit 136. In addition, the recording processing circuit 133 performs compression and recording processing of images, moving images, sounds, and the like using a predetermined method.

  The camera control unit 119 generates and outputs a timing signal at the time of imaging. The focus detection unit 114 and the photometric sensor 116 detect the focus state of the imaging device and the luminance of the subject, respectively. The lens control unit 107 adjusts the optical system by appropriately driving the lens according to the signal from the camera control unit 119.

  The control system further controls the imaging system, the image processing system, and the recording / reproducing system in conjunction with external operations. For example, the operation detection unit 135 detects that a shutter release button (not shown) is pressed, and controls driving of the image sensor 118, operation of the image processing circuit 132, compression processing of the recording processing circuit 133, and the like. Further, the state of each segment of the display unit 136 that displays information on an optical finder, a liquid crystal monitor, or the like is controlled.

  The adjustment operation of the optical system will be described. A focus detection unit 114 and a photometric sensor 116 are connected to the camera control unit 119, and an appropriate focus position and aperture position are obtained based on these signals. The camera control unit 119 issues a command to the lens control unit 107 via the lens mount contact 108, and the lens control unit 107 appropriately controls the focus lens driving unit 106a and the aperture driving unit 106c. In addition, a camera shake detection sensor (not shown) is connected to the lens control unit 107, and in the camera shake correction mode, the shake correction driving unit 106b is appropriately controlled based on a signal from the camera shake detection sensor. At the time of moving image shooting, since the main mirror 110 and the sub mirror 113 are retracted from the optical path that enters the image sensor 118 from the optical axis 105, the subject optical image does not enter the focus detection unit 114 and the photometric sensor 116. Therefore, the camera control unit 119 uses the driving amount of the focus lens driving unit 106a and the continuous image information exposed by the image sensor 118 to change the focus state of the imaging optical system by a contrast type focus detection method called a so-called hill-climbing method. Adjust. Further, the brightness of the subject image is calculated using the image information exposed by the image sensor 118, and the aperture state is adjusted.

  Next, a noise reduction processing method based on speech prediction will be described with reference to FIG. In the noise reduction process to be described, a prediction process for predicting an audio signal in a driving noise mixing period is performed using audio signals before and after the driving noise mixing period.

  FIG. 4 is a diagram for explaining a prediction process performed on an audio signal of one channel acquired from one microphone. 4A, 4B, 4D, 4E, and 4G, the horizontal axis represents time, and the vertical axis represents the signal level. In FIG. 4C, the vertical axis represents the correlation value, and the horizontal axis represents the correlation value calculation position synchronized with the audio signal of FIGS. FIG. 4A shows a signal in which noise generated by driving the diaphragm is mixed in the subject sound signal. FIG. 4B shows an audio signal in a correlation value reference section for performing pitch detection. FIG. 4C shows a correlation value obtained from the correlation value reference section and the correlation value calculation section, and is synchronized with the correlation value calculation section of FIG. FIG. 4D shows the predicted speech signal in the noise-mixed section using the detected pitch. FIG. 4 (e) shows a signal obtained by multiplying the predicted speech signal of FIG. 4 (d) by a triangular window function. Hereinafter, the front in terms of time and the rear in terms of time will be referred to as “rear” with respect to the noise-containing section.

  In the prediction process, the audio signal processing circuit 137 detects the noise-containing section shown in FIG. 4A using the accelerometer 120 or the like, and the audio signal processing circuit 137 discards the signal in the noise-containing section. The detection of the noise mixed section may be performed by analyzing the frequency of the input voice using the characteristic frequency component of the driving noise. Alternatively, it is possible to detect a noise mixed section by obtaining the timing of a drive command to the lens driving unit. That is, it is possible to detect the driving period of the lens driving unit as a noise mixing section. Next, the audio signal processing circuit 137 performs pitch detection from the correlation value of the signal in the front section of the noise mixed section. As shown in FIG. 4 (a), the sound signal has a relatively high repeatability when focusing on a short-time region, so that the sound signal in the front section of the noise-mixed section is repeatedly reproduced. Thus, the speech signal prediction process is performed. When the correlation value is calculated from the signal in the correlation value reference section and the signal in the correlation value calculation section in FIG. 4A, the position (time length) at which the correlation value is maximized immediately before the noise mixing section in the voice signal is the voice. The pitch period. However, since it is obvious that the correlation value becomes maximum at a position where the correlation value reference interval is temporally synchronized with the correlation value calculation interval, the maximum correlation value is the length of the pitch threshold interval from the noise elimination interval. The search is performed from the correlation maximum value search section shown in FIG. If the pitch threshold interval is the reciprocal of the maximum value of the fundamental frequency included in the target audio signal, a pitch shorter than the repetitive pitch of the audio to be obtained is not erroneously detected. For example, since the basic frequency of Japanese is up to about 400 Hz, the pitch threshold interval may be set to 2.5 msec.

  The audio signal processing circuit 137 generates the first signal by duplicating the audio signal of the first channel in the front section of the noise mixing section. Similarly, the second signal is generated by duplicating the audio signal of the first channel in the rear section of the noise-mixing section. And a 1st prediction audio | voice signal is produced | generated by synthesize | combining a 1st and 2nd audio | voice signal. Specific examples are shown below.

  As shown in FIG. 4 (d), the audio signal processing circuit 137 repeatedly duplicates the detected audio signal in the pitch interval until the end of the prediction interval (noise mixing interval) to generate the first signal. Hereinafter, this first signal is referred to as a “pre-window prediction signal from the front”. Next, as shown in FIG. 4 (e), a forward prediction signal is completed by applying a triangular window function to the created pre-window prediction signal from the front. Hereinafter, the prediction signal at this stage is referred to as a “prediction signal after windowing from the front”. At this time, the window function wf (t) is a function represented by wf (n) = (N−n) / N when the data immediately after the start of prediction is n = 0 when the number of data in the prediction section is N + 1. It is.

  Similarly to the above, the audio signal processing circuit 137 performs pitch detection in the rear section of the noise-mixed section, and repeatedly generates the second signal by replicating the audio signal in the detected pitch section to the start end of the prediction section. This second signal is referred to as a “pre-window prediction signal from behind”. Next, as shown in FIG. 4 (f), the prediction signal before windowing from the rear is multiplied by a triangular window function opposite to that shown in FIG. Prediction signal). The triangular window function wr (n) multiplied by the prediction signal before windowing from the rear is symmetric with the prediction from the front and is expressed by wr (n) = n / N.

  The audio signal processing circuit 137 generates a first predicted audio signal by adding the windowed predicted signal from the front and the windowed predicted signal from the rear. Then, the audio signal processing circuit 137 replaces the audio signal in the noise-mixed section with the first predicted audio signal. As a result, the noise in the noise mixing section is reduced. FIG. 4G shows an example of the resulting signal waveform. As described above, the first predicted speech signal is generated by synthesizing the first signal and the second signal by cross-fading. According to such a crossfade, it is possible to smoothly connect voices at the connection portion between the prediction signal from the front and immediately after the noise mixing interval, and at the connection portion between the prediction signal from the rear and immediately before the noise mixing interval. .

  When two microphones are provided as in the image pickup apparatus of FIG. 1, noise reduction is performed by the above-described prediction processing on the two-channel audio signals from each microphone. In this case, in order to reduce the calculation load of the prediction process, it is conceivable to apply the first predicted speech signal obtained from the speech signal of one channel as it is as the predicted speech signal of the other channel.

  FIG. 5 shows an example in which the first predicted speech signal obtained for the Lch speech signal is used as it is as the predicted speech signal in the Rch noise section. FIG. 5A shows an Lch audio input signal, and FIG. 5B shows an Rch audio input signal. In both cases, signals in the noise-mixing section are omitted. FIG. 5C shows the Lch first predicted speech signal, and FIG. 5D shows the speech signal in which the first predicted speech signal of FIG. 5C is applied as it is as the predicted speech signal in the Rch noise-mixed section.

  As shown in FIG. 5, when the sound from the subject comes from the median plane direction of the microphone array, the Lch and Rch sound signals are almost the same, so there is no problem. However, when the sound from the subject comes from a direction other than the median plane direction of the microphone array, a phase difference occurs between Lch and Rch. For this reason, if the Lch first predicted speech signal is used as it is in the Rch noise-mixed section, the speech signal becomes discontinuous before and after the Rch noise-mixed section, and it may be uncomfortable to hear it. FIG. 6 shows an audio signal when audio from the subject comes from a direction other than the median plane direction of the microphone array. FIG. 6A shows an Lch audio input signal, and FIG. 6B shows an Rch audio input signal. In both cases, signals in a noise-mixed section are omitted. FIG. 6C shows the first predicted speech signal of Lch, and FIG. 6D shows the first predicted speech signal of FIG. 6C applied as it is as the predicted speech signal of the Rch noisy section. Since there is a phase difference between the Lch and Rch audio input signals, it can be seen that the audio signal is discontinuous when the Lch first predicted audio signal is used as it is for the Rch. When this is actually heard, a sense of incongruity can occur.

  As described above with reference to FIGS. 5 and 6, this method is effective when the sound from the subject comes from the front of the microphone array (that is, the midplane direction). However, when the sound from the subject comes from a direction other than the median plane direction of the microphone array, a phase difference occurs between the two sound signals. Therefore, when the other interpolation signal is applied as it is, the audio signal becomes discontinuous, and when it is actually heard, a sense of incongruity occurs.

  Hereinafter, the noise reduction processing of this embodiment will be described with reference to FIGS. FIG. 7 is a flowchart of noise reduction processing according to the present embodiment, and FIG. 8 shows an audio signal and a correlation value representing prediction processing of Lch and Rch audio input signals.

  When recording start is instructed and audio recording is started, the audio signal processing circuit 137 determines whether or not driving noise is detected (S1001). This determination may be made by detecting that a lens drive command is issued from the camera control unit 119, or may be made by actually detecting drive noise using the accelerometer 120.

  If drive noise is not detected in S1001, the process proceeds to S1007 and returns to S1001 again until a recording stop instruction is detected by the recording switch, and the flow continues. When driving noise is detected in S1001, the audio signal processing circuit 137 analyzes the output signal of the accelerometer 120 and calculates a noise mixing section (S1002). The audio signal in the noise mixed section is discarded.

  FIG. 8A shows an Lch audio input signal, and FIG. 8B shows an Rch audio input signal. In both cases, the signal in the noise-mixed section is an audio signal after being discarded. The horizontal axis represents time, and the vertical axis represents the sound pressure level of the audio signal.

  When the noise-mixed section is calculated in S1002, the speech signal processing circuit 137 calculates the first predicted speech signal in the noise-mixed section as described in FIG. 5 for the Lch speech input signal (S1003). At this time, as shown in FIG. 8C, the first predicted speech signal is calculated longer than the noise-mixed section by the surplus prediction section. This is obtained by adding a section (extra prediction section) having a predetermined time length before and after the noise-mixing section for Rch phase correction to be described later. In the surplus prediction section, the calculated repetition pitch may be connected before and after the normal prediction section by the length of the surplus prediction section, or the signal at the position synchronized with the surplus prediction section of the actual Lch speech signal is left as it is. It may be used.

  Next, in S1004, the first predicted speech signal calculated in S1003 is written as a signal of the Lch noise mixed section. In this way, the signal in the noise-containing section is replaced with the predicted speech signal in the section before and after the noise-containing section (first replacement process). At this time, the signal in the extra prediction section is omitted, and only the audio signal in the normal prediction section is written. In this way, noise reduction processing of the Lch audio signal is performed.

  In step S1005, a phase difference between the Lch audio signal and the Rch audio signal is detected. A method of detecting the phase difference between the Lch and Rch audio signals will be described with reference to FIG. FIG. 8D shows a result of taking a correlation value between the Lch correlation reference section shown in FIG. 8A and the Rch correlation reference section shown in FIG. 8B. Since it is displayed in synchronization with the audio signal, the calculated correlation center position corresponds to the right end of the correlation reference section. The correlation reference section is set to a predetermined section in front of or behind the noise mixture section. When there is no phase difference between the Lch and Rch audio signals, the correlation value becomes maximum at the calculated correlation center position. When there is a phase difference, the length from the calculated correlation center position to the maximum correlation value position is detected as a phase difference as Lch and Rch.

  Next, in S1006, a phase-corrected predicted signal (second predicted speech signal) obtained by shifting the first predicted speech signal calculated in S1003 by the number of samples corresponding to the phase difference detected in S1005 is generated, and Rch noise is generated. Write as mixed section signal. In this way, the Rch signal in the noise-mixed section is replaced with the second predicted speech signal obtained by correcting the phase of the Lch first predicted speech signal (second replacement process). At this time, an extra prediction signal that does not apply to the noise-containing section of the prediction signal is discarded and written. FIGS. 8E and 8F show the operation of applying the second predicted audio signal to the noise mixed section of the Rch audio signal. The second predicted speech signal in FIG. 8E is obtained by shifting the first predicted speech signal in FIG. 8C by the number of samples corresponding to the phase difference detected in S1005. The extra audio signals at both ends of the second predicted audio signal are discarded and written as audio signals in the Rch noise-mixed section. As a result, an Rch audio signal as shown in FIG. 8F is obtained.

  The process returns to S1001 and is repeated until the operation detection unit 135 detects that the recording switch is turned OFF in S1007. When the recording switch OFF is detected, the recording operation is terminated.

  As described above, in the present embodiment, the first predicted speech signal is calculated for only the Lch with respect to the noise mixed section. Thereafter, the phase difference of the audio signal between Lch and Rch is detected. Then, the first predicted speech signal is corrected based on the phase difference to generate a second predicted speech signal, which is used as the predicted speech signal in the Rch noise-mixed section. This realizes a high-quality noise reduction process that reduces the computational load while maintaining the stereo feeling.

  The surplus prediction interval in FIG. 8C and the Lch and Rch correlation reference intervals in FIGS. 8A and 8B may be set to the interval length corresponding to the assumed maximum phase difference. That is, when the subject sound comes from the direction of the extension line connecting the two microphones, the maximum phase difference is generated between the Lch and Rch audio signals. Therefore, if the distance between the two microphones is Lm (mm) and the transmission speed of sound by air propagation is 340000 (mm / sec), the surplus prediction section and the correlation reference section are Lm / 340000 (sec) long. Only need to be provided.

  In the above-described embodiment, the speech signals of the normal prediction section and the surplus prediction section are obtained from the Lch speech signal. However, if the phase difference between Lch and Rch is detected first, the necessary phase correction amount can be known, so that only the signal necessary for correcting the prediction signal needs to be obtained for the signal in the extra prediction section. Depending on the sequence of the audio signal processing circuit, this procedure can reduce the calculation load.

  In the above-described embodiment, a two-channel microphone array is configured, but a three-channel or more microphone array may be configured. In this case, the prediction signal calculation of S1003 is performed only for the audio signal of the first channel, and the phase difference is detected for the audio signals of the first channel and the other channels for the other channels. Then, the prediction signal of the first channel may be corrected with each phase difference and used. As the number of channels increases, the calculation load is reduced as compared with the case where the prediction signal is calculated for each channel.

  Further, in the above-described embodiment, in the prediction signal calculation process, the correlation is detected before and after the noise-containing section and the pitch is detected to calculate the prediction signal of the noise-containing section. However, the present invention is not limited to this. For example, a linear prediction coefficient is obtained by predicting a speech signal in a certain section immediately before and immediately after a noise-mixing section by linear prediction. By multiplying the obtained linear prediction coefficient by the speech signal before and after the noise-mixed section, the speech signal in the noise-mixed section can be predicted one after another. In the linear prediction method, the calculation load of the prediction process becomes very high depending on the order of the linear prediction coefficient. Therefore, as described above, it is effective to reduce the amount of calculation by performing the prediction process only for the first channel and correcting and using the prediction signal result in the noise mixed section of the other channels.

  In the present embodiment, the predicted speech signal is generated using the speech signals before and after the noise-mixed section. However, the predicted signal may be generated from at least one before and after the noise-mixed section.

  In the above-described embodiment, reduction of noise generated by driving the lens has been described. However, if the noise that can be detected by the accelerometer 120 to detect the operation noise generated when operating the operation member, the touch noise generated when the operator touches the camera body, etc., can be detected. Noise can also be reduced.

  Moreover, although the noise reduction process in the imaging device has been described in the above-described embodiment, the present invention can also be applied to other audio processing devices that reduce noise in the acquired audio signal.

Second Embodiment
In the first embodiment, a speech prediction signal in a noise-mixed section is calculated for only Lch, the phase difference between Lch and Rch is detected, and the Lcn prediction signal is phase-corrected and used for Rch. However, when the phase difference between Lch and Rch is sufficiently small, there is no audible effect. Therefore, in the second embodiment, when the phase difference between the Lch and the Rch is smaller than the threshold value, the prediction signal from the Lch is used as it is as the prediction signal of the Rch noise mixing section without correcting the phase difference. This is advantageous in terms of calculation load compared to the case where all phase difference correction is performed uniformly.

  Since the apparatus configuration is the same as that of the first embodiment, the description thereof is omitted. FIG. 9 is a flowchart of the second embodiment. First, when the moving image shooting operation switch is turned on, a flowchart for starting the recording operation starts.

Since the operations of S2001 and S2002 are the same as the operations of S1001 and S1002 of the first embodiment, description thereof will be omitted.
In S2003, the phase difference between the Lch and Rch audio signals is detected as in S1005. In step S2004, it is determined whether the phase difference detected in step S2003 is equal to or greater than the threshold value Tfs. If the detected phase difference is equal to or greater than the threshold value Tfs, the process proceeds to S2005. In S2005 to S2007, similarly to S1003 to S1004 and S1006, a predicted speech signal including a surplus prediction section is calculated from the Lch speech signal, is corrected by the detected phase difference, and is used as a predicted speech signal of the Rch noise-mixed section. .

  On the other hand, if the detected phase difference is less than the threshold, the process proceeds to S2009, and an Lch predicted speech signal is calculated. At this time, the predicted speech signal of only the normal prediction section excluding the excess prediction section is calculated. Next, in S2010, the predicted speech signal calculated in S2009 is written without performing phase correction as the predicted speech signal of the Lch and Rch noise mixed sections.

  Thereafter, the process returns to S2001 and is repeated until the operation detection unit 135 detects that the recording switch is turned off in S2008. When the recording switch OFF is detected, the recording operation is terminated.

  According to the second embodiment described above, when the phase difference between the Lch and Rch audio signals is less than the threshold value, the Lch predicted audio signal is directly used as the predicted audio signal of the Rch noisy section without phase correction. Use. When the phase difference between the two audio signals is sufficiently small, there is little influence on the audibility even if the predicted audio signal from the Lch is used as it is without performing phase correction, so that it is possible to eliminate the calculation load related to phase correction. .

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 101: Imaging device, 102: Camera body, 103: Shooting lens, 104: Imaging optical system, 106: Lens drive part, 107: Lens control part, 115: Microphone, 119: Camera control part

Claims (15)

Obtaining means for obtaining audio signals of a plurality of channels including the first channel and the second channel;
Phase difference detection means for detecting a phase difference between the audio signal of the first channel and the audio signal of the second channel;
A first predicted speech signal for replacing the speech signal in the noise-mixed section is generated using the voice signal of the first channel in at least one section before and after the noise-mixed section in the speech signal, and the first predicted speech First replacement means for replacing the audio signal of the first channel in the noise-mixed section with a signal;
Second replacement means for generating a second predicted speech signal by correcting the first predicted speech signal by the phase difference and replacing the speech signal of the second channel in the noise-mixed section with the second predicted speech signal. When,
A speech processing apparatus comprising:   2. The speech according to claim 1, wherein when the phase difference is less than a threshold, the second replacement unit generates the second predicted speech signal without correcting the first predicted speech signal. Processing equipment.   The first replacement means includes a section in which a section having a length corresponding to the maximum phase difference between the sound signal of the first channel and the sound signal of the second channel is added before and after the noise-mixing section. The speech processing apparatus according to claim 1, wherein the first prediction signal is generated. The first replacement means includes
Means for generating a first signal by replicating the audio signal of the first channel in a front section of the noise-mixing section;
Means for generating a second signal by replicating the audio signal of the first channel in a rear section of the noise-mixing section;
Means for generating the first predicted speech signal by combining the first signal and the second signal;
The speech processing apparatus according to claim 1, comprising: The means for generating the first signal performs pitch detection of the audio signal of the first channel in the front section, and generates the first signal by repeatedly duplicating the detected pitch section,
The means for generating the second signal detects the pitch of the audio signal of the first channel in the rear section, and generates the second signal by repeatedly duplicating the detected pitch section. The voice processing apparatus according to claim 4.   The phase difference detection means is configured to detect the phase difference based on a correlation value between the first channel audio signal and the second channel audio signal in a predetermined interval set in front of or behind the noise mixture interval. The speech processing apparatus according to claim 1, wherein:   An image pickup apparatus comprising the audio processing apparatus according to claim 1.   The imaging apparatus according to claim 7, wherein the noise mixing section is a driving period of a lens driving unit included in the imaging apparatus. An acquisition step of acquiring audio signals of a plurality of channels including the first channel and the second channel;
A phase difference detection step of detecting a phase difference between the audio signal of the first channel and the audio signal of the second channel;
A first predicted speech signal for replacing the speech signal in the noise-mixed section is generated using the voice signal of the first channel in at least one section before and after the noise-mixed section in the speech signal, and the first predicted speech A first substituting step of substituting the audio signal of the first channel in the noisy section with a signal;
A second replacement step of generating a second predicted speech signal by correcting the first predicted speech signal by the phase difference, and replacing the speech signal of the second channel in the noise-mixed section with the second predicted speech signal. When,
A voice processing method characterized by comprising:   10. The speech according to claim 9, wherein the second replacement step generates the second predicted speech signal without correcting the first predicted speech signal when the phase difference is less than a threshold value. Processing method.   In the first replacement step, a section having a length corresponding to the maximum phase difference between the audio signal of the first channel and the audio signal of the second channel is added before and after the noise mixing section. The speech processing method according to claim 9 or 10, wherein the first prediction signal is generated. The first replacement step includes
Generating a first signal by replicating the audio signal of the first channel in the front section of the noise-mixing section;
Generating a second signal by replicating the audio signal of the first channel in a rear section of the noise-mixing section;
Generating the first predicted speech signal by combining the first signal and the second signal;
The voice processing method according to claim 9, comprising: The step of generating the first signal performs pitch detection of the audio signal of the first channel in the front section, and generates the first signal by repeatedly duplicating the detected pitch section.
The step of generating the second signal includes performing pitch detection of the audio signal of the first channel in the rear section, and generating the second signal by repeatedly duplicating the detected pitch section. The voice processing method according to claim 12.   The phase difference detection step includes the phase difference based on a correlation value between the audio signal of the first channel and the audio signal of the second channel in a predetermined interval set in front of or behind the noise mixture interval. The voice processing method according to claim 9, wherein the voice processing method is detected.   The program for functioning a computer as each means which the audio | voice processing apparatus of any one of Claims 1 thru | or 6 has.

Images