JP2008197577A - Voice processing device, voice processing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice processing device, a voice processing method and a program. <P>SOLUTION: The voice processing device comprises: a voice determination section for determining whether or not, first voice generated from a specified sound source is included in input voice, based on position information of the sound source; a voice separating section in which, when it is determined that the first voice is included in the input voice by the voice determination section, the input voice is separated into the first voice and second voice which is generated by a sound source other than the specific sound source; and a voice mixing section 150 for mixing the first voice and the second voice separated by the voice separating section, with an arbitrary sound volume ratio. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a voice processing device, a voice processing method, and a program.

  In the near future, video / audio recording apparatuses capable of recording a video of a subject and a sound emitted from the subject are widely used. An operator of the video / audio recording apparatus can adjust the imaging direction of the video / audio recording apparatus, or operate an operation unit provided in the video / audio recording apparatus to enlarge or reduce the video of the subject.

  Here, the sound volume decreases as the distance from the sound source increases. Accordingly, in the video / audio recording apparatus as described above, when the voice caused by the operator, such as the voice of the operator of the video / audio recording apparatus or the operation sound of the operation means, is recorded at a volume higher than the sound emitted from the subject. was there.

  Patent Document 1 discloses a sound processing apparatus for recording a sound in which the sound volume caused by such an operator is suppressed. Specifically, the sound processing device includes a total of five directional microphones: front left, front right, rear left, rear right, and detachable microphones. Therefore, the operator's voice located in the rear center is hardly collected by any of the front left, front right, rear left, and rear right microphones, and is collected by a removable microphone according to necessity or purpose. Can be made.

  Further, Patent Document 2 uses a BSS (Blind Source Separation) method based on an ICA (Independent Component Analysis) method for signals from one or more sound sources among mixed sounds including sounds from a plurality of sound sources. Techniques for separating are disclosed.

Japanese Patent Laid-Open No. 2005-341073 JP 2006-154314 A

  However, since the conventional voice processing apparatus needs to be provided with a large number of microphones, the hardware scale of the voice processing apparatus becomes large. Further, the conventional voice processing apparatus has a problem that restrictions are imposed on the position of the operator because the voice of the operator is selected using the directivity of the microphone.

  Therefore, the present invention has been made in view of the above problems, and the object of the present invention is to adjust and record the volume ratio of the entire sound emitted from a specific sound source. It is an object of the present invention to provide a new and improved voice processing apparatus, voice processing method and program.

  In order to solve the above problems, according to an aspect of the present invention, a sound determination unit that determines whether or not a first sound emitted from a specific sound source is included in an input sound based on position information of the sound source And when the sound determination unit determines that the first sound is included in the input sound, the sound that separates the input sound into the first sound and the second sound emitted from a sound source other than the specific sound source An audio processing device is provided, comprising: a separation unit; and an audio mixing unit that mixes the first audio and the second audio separated by the audio separation unit at an arbitrary volume ratio.

  In such a configuration, the sound separation unit separates the first sound emitted from the specific sound source included in the input sound, and the sound mixing unit is, for example, the first sound and another sound included in the input sound. The second sound is mixed so that the volume ratio occupied by the first sound is lower than the volume ratio of the first sound occupied by the input sound. Therefore, when the volume of the first sound emitted from the specific sound source is unnecessarily large among the input sounds, the sound mixing unit determines that the volume ratio occupied by the second sound is greater than the volume ratio of the second sound occupied by the input sound. Increased mixed speech can be obtained. As a result, according to the sound processing apparatus, it is possible to prevent the second sound from being buried in the first sound.

  In addition, the sound mixing unit is configured such that, for example, the volume ratio in which the first sound occupies the first sound emitted from the vicinity and the second sound that is other sound included in the input sound occupies the first sound. You may mix so that it may increase from the volume ratio of one audio | voice. According to this configuration, when it is desired to record the voice of the voice recorder, the first voice produced by the voice recorder can be emphasized. Note that when the voice determination unit determines that the first voice is not included in the input voice, the voice separation unit does not have to separate the input voice.

  The specific sound source may be located within a set distance range from the recording position of the input sound. That is, the first sound may be a sound emitted from a set distance range from the recording position of the input sound. Here, since the sound volume decreases as the distance increases, the sound emitted from the sound source closer to the recording position is often recorded at a higher sound volume in the input sound. Therefore, the sound mixing unit can suppress the volume ratio of the first sound close to the recording position of the input sound, and can correct the unbalanced sound volume relation caused by the difference in the sound source distance from the recording position.

  The first sound may include a sound caused by an operator of the device used when collecting the input sound, and the second sound may include a sound emitted from the sound collection target. According to such a configuration, the volume ratio of the first sound emitted by the operator operating the device in the vicinity of the device used when collecting the input sound is suppressed, and the sound is emitted from the sound collection target. It is possible to prevent the second sound that has been received from being buried by the first sound.

  The sound determination unit may determine whether or not the first sound is included in the input sound based on at least one of the volume and the sound quality of the input sound. Here, the sound determination unit estimates the position information of the sound source of the input sound or the position information of the sound source for each sound emitted from one or more sound sources included in the input sound based on the volume or phase of the input sound. May be.

  The audio processing apparatus further includes an imaging unit that captures an image, and the audio determination unit is configured to detect a sound source based on at least one of a volume or a phase of sound emitted from one or more sound sources included in the input sound. A position information calculation unit that calculates position information, the position information calculation unit calculates that the position of the sound source of the input sound is behind the imaging direction of the imaging unit, and the input sound has a sound quality that matches or approximates the human voice In some cases, it may be determined that the first sound emitted from the specific sound source is included in the input sound. Here, the operator often operates the audio processing device from behind the imaging direction of the imaging unit. Therefore, when the position of the sound source of the input sound is behind the image capturing direction of the image capturing unit and the input sound has a sound quality that matches or approximates the human sound, the sound determination unit operates as the first sound in the input sound. It is possible to determine that the voice is dominantly included. As a result, a mixed sound in which the volume ratio of the operator's sound is reduced by the sound mixing unit can be obtained.

  If the position of the sound source of the input sound is within the set distance from the sound collection position, the input sound includes impulse sound, and the input sound is large compared to the past average volume, the sound determination unit You may determine with the 1st audio | voice emitted from the specific sound source being included. Here, in many cases, an impulse sound such as “click” or “bang” is generated when an operator of a device that records input sound operates a button of the device or changes the device. Further, since the impulse sound is generated in the apparatus, there is a high possibility that the impulse sound is collected at a relatively large volume. Therefore, the sound determination unit inputs an input sound when the position of the sound source of the input sound is within a set distance from the sound collection position, the input sound includes an impulse sound, and the input sound is larger than the past average sound volume. It can be determined that noise resulting from the operation of the operator is dominantly included in the voice as the first voice. As a result, it is possible to obtain a mixed sound in which the volume ratio of noise caused by the operation of the operator is reduced by the sound mixing unit.

  The sound processing apparatus may include a plurality of sound collecting units that collect input sound and a recording unit that records the mixed sound mixed by the sound mixing unit on a storage medium. In such a configuration, the recording unit records the mixed sound in which the volume ratio occupied by the first sound is lower than the volume ratio of the first sound occupied in the input sound in the storage medium. Therefore, it is possible to reproduce the mixed sound in which the volume ratio of the first sound is adjusted in the reproducing apparatus without implementing a special volume correction function in the reproducing apparatus that reproduces the mixed sound.

  The audio processing device reproduces the storage medium storing the input sound and the input sound stored in the storage medium, and outputs it to at least one of the position information calculation unit, the sound determination unit, and the sound separation unit May be provided. In this configuration, the position information calculation unit, the sound determination unit, and the sound separation unit can generate mixed sound based on the input sound input from the reproduction unit and output the mixed sound as reproduced sound. Therefore, it is possible to reproduce the mixed sound in which the volume ratio of the first sound is adjusted without implementing a special sound volume correction function in the recording apparatus that records the input sound on the storage medium.

  The sound processing device may include a volume correction unit that performs reverse correction according to the degree of correction on the volume of the second sound separated by the sound separation unit when the volume of the input sound is corrected. . For example, when the volume of the input voice is suppressed as a whole because the volume of the first voice is excessive, the volume of the second voice is also suppressed. In such a case, the volume correction unit can increase the volume of the second sound according to the degree to which the volume of the input sound is suppressed, and can prevent the second sound from becoming too low.

  In order to solve the above problem, according to another aspect of the present invention, a voice separation unit that separates input voices, and a first voice emitted from a specific sound source to the voice separated by the voice separation unit A sound determination unit that determines whether or not a sound source is included, and a sound that mixes the first sound separated by the sound separation unit and the second sound emitted from a sound source other than the specific sound source at an arbitrary volume ratio And a mixing unit. A voice processing device is provided.

  In order to solve the above problem, according to another aspect of the present invention, the computer determines whether or not the input sound contains the first sound emitted from the specific sound source based on the position information of the sound source. A voice determination unit that determines the input voice and the voice determination unit determines that the input voice includes the first voice, the second voice generated from the sound source other than the first voice and the specific sound source A voice processing device comprising: a voice separation unit that separates into voice; and a voice mixing unit that mixes the first voice and the second voice separated by the voice separation unit at an arbitrary volume ratio A program is provided to make it happen.

  Such a program can cause the hardware resources of a computer including, for example, a CPU, ROM, or RAM to execute the functions of the position information calculation unit, the voice determination unit, and the voice separation unit as described above. That is, it is possible to cause a computer using the program to function as the above-described voice processing device.

  The sound determination unit may determine whether or not the first sound is included in the input sound based on at least one of the position information of the sound source, the volume or the sound quality of the input sound.

  A position that further includes an image pickup unit that picks up an image, and the sound determination unit calculates position information of the sound source based on at least one of a volume and a phase of sound emitted from one or more sound sources included in the input sound Provided with an information calculation unit, the position information calculation unit calculates that the position of the sound source of the input sound is behind the imaging direction of the imaging unit, and if the input sound has a sound quality that matches or approximates a human voice, You may determine with the 1st audio | voice emitted from the specific sound source being included.

  If the position of the sound source of the input sound is within the set distance from the sound collection position, the input sound includes impulse sound, and the input sound is large compared to the past average volume, the sound determination unit You may determine with the 1st audio | voice emitted from the specific sound source being included.

  In order to solve the above problem, according to another aspect of the present invention, it is determined based on position information of a sound source whether or not the first sound emitted from a specific sound source is included in the input sound. Separating the input sound into a first sound and a second sound emitted from a sound source other than the specific sound source when it is determined that the first sound is included in the input sound; and And a step of mixing the first sound and the second sound, which are performed at an arbitrary volume ratio, to provide a sound processing method.

  As described above, according to the sound processing device, sound processing method, and program according to the present invention, it is possible to arbitrarily adjust and output or record the volume ratio of the sound emitted from the specific sound source.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

(First embodiment)
First, the audio recording apparatus 10 according to the first embodiment of the present invention will be described. In the description of this embodiment, an example of a scene in which the audio recording device 10 is used will be described with reference to FIGS. 1 and 2, and then the configuration and operation of the audio recording device 10 will be described with reference to FIGS. 3 to 10. explain.

  FIG. 1 is an explanatory diagram showing an example of a scene in which the audio recording device 10 according to the present embodiment is used. In the example shown in FIG. 1, a child who is a subject stands in front of the school gate of Shinagawa City Ichiban Elementary School, and an operator who holds the voice recording device 10 in which the video imaging function is mounted is in the voice recording device 10. Is facing the subject.

  In addition, the subject responds “yes” to the operator's call “oi”. At this time, the audio recording device 10 in which the video imaging function is implemented records the call of the operator “Ooi” and the answer of the subject “Hai” together with the video of the subject. Here, the audio recorded by the normal audio recording method will be described with reference to FIG.

  FIG. 2 is an explanatory diagram showing the amplitude in the time domain of audio recorded by a normal audio recording method. Assuming that the sound source is a point sound source, the volume of sound collected is inversely proportional to the square of the distance between the sound source and the sound collection position. That is, as the sound collection position moves away from the sound source, the collected sound volume decreases. Therefore, the operator's call “oi” close to the sound collection position is collected as a sound having an amplitude as shown in FIG.

  On the other hand, the answer “yes” of the subject that is farther from the operator than the sound collection position is collected as a sound having a smaller amplitude than the voice of the operator as shown in FIG. In this case, according to the normal audio recording method, as shown in FIG. 2C, audio in which the operator's call “oi” and the subject “yes” are superimposed is recorded. .

  However, the voice shown in FIG. 2 (c) predominantly includes an operator's call “Ooi”, and the subject's answer “Hai” is buried. Similarly, operation noise by the operator is recorded relatively large compared to the sound emitted by the subject. For this reason, there is a problem that the sound emitted from the subject is often masked by the sound originating from the operator, and the sound emitted from the subject cannot be recorded with an appropriate volume balance intended by the operator.

  In view of the above, the audio recording apparatus 10 according to the present embodiment has been created with the above problem as a focus. The audio recording apparatus 10 according to the present embodiment can suppress the volume ratio of audio caused by the operator, and can record the audio emitted by the subject and the audio caused by the operator with an appropriate volume balance. Hereinafter, the detailed configuration and operation of the audio recording apparatus 10 will be described.

  FIG. 3 is a functional block diagram showing the configuration of the audio recording apparatus 10 as an example of the audio processing apparatus according to the present embodiment. The audio recording device 10 includes an audio pickup unit 110, an audio determination unit 120, a sound source separation unit 140, an audio mixing unit 150, a recording unit 160, and a storage unit 170. In FIG. 1, a video camera is shown as the audio recording device 10, but the audio recording device 10 is not limited to a video camera, and is a PC (Personal Computer), a mobile phone, a PHS (Personal Handyphone System), and a portable audio process. It may be an information processing device such as a device, a portable video processing device, a PDA (Personal Digital Assistant), a home game device, or a portable game device.

  The sound collection unit 110 collects sound and discretely quantizes the collected sound. In addition, the sound collection unit 110 includes two or more sound collection units (for example, microphones) that are physically separated. In the example illustrated in FIG. 3, the sound collecting unit 110 includes two parts, a sound collecting unit that collects the left sound L and a sound collecting unit that collects the right sound R. The sound collection unit 110 outputs the discretely quantized left sound L and right sound R as input sounds to the sound determination unit 120 and the sound source separation unit 140.

  The sound determination unit 120 adds a nearby sound (first sound) emitted from the vicinity of the sound recording device 10 to the input sound input from the sound collecting unit 110, such as the operator's voice or noise caused by the operator's action. ) Is included. A detailed configuration of the voice determination unit 120 will be described with reference to FIG.

  FIG. 4 is a functional block diagram illustrating the configuration of the voice determination unit 120. The sound determination unit 120 includes a sound volume detection unit 122 including a sound volume detector 124, an average sound volume detector 126 and a maximum sound volume detector 128, a sound quality detection unit 130 including a spectrum detector 132 and a sound quality detector 134, and distance direction estimation. And an operator voice estimator 138. In FIG. 4, the left sound L and the right sound R are collectively shown as input sound from the viewpoint of clarity of the drawing.

  The volume detector 124 detects the volume value sequence (amplitude) of the input voice given in frame units (for example, several tens of msec) of a predetermined length, and the volume level sequence of the detected input voice is the average volume detector 126, Output to the volume detector 128, the sound quality detector 134, and the distance direction estimator 136.

  The average sound volume detector 126 detects the average sound volume value of the input sound for each frame, for example, based on the volume value sequence in units of frames input from the sound volume detector 124. The average sound volume detector 126 outputs the detected sound volume average value to the sound quality detector 134 and the operator voice estimator 138.

  The maximum sound volume detector 128 detects the maximum sound volume value of the input sound for each frame, for example, based on the volume value sequence in units of frames input from the sound volume detector 124. Further, the maximum volume detector 128 outputs the detected maximum volume of the input voice to the sound quality detector 134 and the operator voice estimator 138.

  The spectrum detector 132 performs, for example, FFT (Fast Fourier Transform) processing on the input sound, and detects each spectrum in the frequency domain of the input sound. The spectrum detector 132 outputs the detected spectrum to the sound quality detector 134 and the distance direction estimator 136.

  The sound quality detector 134 receives the input sound, the average sound volume value, the maximum sound volume value, and the spectrum. Based on the input, the sound quality detector 134 detects the human sound like the sound, the music likeness, the continuity, the impulsiveness, etc. Output to speech estimator 138. The human voice-likeness may be information indicating whether or not a part or the whole of the input voice matches the human voice, or how close to the human voice. Further, the music likeness may be information indicating whether or not a part or the whole of the input voice is music, or how close it is to music.

  The stationarity refers to a property that the statistical property of the voice does not change so much in time, for example, air-conditioning sound. Impulse property refers to a strong property of noise property in which energy is concentrated in a short time such as a hit sound and a plosive sound.

  For example, the sound quality detector 134 can detect the likelihood of human speech based on the degree of coincidence between the spectral distribution of the input speech and the spectral distribution of the human speech. Also, the sound quality detector 134 may compare the maximum volume value for each frame, and detect that the impulsiveness is higher as the maximum volume value is larger than other frames.

  Note that the sound quality detector 134 may analyze the sound quality of the input sound by using a signal processing technique such as a zero crossing method or LPC (Linear Predictive Coding) analysis. Since the fundamental period of the input speech is detected according to the zero crossing method, the sound quality detector 134 determines whether the fundamental period is included in the fundamental period of human speech (for example, 100 to 200 Hz). The likelihood may be detected.

  The distance direction estimator 136 receives an input sound, a volume value sequence of the input sound, a spectrum of the input sound, and the like, and a sound source from which the dominant sound included in the input sound or the input sound is generated based on the input. It functions as a position information calculation unit that estimates position information such as direction information and distance information. The distance direction estimator 136 combines the estimation method of the position information of the sound source based on the phase, volume, volume value sequence, past average volume value, maximum volume value, and the like of the input sound, so that the reverberation and the video recording apparatus 10 main body. Even when the influence of sound reflection is large, the sound source position can be estimated comprehensively. An example of the direction information and distance information estimation method by the distance direction estimator 136 will be described with reference to FIGS.

  FIG. 5 is an explanatory diagram showing a state in which the sound source position of the input sound is estimated based on the phase difference between the two input sounds. If it is assumed that the sound source is a point sound source, the phase difference between each input sound and the phase of each input sound that reaches the microphone M1 and the microphone M2 constituting the sound collection unit 110 can be measured. Further, the difference between the distance from the microphone M1 to the sound source position of the input sound and the distance from the microphone M2 can be calculated from the phase difference and the values of the frequency f and the sound speed c of the input sound. The sound source exists on a set of points where the distance difference is constant. It is known that such a set of points having a constant distance difference is a hyperbola.

For example, assume that the microphone M1 is located at (x1, 0) and the microphone M1 is located at (x2, 0) (this assumption does not lose generality). Further, if a point on the set of sound source positions to be obtained is set as (x, y) and the distance difference is set as d, the following formula 1 is established.

Furthermore, Formula 1 can be expanded as Formula 2, and formula 3 is derived by formulating Formula 2 to represent a hyperbola.

  Further, since the distance direction estimator 136 can determine whether the sound source is near the microphone M1 or the microphone M2 based on the volume difference between the input sounds collected by the microphone M1 and the microphone M2, for example, FIG. It can be determined that the sound source exists on the hyperbola 1 close to the microphone M2 as shown in FIG.

The frequency f of the input sound used for the phase difference calculation needs to satisfy the condition of the following formula 4 with respect to the distance between the microphone M1 and the microphone M2.

  FIG. 6 is an explanatory diagram showing a state in which the sound source position of the input sound is estimated based on the phase difference between the three input sounds. Assuming the arrangement of the microphone M3, the microphone M4, and the microphone M5 that constitute the sound pickup unit 110 as shown in FIG. 6, the microphone M5 reaches the microphone M5 in comparison with the phase of the input sound that reaches the microphone M4. If the phase of the input speech to be delayed is delayed, the distance direction estimator 136 can determine that the sound source is located on the opposite side of the microphone M5 with respect to the straight line 1 connecting the microphone M4 and the microphone M5 (front / back determination).

  Further, the distance direction estimator 136 calculates a hyperbola 2 in which a sound source can exist based on the phase difference between the input sounds reaching the microphones M3 and M4, and the input sounds reaching the microphones M4 and M5. Based on the phase difference, a hyperbola 3 in which a sound source can exist can be calculated. As a result, the distance direction estimator 136 can estimate the intersection P1 of the hyperbola 2 and the hyperbola 3 as the sound source position.

  FIG. 7 is an explanatory diagram showing a state in which the sound source position of the input sound is estimated based on the volumes of the two input sounds. Assuming that the sound source is a point sound source, the sound volume observed at a certain point is inversely proportional to the square of the distance according to the inverse square law. Assuming the microphone M6 and the microphone M7 constituting the sound pickup unit 110 as shown in FIG. 7, the set of points at which the volume ratio reaching the microphone M6 and the microphone M7 is constant is a circle. The distance direction estimator 136 calculates the volume ratio from the volume value input from the volume detector 124, and can calculate the radius and center position of the circle where the sound source exists.

As shown in FIG. 7, when the microphone M6 is located at (x3, 0) and the microphone M7 is located at (x4, 0) (this assumption does not lose generality), the sound source position to be obtained If the points on the set of (x, y) are placed, the distances r1 and r2 from each microphone to the sound source can be expressed as in the following Equation 5.

Here, the following formula 6 is established from the inverse square law.

Formula 6 is transformed into Formula 7 using a positive constant d (for example, 4).

Substituting Equation 7 into r1 and r2 and rearranging it leads to Equation 8 below.

From Equation 8, the distance / direction estimator 136 can estimate that the sound source exists on the circle 1 whose center coordinates are expressed by Equation 9 and whose radius is expressed by Equation 10, as shown in FIG.

  FIG. 8 is an explanatory diagram showing a state in which the sound source position of the input sound is estimated based on the volumes of the three input sounds. Assuming the arrangement of the microphone M3, the microphone M4, and the microphone M5 constituting the sound pickup unit 110 as shown in FIG. 8, the microphone M5 reaches the microphone M5 in comparison with the phase of the input sound that reaches the microphone M4. If the phase of the input speech to be delayed is delayed, the distance direction estimator 136 can determine that the sound source is located on the opposite side of the microphone M5 with respect to the straight line 2 connecting the microphone M4 and the microphone M5 (front / back determination).

  Further, the distance direction estimator 136 calculates a circle 2 in which a sound source can exist based on the volume ratio of the input sound that reaches each of the microphone M3 and the microphone M4, and the input sound that reaches each of the microphone M4 and the microphone M5. The circle 3 where the sound source can exist can be calculated based on the volume ratio. As a result, the distance direction estimator 136 can estimate the intersection P2 of the circles 2 and 3 as the sound source position. When four or more microphones are used, the distance / direction estimator 136 can perform estimation with higher accuracy including spatial arrangement of sound sources.

The distance direction estimator 136 estimates the position of the sound source of the input sound based on the phase difference and volume ratio of each input sound as described above, and sends the estimated sound source direction information and distance information to the operator sound estimator 138. Output. Table 1 below summarizes the input / output of each component of the sound volume detector 122, the sound quality detector 130, and the distance direction estimator 136 described above.

  When voices emitted from a plurality of sound sources are superimposed on the input voice, it is difficult for the distance direction estimator 136 to accurately estimate the sound source position of the voice dominantly included in the input voice. . However, the distance direction estimator 136 can estimate a position close to the sound source position of the sound that is dominantly included in the input sound. Further, since the estimated sound source position may be used as an initial value for sound separation in the sound source separation unit 140, even if there is an error in the sound source position estimated by the distance direction estimator 136, the sound recording device 10 Can perform a desired operation.

  Returning to the description of the configuration of the speech determination unit 120 with reference to FIG. 4, the operator speech estimator 138 determines that the operator's speech or It is comprehensively determined whether or not the vicinity sound emitted from the specific sound source in the vicinity of the sound recording device 10 such as noise caused by the operation of the operator is included. Further, when the operator speech estimator 138 determines that the input speech includes a nearby speech, the operator speech estimator 138 indicates that the input speech includes the nearby speech (operator speech presence information) or distance direction estimation. A function as a voice determination unit that outputs position information estimated by the device 136.

  Specifically, the operator voice estimator 138 estimates that the position of the sound source of the input voice is behind the imaging direction of the imaging unit (not shown) that captures the video, and inputs it to the distance direction estimator 136. When the sound has a sound quality that matches or approximates that of a human sound, it may be determined that the input sound includes a nearby sound. Here, as shown in FIG. 9, the operator often operates the audio recording device 10 from the rear in the imaging direction of the imaging unit, that is, from the left rear of the finder (during right-handed normal shooting other than self-shooting). .

  Therefore, the operator speech estimator 138 operates as a nearby speech on the input speech when the position of the sound source of the input speech is behind the imaging direction of the imaging unit and the input speech has a sound quality that matches or approximates human speech. It can be determined that the voice of the person is dominantly included. As a result, a mixed sound in which the volume ratio of the operator's sound is reduced can be obtained by the sound mixing unit 150 described later.

  In addition, the operator voice estimator 138 has a position of the sound source of the input voice within a set distance from the sound pickup position (for example, in the vicinity of the voice recording apparatus 10 such as within 1 m of the voice recording apparatus 10). If the impulse sound is included and the input sound is larger than the average sound volume in the past, it may be determined that the input sound includes the near sound emitted from the specific sound source. Here, when an operator of the voice recording apparatus 10 operates a button of the voice recording apparatus 10 or changes the voice recording apparatus 10, impulse sounds such as “pachin” and “bang” are often generated. Further, since the impulse sound is generated in the sound recording device 10, there is a high possibility that the sound is collected at a relatively large volume.

  Therefore, the operator voice estimator 138 has a position of the sound source of the input voice within the set distance from the sound collection position, the input voice includes an impulse sound, and the input voice is larger than the past average volume. In this case, it can be determined that the input voice mainly includes noise caused by the operation of the operator as the vicinity voice. As a result, it is possible to obtain mixed sound in which the volume ratio of noise caused by the operation of the operator is reduced by the sound mixing unit 150 described later.

In addition, Table 2 below summarizes examples of information input to the operator speech estimator 138 and determination results of the operator speech estimator 138 based on the input information. Note that it is possible to increase the accuracy of determination in the operator speech estimator 138 using a combination of a proximity sensor, a temperature sensor, and the like.

  Returning to the description of the configuration of the voice recording apparatus 10 with reference to FIG. 3, when the operator voice presence information is input from the voice determination unit 120, the sound source separation unit 140 receives the sound source input from the voice determination unit 120. Based on the position information, the input sound input from the sound pickup unit 110 is separated into a near sound such as an operator's sound and a sound collection target sound (second sound) such as a sound of a subject other than the near sound. . As a result, the sound source separation unit 140 outputs as many sounds as the number of input sounds to be input. In FIG. 3, the sound source separation unit 140 receives the left sound L and the right sound R as input sounds, outputs the left vicinity sound L and the right vicinity sound R as the vicinity sound, and outputs the left sound collection target sound L and the right sound collection. A state in which the target voice R is output as the voice to be collected is shown.

  Specifically, the sound source separation unit 140 uses a method (ICA) that uses independent component analysis, a method that uses a small amount of overlap between temporal frequency components of sound, and the like to separate sound according to the sound source. Functions as a separation unit.

  The sound mixing unit 150 mixes the near sound and the sound to be collected input from the sound source separation unit 140 such that the volume ratio occupied by the near sound is reduced from the volume ratio of the near sound occupied in the input sound. According to such a configuration, when the volume of the near voice emitted from the specific sound source is unnecessarily large among the input voices, the voice mixing unit 150 causes the volume ratio of the voice to be picked up to be the volume of the voice to be picked up. It is possible to obtain mixed sound that is increased from the volume ratio. As a result, according to the voice recording device 10, it is possible to prevent the voice to be collected from being buried in the neighboring voice.

  The sound mixing unit 150 generates a mixed left sound L by mixing the input left vicinity sound L and the left sound collection target sound L, and mixes the input right vicinity sound R and the right sound collection target sound R. Then, the mixed right sound R is generated, and the mixed left sound L and the mixed right sound R are output to the recording unit 160 as mixed sound.

  In addition, the sound mixing unit 150 calculates an appropriate mixing ratio from the average volume ratio of the nearby sound and the sound collection target sound separated by the sound source separation unit 140, and mixes the vicinity sound and the sound collection target sound at the calculated mixing ratio. May be. Further, the audio mixing unit 150 may change the mixing ratio to be applied in a range where the difference from the mixing ratio applied to the previous frame does not exceed a predetermined upper limit value. The mixing ratio may be set by the user.

  The recording unit 160 records the mixed sound input from the sound mixing unit 150 in the storage unit 170. The storage unit 170 is a non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory) or an EPROM (Erasable Programmable Read Only Memory), a hard disk and a disk type magnetic disk such as a disk. (Recordable) / RW (ReWritable), DVD-R (Digital Versatile Disk Recordable) / RW / + R / + RW / RAM (Ramdam Access Memory) and BD (Blu-Ray Disc-R) Optical disc and MO (Magneto Optical) It may be a storage medium such as a disk. The storage unit 170 can also store subject video data.

  As described above, according to the audio recording apparatus 10 according to the present embodiment, the recording unit 160 records in the storage unit 170 the mixed sound in which the volume ratio occupied by the nearby sound is reduced from the volume ratio of the nearby sound occupied by the input sound. . Therefore, it is possible to reproduce the mixed sound in which the volume ratio of the neighboring sound is adjusted in the reproducing apparatus without implementing a special volume correction function in the reproducing apparatus that reproduces the mixed sound.

  The configuration of the audio recording apparatus 10 according to the present embodiment has been described above. Next, an audio processing method executed in the audio recording apparatus 10 according to the present embodiment will be described with reference to FIG.

  FIG. 10 is a flowchart showing the flow of the sound processing method executed in the sound recording apparatus 10 according to the present embodiment. First, the sound collection unit 110 of the sound recording apparatus 10 collects sound (S210). If there is no input voice, the process ends. If there is input voice, the distance / direction estimator 136 estimates position information such as the distance and direction of the sound source from which the whole or part of the input voice is emitted. (S230).

  Thereafter, the operator voice estimator 138 determines whether or not the input voice includes a voice uttered by the operator or a nearby voice such as noise caused by the operation of the operator (S240). When the operator voice estimator 138 determines that the input voice includes the near voice, the sound source separation unit 140 separates the input voice into the near voice and the other voices to be collected (S250).

  Subsequently, the sound mixing unit 150 mixes the nearby sound separated by the sound source separation unit 140 and the sound to be collected at an arbitrary ratio to generate mixed sound (S260). After S260, or in S240, if it is determined that the input sound does not include the sound generated by the operator or the nearby sound such as noise caused by the operation of the operator, the recording unit 160 determines whether the input sound is mixed sound or input sound. Is stored in the storage unit 170 (S270).

  As described above, in the audio recording apparatus 10 according to the present embodiment, the sound source separation unit 140 is a sound source of the input sound that is estimated by the distance direction estimator 136 for the nearby sound emitted from the specific sound source included in the input sound. The sound mixing unit 150 divides the vicinity sound and the sound to be collected, which is another sound included in the input sound, and the sound volume ratio of the vicinity sound occupies the input sound. Mix to reduce the volume ratio.

  Therefore, when the volume of the near voice emitted from the specific sound source is unnecessarily large among the input voices, the voice mixing unit 150 increases the volume ratio occupied by the voice to be collected from the volume ratio of the voice to be collected occupying the input voice. The mixed voice can be obtained. As a result, according to the audio recording device 10, it is possible to relatively suppress the nearby sound and prevent the sound collection target sound from being buried in the nearby sound. In addition, it is possible to record a high-quality mixed sound in which the influence of the nearby sound such as a sound and noise generated by the operator included in the input sound is reduced.

  In addition, the audio recording device 10 can record the mixed sound in which the volume ratio occupied by the near voice in the storage unit 170 is lower than the volume ratio of the near voice occupied in the input sound. Therefore, it is possible to reproduce the mixed sound in which the volume ratio of the neighboring sound is adjusted in the reproducing apparatus without implementing a special volume correction function in the reproducing apparatus that reproduces the mixed sound.

  In addition, since the audio recording apparatus 10 according to the present embodiment can process the input audio in software and record the mixed audio in which the volume ratio of the adjacent audio and the sound to be collected is adjusted, the hardware scale such as the number of microphones can be increased. Can be reduced.

(Second Embodiment)
Next, the audio reproducing device 11 according to the second embodiment of the present invention will be described. The audio reproduction device 11 according to the present embodiment can reproduce mixed audio in which the volume ratio of the neighboring audio included in the already stored audio is adjusted. Hereinafter, the configuration of the audio reproduction apparatus 11 will be described with reference to FIG.

FIG. 11 is a functional block diagram showing the configuration of the audio playback device 11 according to the present embodiment. The audio reproduction device 11 according to the present embodiment includes an audio determination unit 120, a sound source separation unit 140, an audio mixing unit 150, a storage unit 172, a reproduction unit 174, and an audio output unit 180.
In the description of the present embodiment, the description of the configuration that is substantially the same as the content described in the first embodiment will be omitted, and the description will be given with a focus on the configuration that is different from the first embodiment.

  The storage unit 172 stores voice recorded in any device having a voice recording function. The reproduction unit 174 reads the sound stored in the storage unit 172 and decodes it as necessary. Then, the reproduction unit 174 outputs the audio stored in the storage unit 172 to the audio determination unit 120 and the sound source separation unit 140. The sound determination unit 120 and the sound source separation unit 140 treat the output from the reproduction unit 174 as input sound, and perform processing substantially the same as the content described in the first embodiment.

  The audio output unit 180 outputs the mixed audio mixed by the audio mixing unit 150. The audio output unit 180 may be a speaker or an earphone, for example. Note that the storage unit 172 according to the present embodiment is similar to the storage unit 170 according to the first embodiment, such as a nonvolatile memory such as an EEPROM or an ERPPM, a magnetic disk such as a hard disk or a disk-type magnetic disk, or a CD-R. / RW, DVD-R / RW / + R / + RW / RAM and BD (Blu-Ray Disc (registered trademark))-R / BD-RE and other storage media such as an MO disk.

  As described above, in the audio reproduction device 11 according to the present embodiment, the audio determination unit 120, the sound source separation unit 140, and the audio mixing unit 150 generate mixed audio based on the input audio input from the reproduction unit 174, and the mixed audio Can be output as reproduced sound. Therefore, it is possible to reproduce the mixed sound in which the volume ratio occupied by the nearby sound is adjusted without mounting a special sound volume correction function in the recording device that records the input sound in the storage unit 172. In addition, it is possible to output high-quality mixed speech in which the influence of nearby speech such as speech and noise emitted by the operator is reduced.

(Third embodiment)
Next, an audio reproducing device 12 according to the third embodiment of the present invention will be described. When the input sound is subjected to AGC (Auto Gain Control), the sound reproducing device 12 according to the present embodiment reversely corrects the volume of the sound collecting target sound included in the input sound and emphasizes the sound collecting target sound ( Boost). Hereinafter, the configuration and operation of the audio reproduction device 12 according to the present embodiment will be described with reference to FIGS. 12 and 13.

FIG. 12 is a functional block diagram showing the configuration of the audio reproduction device 12 according to the present embodiment. The audio reproduction device 12 includes an audio determination unit 120, a sound source separation unit 140, an audio mixing unit 150, a storage unit 172, a reproduction unit 174, an audio output unit 180, and a volume correction unit 190.
In the description of the present embodiment, the description of the configuration that is substantially the same as the content described in the second embodiment will be omitted, and a description will be given with emphasis on the configuration different from the second embodiment.

  The storage unit 172 according to the present embodiment includes a sound determination unit 120, a sound source separation unit 140, a sound source mixing unit 150, a storage unit 172, a playback unit 174, and a sound output unit 180, in part or in whole. Prepare. In the description of the present embodiment, the description of the configuration that is substantially the same as the content described in the first embodiment will be omitted, and the description will be given with a focus on the configuration that is different from the first embodiment.

  The storage unit 172 according to the present embodiment stores a sound partly or entirely subjected to AGC (volume correction). Here, AGC is a compressor mechanism whose purpose is to automatically reduce the volume level and prevent sound cracking in response to excessive input of the volume. The volume of the sound subjected to such AGC will be described with reference to FIG.

  FIG. 13 is an explanatory diagram that compares the volume of the sound (original sound) before application of AGC and the volume of the sound after application of AGC. When the volume of the sound before application of AGC exceeds the threshold th, the AGC compresses the volume to a predetermined ratio (ratio) in the time set as the attack time. The example shown in FIG. 13 shows a case where the volume of the sound before application of AGC is compressed to about 1/2 to 2/3 in the time set as the attack time. Thereafter, when the volume of the sound before application of AGC falls below the threshold th, AGC is canceled within the time set as the release time.

  Here, the sound volume exceeds the threshold th and the AGC operates in many cases when there is an input of an excessive level of near sound from the vicinity of the sound recording device. That is, there are few cases in which AGC is operated by the sound collection target voice of a distant sound source. However, since the input speech is compressed by AGC as a whole, there is a problem that not only the nearby speech included in the input speech but also the originally weakly collected sound is further compressed.

  In view of the above, the audio reproduction device 12 according to the present embodiment has been created with the above problem as a point of focus. The audio reproduction device 12 according to the present embodiment can boost the sound to be collected based on the function of the volume correction unit 190 even when AGC is applied to the input audio.

  The sound volume correction unit 190 detects an attack time to which AGC will be applied from the change in the sound volume of the nearby sound separated by the sound source separation unit 140, and the attack in the sound collection target sound separated by the sound source separation unit 140 The section corresponding to the time is scanned. The sound to be picked up may include background environmental sound, sound generated by the subject, and the like, but when only the background environmental sound is included, it can be approximated that the volume level is almost constant. Therefore, the volume correction unit 190 can determine that AGC has been applied to a section in which the volume of the sound to be picked up has changed by a predetermined level or more.

  Therefore, the volume correction unit 190 can perform reverse correction that adjusts the volume of the section of the sound to be collected so as to be approximately the same as the volume of the section before and after the section, and can boost the sound to be collected.

  It should be noted that the estimated values of the attack time and release time and the degree of reverse correction performed by the sound volume correction unit 190 are retained, so that the sound can be effectively used when the sound to be collected includes the sound emitted from the subject. be able to. That is, even when the sound to be picked up includes sound emitted from the subject, the volume correction unit 190 detects the attack time from the nearby sound and extends before and after the section corresponding to the attack time in the sound to be picked up. Scan the volume value. The sound volume correction unit 190 can determine that the AGC has been operated and perform reverse correction when the sound volume value has changed in a time width that matches the attack time or release time as a result of scanning.

  In the sound mixing unit 150, the volume ratio of the vicinity sound that occupies the whole of the sound to be collected whose sound volume has been reversely corrected by the sound volume correction unit 190 and the vicinity sound separated by the sound source separation unit 140 is suppressed. The mixed sound can be generated by mixing at such a volume ratio.

  As described above, in the audio reproduction device 12 according to the third exemplary embodiment of the present invention, the volume of the input audio is suppressed as a whole because the volume of the nearby audio is excessive, and the volume of the sound to be collected is also suppressed. If it has been done, it is possible to increase the volume of the sound to be collected depending on the degree to which the volume of the input sound is suppressed, and to prevent the sound to be collected from becoming too small.

  In the present embodiment, the case where the sound volume correction unit 190 is provided in the sound reproduction device 12 has been described. However, the input sound is subjected to AGC by being provided in the sound recording device 10 described in the first embodiment. It is also possible to record the mixed sound including the sound to be collected boosted according to the degree of AGC in the storage unit 170.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  For example, each step in the processing of the audio recording apparatus 10 of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, but is performed in parallel or individually (for example, parallel processing) Alternatively, processing by an object) may be included.

  FIG. 3 shows an example in which the sound determination unit 120 determines whether or not the input sound collected by the sound collection unit 110 includes a nearby sound, but the present invention is not limited to such an example. . For example, the sound determination unit 120 receives the sound separated by the sound source separation unit 140, estimates the sound source position of the separated sound, and determines whether the separated sound includes a nearby sound. The separated sound may be output to the sound mixing unit 150. In this case, the sound source separation unit 140 blindly separates the sound for each sound source without an initial value.

  Further, hardware such as a CPU, a ROM, and a RAM incorporated in the audio recording device 10, the audio reproduction device 11, and the audio reproduction device 12 are configured as the above-described configurations of the audio recording device 10, the audio reproduction device 11, and the audio reproduction device 12, respectively. It is also possible to create a computer program for demonstrating the same function as. A storage medium storing the computer program is also provided. Moreover, a series of processing can also be realized by hardware by configuring each functional block shown in each functional block diagram of the audio recording device 10, the audio reproduction device 11, and the audio reproduction device 12 with hardware.

It is explanatory drawing which showed an example of the scene where the audio | voice recording apparatus concerning the 1st Embodiment of this invention is used. It is explanatory drawing which showed the amplitude of the time domain of the audio | voice recorded by the normal audio | voice recording method. FIG. 2 is a functional block diagram showing a configuration of a sound recording device as an example of a sound processing device according to the embodiment. It is the functional block diagram which showed the structure of the audio | voice determination part. It is explanatory drawing which showed a mode that the sound source position of an input audio | voice was estimated based on the phase difference of two input audio | voices. It is explanatory drawing which showed a mode that the sound source position of an input audio | voice was estimated based on the phase difference of three input audio | voices. It is explanatory drawing which showed a mode that the sound source position of an input audio | voice was estimated based on the volume of two input audio | voices. It is explanatory drawing which showed a mode that the sound source position of an input audio | voice was estimated based on the volume of three input audio | voices. It is explanatory drawing which showed the positional relationship of an audio recording device and an operator. It is the flowchart which showed the flow of the audio | voice processing method performed in the audio | voice recording device concerning the embodiment. It is the functional block diagram which showed the structure of the audio | voice reproduction apparatus concerning the 2nd Embodiment of this invention. It is the functional block diagram which showed the structure of the audio | voice reproduction apparatus concerning the 3rd Embodiment of this invention. It is explanatory drawing which represented the volume of the sound before application of AGC and the volume of the sound after application of AGC in contrast.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Audio | voice recording apparatus 11, 12 Audio | voice reproduction apparatus 110 Audio | voice sound collection part 120 Audio | voice determination part 124 Volume | volume detector 134 Sound quality detector 136 Distance direction estimator 138 Operator audio | voice estimator 140 Sound source separation part 150 Audio | voice mixing part 160 Recording part 170 , 172 Storage unit 174 Playback unit 180 Audio output unit 190 Volume correction unit

Claims (15)

A sound determination unit for determining whether or not the first sound emitted from the specific sound source is included in the input sound;
When the sound determination unit determines that the first sound is included in the input sound, the input sound is the first sound and a second sound emitted from a sound source other than the specific sound source. A voice separation unit that separates into;
A sound mixing unit that mixes the first sound and the second sound separated by the sound separation unit at an arbitrary volume ratio;
An audio processing apparatus comprising:   The sound processing apparatus according to claim 1, wherein the specific sound source is located within a set distance from a recording position of the input sound. The first voice includes a voice attributed to an operator of the device used when picking up the input voice;
The voice processing apparatus according to claim 2, wherein the second voice includes a voice emitted from a sound collection target.   4. The voice determination unit according to claim 3, wherein the voice determination unit determines whether or not the first voice is included in the input voice based on at least one of a volume and a sound quality of the input voice. Voice processing device. It further includes an imaging unit that captures images,
The sound determination unit includes a position information calculation unit that calculates position information of the sound source based on at least one of a volume or a phase of sound emitted from one or more sound sources included in the input sound, The position information calculation unit calculates that the position of the sound source of the sound is behind the imaging direction of the imaging unit, and when the input sound has a sound quality that matches or approximates a human sound, The speech processing apparatus according to claim 4, wherein the speech processing apparatus determines that the emitted first speech is included.   When the position of the sound source of the input sound is within a set distance from the sound pickup position, the input sound includes an impulse sound, and the input sound is larger than the past average volume, the sound determination unit The sound processing apparatus according to claim 4, wherein the input sound is determined to include the first sound emitted from a specific sound source. A plurality of sound collection units for collecting the input voice;
A recording unit for recording the mixed audio mixed by the audio mixing unit on a storage medium;
The speech processing apparatus according to claim 1, comprising: A storage medium storing the input voice;
A reproduction unit that reproduces the input voice stored in the storage medium and outputs the reproduced voice to at least one of the position information calculation unit, the voice determination unit, and the voice separation unit;
The speech processing apparatus according to claim 1, comprising: A volume correction unit that performs reverse correction on the volume of the second sound separated by the sound separation unit when the volume of the input sound is corrected;
The speech processing apparatus according to claim 1, comprising: A speech separation unit for separating input speech;
A sound determination unit that determines whether or not the sound separated by the sound separation unit includes a first sound emitted from a specific sound source;
A sound mixing unit that mixes the first sound separated by the sound separation unit and the second sound emitted from a sound source other than the specific sound source at an arbitrary volume ratio;
An audio processing apparatus comprising: Computer
A sound determination unit that determines whether or not the first sound emitted from the specific sound source is included in the input sound based on the position information of the sound source;
When the sound determination unit determines that the first sound is included in the input sound, the input sound is the first sound and a second sound emitted from a sound source other than the specific sound source. A voice separation unit that separates into;
A sound mixing unit that mixes the first sound and the second sound separated by the sound separation unit at an arbitrary volume ratio;
A program for causing a voice processing apparatus to function.   12. The voice determination unit according to claim 11, wherein the voice determination unit determines whether or not the first voice is included in the input voice based on at least one of a volume and a sound quality of the input voice. Program. It further includes an imaging unit that captures images,
The sound determination unit includes a position information calculation unit that calculates position information of the sound source based on at least one of a volume or a phase of sound emitted from one or more sound sources included in the input sound, The position information calculation unit calculates that the position of the sound source of the sound is behind the imaging direction of the imaging unit, and when the input sound has a sound quality that matches or approximates a human sound, 13. The program according to claim 12, wherein it is determined that the first voice that is uttered is included.   When the position of the sound source of the input sound is within a set distance from the sound pickup position, the input sound includes an impulse sound, and the input sound is larger than the past average volume, the sound determination unit The program according to claim 12, wherein it is determined that the first voice emitted from a specific sound source is included in the input voice. Determining whether or not a first sound emitted from a specific sound source is included in the input sound based on position information of the sound source;
Separating the input sound into the first sound and a second sound emitted from a sound source other than the specific sound source when it is determined that the first sound is included in the input sound; ;
Mixing the separated first sound and the second sound at an arbitrary volume ratio;
An audio processing method comprising:

Images