JP2009258423A - Sound processing device, electronic appliance, and sound processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound processing device or a sound processing method, capable of effectively providing a sound signal intended by a user while reducing the size and cost thereof, and an electronic appliance provided with the sound processing device. <P>SOLUTION: A sound processing part 7 provided on an imaging device comprises a sound collection environment deciding part 72 which decides whether an input sound signal is collected in air or in water; and an in-water characteristic correction part 73 which performs in-water characteristic correction processing to the input sound signal based on the deciding result of the deciding part 72. A sound signal collected in water is corrected by the in-water characteristic correction part 73, whereby a sound signal to be output can be made close to the sound signal intended by the user. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an audio processing apparatus and an audio processing method for processing an audio signal collected in water, and an electronic apparatus such as an imaging apparatus equipped with the audio processing apparatus.

  In recent years, various electronic devices such as video cameras and IC recorders that can collect sound not only in the air but also in the water have been proposed. These electronic devices have a waterproof structure and can collect sound not only in the air but also in the water.

  However, since the sound collection characteristic of sound in water is significantly different from the sound collection characteristic in air, an audio signal collected in water using an electronic device is significantly different from an audio signal collected in air. . For this reason, when an audio signal collected in water is reproduced, it becomes a problem that the audio signal is contrary to the user's intention, such as being very difficult to hear or annoying.

For this problem, Patent Document 1 proposes a video camera that determines whether or not the environment in which sound is collected is underwater and performs gain control and filtering according to the determination result. This video camera is provided with a pressure sensor and determines whether or not it is underwater based on the presence or absence of water pressure.
Japanese Patent Laid-Open No. 7-30790

  However, when a special device such as a pressure sensor is added, there arises a problem that the electronic device is increased in size and cost is increased. In particular, there is a problem that an electronic device that has been increased in size due to the need to have a waterproof structure will be further increased in size.

  Furthermore, if it is set as the structure using a pressure sensor, it can be determined whether it is underwater only at the time of recording of an audio | voice signal. For this reason, when reproducing an audio signal recorded without processing the audio signal, it is impossible to determine whether the audio signal is collected in water. Therefore, there is a problem that it is not possible to perform an operation such as processing and reproducing as necessary.

  In addition, the sound collected through propagation in the water becomes difficult to hear or is harsh due to the influence of the sound collection characteristics of the sound in the water. Therefore, there is a need for a sound correction process that effectively reduces this influence.

  Therefore, the present invention provides an audio processing apparatus, an audio processing method, and an electronic apparatus including an audio processing apparatus that can achieve downsizing and cost reduction and can effectively obtain an audio signal intended by a user. The purpose is to do.

  In order to achieve the above object, an audio processing apparatus according to the present invention is an audio processing apparatus for processing an input audio signal, wherein the audio signal is collected underwater based on the input audio signal. A sound collection environment determination unit that determines whether or not the sound signal is underwater, and an underwater characteristic correction unit that performs an underwater characteristic correction process for reducing the influence of the sound collection characteristic of the underwater sound from the audio signal. And when the sound signal input by the sound collection environment determination unit is determined to be collected in water, the underwater characteristic correction unit performs the underwater characteristic correction process on the input sound signal. It is characterized by giving.

  Further, in the audio processing device having the above-described configuration, the underwater characteristic correction unit includes an attenuation unit that attenuates an input audio signal, and the process of attenuating a component having a frequency equal to or lower than the first frequency of the audio signal by the attenuation unit includes: It may be included in the underwater characteristic correction process.

  If comprised in this way, it will become possible to reduce the low frequency band component of the audio | voice signal where intensity | strength concentrates with the sound collection characteristic of the sound in water. Therefore, it is possible to reduce the influence of the sound collection characteristics of the underwater sound and bring it closer to the sound signal intended by the user. Note that the process of attenuating components below the first frequency includes a process of attenuating components in a predetermined band whose upper limit is the first frequency.

  Further, in the audio processing device having the above-described configuration, the underwater characteristic correction unit includes an amplification unit that amplifies an input audio signal, and the amplification unit amplifies a component having a second frequency or higher of the audio signal. It may be included in the underwater characteristic correction process.

  If comprised in this way, it will become possible to amplify the high frequency band component of the audio | voice signal to which an intensity | strength attenuate | damps by the sound collection characteristic of the sound in water. Therefore, it is possible to reduce the influence of the sound collection characteristics of the underwater sound and bring it closer to the sound signal intended by the user. Note that the process of amplifying a component of the second frequency or higher includes a process of amplifying a component in a predetermined band whose lower limit is the second frequency.

  Further, in the audio processing device having the above-described configuration, the underwater characteristic correction unit includes a smoothing unit that smoothes the input audio signal, and the process of smoothing the audio signal by the smoothing unit is the underwater characteristic correction process. It may be included.

  If comprised in this way, it becomes possible to reduce the fray noise emphasized by the sound collection characteristic of the underwater sound, noise that appears in the frequency axis direction and the time axis direction, and whose intensity protrudes from the surroundings. Therefore, it is possible to reduce the influence of the sound collection characteristics of the underwater sound and bring it closer to the sound signal intended by the user.

  The speech processing apparatus having the above configuration further includes an AGC unit that monitors and adjusts the intensity of the input audio signal in a predetermined time unit, and the underwater characteristic correction unit includes a predetermined length of time of the AGC unit. A response speed control unit for controlling the length, and a process of shortening the predetermined time length of the AGC unit by the response speed control unit may be included in the underwater characteristic correction process.

  If comprised in this way, it will react rapidly with respect to the noise emphasized by the sound collection characteristic of the sound in water, and it will become possible to reduce the intensity | strength of noise. In particular, it is possible to respond quickly to suddenly generated noise such as click noise, and to reduce the strength. Therefore, it is possible to reduce the influence of the sound collection characteristics of the underwater sound and bring it closer to the sound signal intended by the user.

  Further, in the audio processing device having the above-described configuration, the underwater characteristic correction unit includes a synthesis unit that synthesizes a predetermined signal with an input audio signal, and the synthesis unit synthesizes the predetermined signal with the audio signal. It may be included in the underwater characteristic correction process.

  Further, in the audio processing device having the above-described configuration, the underwater characteristic correction unit may have a frequency other than the frequency range based on a component in a frequency range that is greater than or equal to the third frequency and less than or equal to the fourth frequency of the input audio signal. The underwater characteristic correction process includes a restoration unit that restores another component, which is a component, and the synthesis of the other component restored by the restoration unit into an audio signal by the synthesis unit. It doesn't matter. In the audio processing device having the above-described configuration, the other frequency component may include at least one of a component having a frequency lower than the third frequency and a component having a frequency higher than the fourth frequency.

  If comprised in this way, it will restore | restore based on the component more than 3rd frequency and below 4th frequency which is a component which is hard to be influenced by the sound collection characteristic of underwater sound, and it will be influenced by the sound collection characteristic of underwater sound. It becomes possible to synthesize the restored other components with respect to the other components that are easy. Therefore, it is possible to reduce the influence of the sound collection characteristics of the underwater sound and bring it closer to the sound signal intended by the user.

  In the speech processing device having the above configuration, the underwater characteristic correction unit further includes an alternative component supply unit that supplies an alternative component that is supplied to the synthesis unit and synthesized, and the synthesis unit converts the alternative component into an audio signal. Processing to be combined may be included in the underwater characteristic correction processing.

  If comprised in this way, it will become possible to synthesize | combine arbitrary alternative components with respect to the component of the audio | voice signal which is easy to be influenced by the sound collection characteristic of the sound in water. Therefore, it is possible to reduce the influence of the sound collection characteristics of the underwater sound and bring it closer to the sound signal intended by the user.

  In the audio processing apparatus having the above-described configuration, the minimum signal selection is performed such that the audio signals of a plurality of channels are input and the audio signal having the lowest intensity is selected and output from the input audio signals of the plurality of channels. A process in which the synthesis unit synthesizes an audio signal output from the minimum signal selection unit with each of the audio signals of the plurality of channels is included in the underwater characteristic correction process. I do not care.

  With this configuration, when collecting sound signals of a plurality of channels using a plurality of microphones, even if a noise having a large intensity occurs in any of the microphones, the sound signal having the smallest intensity (that is, the intensity) Can be synthesized into the audio signals of the respective channels. Therefore, it is possible to reduce noise contained in the audio signal. Therefore, it is possible to reduce the influence of the sound collection characteristics of the underwater sound and bring it closer to the sound signal intended by the user.

  An electronic device according to the present invention includes any one of the above-described audio processing devices as an audio processing unit that realizes the correction function in an electronic device having a correction function for correcting an audio signal obtained by collecting sound. It is characterized by that.

  In addition, the electronic apparatus may be an imaging apparatus that further includes an imaging unit that creates an image signal, or may be a playback apparatus that plays back an input audio signal. Even if comprised in this way, said audio | voice processing apparatus can determine whether it is what was collected in water based on an audio | voice signal. Therefore, it can be installed in a playback device and can be determined during playback.

  Moreover, in the electronic device having the above-described configuration, the electronic device further includes a control unit that controls the operation of the own device, and the underwater characteristic correction unit provided in the sound processing unit outputs when the control unit operates the own device. A process of operating based on the control information and attenuating a component of a predetermined frequency band of an input audio signal, and a process of attenuating the component of the predetermined frequency band by the predetermined signal attenuation unit, It may be included in the underwater characteristic correction process.

  If comprised in this way, it will become possible to reduce the noise of the predetermined | prescribed frequency band emphasized by the sound collection characteristic of the sound in water, especially the noise by the self-generated drive sound which generate | occur | produces with operation | movement of an electronic device. Furthermore, it is possible to reduce the component of the predetermined frequency band of the audio signal only when noise is generated. Therefore, it is possible to reduce the influence of the sound collection characteristics of the underwater sound and bring it closer to the sound signal intended by the user.

  The audio processing method according to the present invention includes a first step for determining whether or not the audio signal is collected in water based on the input audio signal, and input in the first step. When it is determined that the sound signal to be collected has been collected underwater, an underwater characteristic correction process for reducing the influence of the sound collection characteristic of the sound in water from the sound signal is performed on the input sound signal. Two steps.

  With the configuration of the present invention, an environment in which an audio signal is collected is determined based on an input audio signal, and when it is determined that the audio signal is collected in water, It is possible to reduce the influence of the sound collection characteristic from the audio signal. Therefore, even if the sound is collected in water, it is possible to reduce the influence of the sound collection characteristics of the sound in water, so that a good audio signal can be obtained. That is, it is possible to approximate the audio signal intended by the user. Furthermore, since the sound collection environment is determined based on the input audio signal, it is not necessary to provide a special device for determining the pressure sensor or the like. Therefore, it is possible to reduce the size and cost.

  Hereinafter, embodiments of a sound processing device and an electronic apparatus including the sound processing device according to the present invention will be described with reference to the drawings. First, an imaging apparatus capable of recording an audio signal as well as an audio signal will be described as an example of an electronic apparatus including the audio processing apparatus.

<< Imaging device >>
(Basic configuration of imaging device)
First, the basic configuration of the imaging apparatus will be described with reference to FIG. FIG. 1 is a block diagram illustrating a basic configuration of an imaging apparatus according to an embodiment of the present invention.

  As shown in FIG. 1, an imaging apparatus 1 includes an image sensor 2 including a solid-state imaging device such as a CCD (Charge Coupled Devices) or a CMOS (Complementary Metal Oxide Semiconductor) sensor that converts incident light into an electrical signal, and a subject. And the lens unit 3 that adjusts the amount of light and the like. Although not shown, the lens unit 3 includes a zoom lens and a motor that changes an optical zoom magnification that is a focal length of the zoom lens.

  Further, the imaging apparatus 1 has an AFE (Analog Front End) 4 that converts an image signal that is an analog signal output from the image sensor 2 into a digital signal, and audio that is input from the left and right directions in front of the imaging apparatus 1. A stereo microphone 5 that independently converts to an electric signal, an image processing unit 6 that performs various image processing such as gradation correction processing on an image signal that is a digital signal output from the AFE 4, and a stereo microphone 5 MPEG for each of an audio processing unit 7 that converts an analog audio signal to a digital signal and performs audio correction processing, an image signal output from the image processing unit 6 and an audio signal output from the audio processing unit 7 (Moving Picture Experts Group) A compression processing unit 8 that performs compression encoding processing for moving images such as a compression method, and a compression code that is compression encoded by the compression processing unit 8 An external memory 9 for recording signals, a driver unit 10 for recording and reading compressed encoded signals in the external memory 9, and a decompression for expanding and decoding the compressed encoded signals read from the external memory 9 in the driver unit 10 And a processing unit 11.

  The imaging apparatus 1 also includes an image output circuit unit 12 that converts an image signal obtained by decoding by the expansion processing unit 11 into an analog signal for display on a display device (not shown) such as a display, and an expansion processing unit 11. And an audio output circuit unit 13 that converts an audio signal obtained by decoding into an analog signal for reproduction by a reproduction device (not shown) such as a speaker.

  The imaging apparatus 1 also stores a CPU (Central Processing Unit) 14 that controls the overall operation of the imaging apparatus 1 and a memory 15 that stores each program for performing each process and temporarily stores data when the program is executed. And a timing generator (TG) unit that outputs a timing control signal for matching the operation timing of each unit, such as a button for starting imaging, a button for adjusting imaging conditions, etc. 17, a bus line 18 for exchanging data between the CPU 14 and each block, and a bus line 19 for exchanging data between the memory 15 and each block.

  The external memory 9 may be anything as long as it can record image signals and audio signals. For example, a semiconductor memory such as an SD (Secure Digital) card, an optical disk such as a DVD, a magnetic disk such as a hard disk, or the like can be used as the external memory 9. Further, the external memory 9 may be detachable from the imaging device 1.

(Basic operation of the imaging device)
Next, the basic operation of the imaging apparatus 1 will be described with reference to FIG. First, the imaging device 1 acquires an image signal that is an electrical signal by photoelectrically converting light incident from the lens unit 3 in the image sensor 2. Then, the image sensor 2 sequentially outputs image signals to the AFE 4 in a predetermined frame period (for example, 1/60 seconds) in synchronization with the timing control signal input from the TG unit 17.

  Then, the image signal converted from the analog signal to the digital signal by the AFE 4 is input to the image processing unit 6. The image processing unit 6 converts the input image signal into an image signal composed of a luminance signal and a color difference signal, and performs various image processing such as gradation correction and contour enhancement. The memory 15 operates as a frame memory, and temporarily holds an image signal when the image processing unit 6 performs processing.

  At this time, based on the image signal input to the image processing unit 6, the lens unit 3 adjusts the position of various lenses to adjust the focus, or adjusts the aperture and adjusts the exposure. It is done. This adjustment of focus and exposure is automatically performed based on a predetermined program so as to be in an optimum state, or manually performed based on a user instruction.

  On the other hand, an audio signal converted into an electrical signal in the stereo microphone 5 is input to the audio processing unit 7. The sound processing unit 7 converts an input sound signal into a digital signal and performs sound correction processing such as noise removal and sound signal intensity control. Further, it is determined whether or not the input audio signal is collected in water, and underwater characteristic correction processing is performed based on the determination result. Details of the configuration of the audio processing unit 7 and the underwater characteristic correction process will be described later.

  The image signal output from the image processing unit 6 and the audio signal output from the audio processing unit 7 are both input to the compression processing unit 8 and compressed by the compression processing unit 8 using a predetermined compression method. At this time, the image signal and the audio signal are associated with each other in time, and the image and the sound are not shifted during reproduction. The compressed image signal and audio signal are recorded in the external memory 9 via the driver unit 10.

  In the case of recording only audio, the audio signal is compressed by the compression processing unit 8 by a predetermined compression method and recorded in the external memory 9.

  The compressed encoded signal recorded in the external memory 9 is read out to the decompression processing unit 11 based on a user instruction. The decompression processing unit 11 decompresses and decodes the compressed encoded signal, and generates an image signal and an audio signal. The image signal is output to the image output circuit unit 12 and the audio signal is output to the audio output circuit unit 13. Then, in the image output circuit unit 12 and the audio output circuit unit 13, it is converted into a format that can be reproduced by a display device or a speaker and output.

  The display device and the speaker may be integrated with the imaging device 1 or may be separated and connected to a terminal provided in the imaging device 1 with a cable or the like. It does not matter.

  Further, in a so-called preview mode in which the user confirms an image displayed on a display device or the like without recording an image signal, the image output circuit without compressing the image signal output from the image processing unit 6 It may be output to the unit 12. Further, when recording an image signal, the image signal may be output to a display device or the like via the image output circuit unit 12 in parallel with the operation of compressing the image signal by the compression processing unit 8 and recording the image signal in the external memory 9. I do not care.

(Basic configuration of the audio processing unit)
Next, the basic configuration of the audio processing unit 7 shown in FIG. 1 will be described with reference to the drawings. FIG. 2 is a block diagram illustrating the basic configuration of the audio processing unit of the imaging apparatus according to the embodiment of the present invention.

  As shown in FIG. 2, the audio processing unit 7 includes an A / D conversion unit 71 that converts two audio signals (Rch and Lch) input from the stereo microphone 5 of FIG. A sound collection environment determination unit 72 that determines a sound collection environment based on the two sound signals output from the conversion unit 71, and two sound signals output from the A / D conversion unit 71 and the sound collection environment An underwater characteristic correction unit 73 that performs underwater characteristic correction processing on an audio signal input based on the determination result of the determination unit 72.

  The sound collection environment determination unit 72 determines whether or not the input audio signal is collected in water, and outputs the determination result to the underwater characteristic correction unit 73. When the sound collection environment determination unit 72 determines that the input sound signal is collected underwater, the underwater characteristic correction unit 73 reduces the influence of the sound collection characteristic of the sound underwater from the sound signal. The underwater characteristic correction process to be performed is applied to the input audio signal and output. On the other hand, when the sound collection environment determination unit 72 determines that the input audio signal is not collected underwater, the underwater characteristic correction unit 73 does not perform the underwater characteristic correction process on the input audio signal. Output.

  By configuring as described above, it is possible to perform underwater characteristic correction processing on an audio signal collected underwater and record the corrected audio signal in the external memory 9 or output it to a speaker or the like. Become. Therefore, an audio signal intended by the user can be obtained.

  When the sound collection environment determination unit 72 determines the environment in which sound is collected based on the input audio signal, the determination result is displayed on a display provided in the imaging apparatus 1 or a display LED (Light Emitting Diode) may be emitted or flashed to notify the user. If comprised in this way, it will become possible to notify a user that determination is performed correctly.

  The user may set a normal mode in which the underwater characteristic correction process is not performed and an underwater mode in which the underwater characteristic correction process is performed by the operation of the operation unit 16. Also in this case, it may be determined whether or not the sound collection environment is underwater, and either the determination result or the set mode may be prioritized.

  In addition, the sound collection environment is determined based on the audio signal and the underwater characteristic correction process is performed on the audio signal. However, in addition to the audio signal (or instead of the audio signal), the underwater characteristic correction process is performed on the image signal. A characteristic correction process may be performed. Further, in addition to the audio signal (or instead of the audio signal), it may be determined whether it is underwater based on the image signal. For example, correction for reducing the influence of optical characteristics (such as light absorption) of water or determination using the optical characteristics of water may be performed.

  In addition, the sound processing unit 7 may include a processing unit that performs sound correction processing in addition to the underwater characteristic correction unit 73. In addition, the processing unit may perform a sound correction process on a sound signal determined not to be collected in water, or sound correction may be performed on all input sound signals. Processing may be performed.

  The basic configuration described above relates to the imaging apparatus 1, but can be applied to any electronic device as long as the electronic device has a sound collecting function. For example, the present invention can also be applied to an IC recorder or the like that is not provided with an imaging function but has only a sound collection function. In this case, the blocks (the lens unit 3, the image sensor 2, the image processing unit 6, the image output circuit unit 12, etc.) that indicate the part that acquires and processes the image signal may not be provided. Further, the present invention can also be applied to an electronic device that does not have a sound collection function but has a reproduction function for reproducing an input audio signal. Note that details of the electronic device having the reproduction function will be described later.

(Sound collection environment judgment part)
Examples of the above-described sound collection environment determination unit 72 and underwater characteristic correction unit 73 will be described below with reference to the drawings. First, each example of the sound collection environment determination unit 72 will be described, and then each example of the underwater characteristic correction unit 73 will be described. Moreover, in the following description, the same code | symbol is attached | subjected about the part similar to the basic composition mentioned above, and the detailed description is abbreviate | omitted.

<First Example of Sound Collection Environment Determination Unit>
First, a first embodiment of the sound collection environment determination unit 72 shown in FIG. 2 will be described with reference to the drawings. FIG. 3 is a block diagram showing a first example of the sound collection environment determination unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 3, the audio processing unit 7 a includes an A / D conversion unit 71, a sound collection environment determination unit 72 a, and an underwater characteristic correction unit 73.

  The sound collection environment determination unit 72a collects the input sound signal based on the difference between the frequency characteristic of the sound signal collected in the air and the frequency characteristic of the sound signal collected in water. A frequency characteristic determination unit 721 for determining a sounded environment is provided.

  Here, the frequency characteristic determination unit 721 will be described with reference to the drawings. FIG. 4 is a graph showing an example of frequency characteristics of an audio signal collected in the air, and FIG. 5 is a graph showing an example of frequency characteristics of an audio signal collected in water. In the graphs shown in FIGS. 4 and 5, the horizontal axis represents frequency and the vertical axis represents gain (attenuation).

  As shown in FIGS. 4 and 5, when sound is collected in water, attenuation in the high frequency band is larger than in the case where sound is collected in air, and intensity is concentrated in the low frequency band. Therefore, the frequency characteristic determination unit 721 determines whether or not the audio signal input to the sound collection environment determination unit 72a is collected in water using the difference in frequency characteristics.

  When it is determined by the frequency characteristic determination unit 721 that the audio signal input to the sound collection environment determination unit 72a is collected in water, the underwater characteristic correction unit 73 adds the underwater characteristic to the input audio signal. Perform correction processing and output. On the other hand, when it is determined that the sound is not collected underwater, the underwater characteristic correction unit 73 outputs the input audio signal without performing underwater characteristic correction processing.

  If the sound collection environment determination unit 72a is configured as in the present embodiment, the sound collection environment can be determined based on the collected sound signal. Therefore, it is possible to determine the sound collection environment without using a special device or a dedicated part for determining the sound collection environment. Moreover, since the difference in frequency characteristics between the air and the water is large, it is possible to increase the accuracy of the determination, and it is possible to suppress erroneous determination.

  The determination method executed by the sound collection environment determination unit 72a may be the following method. In the method of this example, two audio signals output from the A / D converter 71 are targeted, and a low band (for example, several tens (70) Hz to 3 kHz), a medium band (for example, 6 kHz to 9 kHz), and a high band. The average value of the signal level is calculated in each band (for example, 12 kHz to 15 kHz). Each band is not limited to the above specific example, and there is no problem as long as the magnitude relationship between the bands is correct. Further, the low band and the middle band may partially overlap, and the middle band and the high band may partially overlap.

  The signal level ratio of the low band to the high band (low band / high band) R1 and the signal level ratio of the low band to the medium band (low band / medium band) that can be calculated from the average value of the signal level in each band ) R2 and the signal level ratio (medium band / high band) R3 of the medium band to the high band change with time as shown in FIG. 6 when the stereo microphone 5 is inserted from the air into the water and returned to the air again. Indicates. Periods T1 and T3 in FIG. 6 are periods in which the stereo microphone 5 is located in the air, and periods T2 in FIG. 6 are periods in which the stereo microphone 5 is located in the water. The signal level ratio (medium band / high band) R3 of the medium band to the high band is a substantially constant value regardless of whether in the air or underwater. On the other hand, the low-band signal level ratio (low band / high band) R1 to the high band and the low-band signal level ratio (low band / middle band) R2 to the medium band are small values in the air. The sound collection sensitivity changes underwater, which is significantly larger than that in air.

  By utilizing this, the sound collection environment determination unit 72a uses a low-band signal level ratio (low band / high band) R1 to a high band and a low-band signal level ratio (low band / middle band) R2 to a medium band. Is calculated from the average value of the signal level in each band, and the signal level ratio of the low band to the high band (low band / high band) R1 and the signal level ratio of the low band to the medium band (low band / medium band) R2 are threshold values. When it becomes larger than the above, it is determined to be underwater. Although the determination accuracy is inferior, the average value of the signal level in the medium band and the signal level ratio of the low band to the medium band (low band / medium band) R2 are not calculated, and the signal level ratio of the low band to the high band ( Low band / high band) When R1 is greater than or equal to the threshold value, it is determined that the water is underwater, or the average value of the signal level in the high band and the signal level ratio of the low band to the high band (low band / high band) ) R1 is not calculated, and it is also possible to determine that it is underwater when the signal level ratio of the low band to the medium band (low band / medium band) R2 becomes larger than the threshold value.

  Even underwater, sudden noise occurs due to the sound of bubbles and the rubbing sound of the housing, and the signal level of the middle band and the high band increases instantaneously, and the signal level ratio of the low band to the high band (low There is a possibility that the signal level ratio (low band / medium band) R2 of the low band with respect to the band (high band) R1 and the medium band instantaneously becomes a small value. Therefore, the low-band signal level ratio (low band / high band) R1 to the high band and the low-band signal level ratio (low band / middle band) R2 to the high band used by the sound collection environment determination unit 72a for determination are: It is desirable to use an average value over a certain period of time.

  Further, regarding the threshold value, it is desirable to provide a hysteresis characteristic and set the threshold value high while determining that it is in the air, and setting the threshold value low while determining that it is underwater.

<Second Example of Sound Collection Environment Determination Unit>
Next, a second embodiment of the sound collection environment determination unit 72 shown in FIG. 2 will be described with reference to the drawings. FIG. 7 is a block diagram showing a second example of the sound collection environment determination unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 7, the audio processing unit 7 b includes an A / D conversion unit 71, a sound collection environment determination unit 72 b, and an underwater characteristic correction unit 73.

  The sound collection environment determination unit 72b collects the input sound signal based on the difference between the propagation characteristic of the sound signal collected in the air and the propagation characteristic of the sound signal collected in water. A propagation characteristic determining unit 722 is provided for determining a sounded environment.

  As a propagation characteristic of the audio signal used for the determination by the propagation characteristic determination unit 722, for example, there is a difference in sound speed. The speed of sound in the air is about 344 m / s, and the speed of sound in water is about 1500 m / s. In this example, the sound collection environment is determined based on the difference in sound speed. Further, when making such a determination, for example, a driving sound of a motor that changes the optical zoom magnification provided in the lens unit 3 can be used. The case where determination is performed in this way will be described below.

  The CPU 14 controls the drive of the motor by outputting a motor drive control signal to the motor provided in the lens unit 3. When the motor is driven, sound is collected by the stereo microphone 5 as drive sound (self-generated drive sound) in the imaging apparatus 1 shown in FIG. On the other hand, the CPU 14 performs time management of the motor drive timing, and outputs the time information to the propagation characteristic determination unit 722 of the sound collection environment determination unit 72b. The propagation characteristic determination unit 722 measures the propagation speed of the self-generated drive sound based on the sound collection time of the self-generated drive sound included in the input audio signal and the time information regarding the motor drive timing. Based on the obtained propagation speed, the propagation characteristic determination unit 722 determines whether or not the audio signal input to the sound collection environment determination unit 72b is collected in water.

  When it is determined by the propagation characteristic determination unit 722 that the audio signal input to the sound collection environment determination unit 72b is collected in water, the underwater characteristic correction unit 73 adds the underwater characteristic to the input audio signal. Perform correction processing and output. On the other hand, if it is determined that the sound is not collected underwater, the underwater characteristic correction unit 73 outputs the input audio signal without performing underwater characteristic correction processing.

  If the sound collection environment determination unit 72b is configured as in the present embodiment, the sound collection environment can be determined based on the collected sound signal as in the first embodiment. Therefore, it is possible to determine the sound collection environment without using a special device or a dedicated part for determining the sound collection environment. In addition, since the determination can be made by collecting the self-generated driving sound by itself, it is possible to make an accurate determination by accurately managing the time.

<Third Example of Sound Collection Environment Determination Unit>
Next, a third embodiment of the sound collection environment determination unit 72 shown in FIG. 2 will be described with reference to the drawings. FIG. 8 is a block diagram showing a third example of the sound collection environment determination unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 8, the audio processing unit 7 c includes an A / D conversion unit 71, a sound collection environment determination unit 72 c, and an underwater characteristic correction unit 73.

  The sound collection environment determination unit 72c includes a first determination unit 723 that determines the sound collection environment of the input audio signal based on the difference in frequency characteristics of the audio signal in air and water, and the input audio signal Comprehensive determination based on the determination results of the second determination unit 724, the first determination unit 723, and the second determination unit 724 that determine the sound collection environment based on the difference in the propagation characteristics of the audio signal in the air and in the water And a comprehensive determination unit 725 that outputs a determination result to the underwater characteristic correction unit 73.

  Similar to the frequency characteristic determination unit 721 shown in FIG. 3, the first determination unit 723 determines the difference between the frequency characteristic of the audio signal collected in the air and the frequency characteristic of the audio signal collected in water. Make a judgment using it. Similarly to the propagation characteristic determination unit 722 shown in FIG. 7, the second determination unit 724 generates the time information input from the CPU 14 and the self-generated driving sound included in the audio signal input via the stereo microphone 5. The propagation velocity is measured based on the sound collection time, and the determination is made using this. The overall determination unit 725 is input to the sound collection environment determination unit 72c only when both the first determination unit 723 and the second determination unit 724 determine that the audio signal is collected in water. It is determined that the audio signal is collected underwater.

  When it is determined by the overall determination unit 725 that the audio signal input to the sound collection environment determination unit 72c is collected in water, the underwater characteristic correction unit 73 performs underwater characteristic correction processing on the input audio signal. To output. In addition, when the first determination unit 723 and the second determination unit 724 make different determinations or determine that both are not collected in water, the underwater characteristic correction unit 73 receives the input audio signal. Are output without underwater characteristics correction.

  When the sound collection environment determination unit 72c is configured as in the present embodiment, it is possible to determine the sound collection environment based on the collected sound signal as in the first and second embodiments. Therefore, it is possible to determine the sound collection environment without using a special device or a dedicated part for determining the sound collection environment. Moreover, since determination can be performed based on a plurality of determination methods, determination can be performed more accurately.

  In addition, about the combination of the determination method in the 1st determination part 723 and the 2nd determination part 724, it is not restricted to the combination mentioned above. For example, the determination method based on the frequency characteristic or the propagation characteristic as described above may be combined with a determination method that is another determination method that is determined based on an input audio signal. Moreover, you may combine the same determination method.

  Furthermore, it is possible to use a combination of a determination method based on an input audio signal and a determination method that is not based on an input audio signal (for example, a pressure sensor described as the prior art). In the case of such a configuration, it is not possible to combine the determination method based on the input audio signal, but compared with the case of determining by combining a plurality of dedicated parts such as a pressure sensor, Cost reduction can be achieved. That is, it is possible to increase the accuracy of determination, and to reduce the size and cost.

  In addition, in this example, a configuration including two determination units (a first determination unit 723 and a second determination unit 724) is illustrated, but a configuration including two or more determination units may be employed.

<Fourth Example of Sound Collection Environment Determination Unit>
Next, a fourth embodiment of the sound collection environment determination unit 72 shown in FIG. 2 will be described with reference to the drawings. FIG. 9 is a block diagram showing a fourth example of the sound collection environment determination unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 9, the sound processing unit 7 d includes an A / D conversion unit 71, a sound collection environment determination unit 72 d, and an underwater characteristic correction unit 73.

  The sound collection environment determination unit 72d includes a propagation characteristic determination unit 726 that determines the sound collection environment of the input sound signal based on the difference in the propagation characteristics of the sound signal in air and water. However, the propagation characteristic determination unit 726 performs determination using a method different from the propagation characteristic determination unit 722 of the second embodiment illustrated in FIG. 7 and the second determination unit 724 of the third embodiment illustrated in FIG. 8.

  A determination method of the propagation characteristic determination unit 726 will be described with reference to FIG. FIG. 10 is a schematic diagram of a stereo microphone. As shown in FIG. 10, when the sound emitted from the sound source is collected by the stereo microphone 5, a path difference d is generated between the microphone 5L and the microphone 5R. Then, due to this path difference d, a difference in arrival time (a value obtained by dividing the path difference d by the speed of sound) occurs. Further, the difference in arrival time causes a phase difference between the audio signal (Rch) input from the microphone 5R and the audio signal (Lch) input from the microphone 5L.

  As described above, the speed of sound differs between air and water. Therefore, the phase difference generated between the audio signal (Rch) and the audio signal (Lch) differs depending on the audio signal collected in the air and the audio signal collected in the water. Therefore, the propagation characteristic determination unit 726 determines whether the sound signal input to the sound collection environment determination unit 72d is collected in water based on the difference in phase difference.

  When it is determined by the propagation characteristic determination unit 726 that the audio signal input to the sound collection environment determination unit 72d is collected in water, the underwater characteristic correction unit 73 corrects the underwater characteristic to the input audio signal. Process and output. On the other hand, when it is determined that the sound is not collected underwater, the underwater characteristic correction unit 73 outputs the input audio signal without performing underwater characteristic correction processing.

  When the sound collection environment determination unit 72d is configured as in the present embodiment, it is possible to determine the sound collection environment based on the collected sound signal as in the first to third embodiments. Therefore, it is possible to determine the sound collection environment without using a special device or a dedicated part for determining the sound collection environment. If a multi-channel microphone array is used and a phase difference between two or more microphones is obtained, the accuracy of determination can be further improved.

  Note that the determination may be made on the sound generated by the imaging apparatus 1. If comprised in this way, since the position of a sound source and each microphone is fixed, it becomes possible to determine with sufficient precision. Further, it may be combined with other determination methods as in the sound collection environment determination unit 72c of the third embodiment shown in FIG.

<Fifth Example of Sound Collection Environment Determination Unit>
Next, a fifth embodiment of the sound collection environment determination unit 72 shown in FIG. 2 will be described with reference to the drawings. FIG. 11 is a block diagram illustrating a fifth example of the sound collection environment determination unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 11, the sound processing unit 7 e includes an A / D conversion unit 71, a sound collection environment determination unit 72 e, and an underwater characteristic correction unit 73.

  The sound collection environment determination unit 72e includes an Rch determination unit 727 that determines a sound collection environment in the audio signal (Rch), an Lch determination unit 728 that determines a sound collection environment in the audio signal (Lch), and an Rch determination. A comprehensive determination unit 729 that performs comprehensive determination based on the determination results of the unit 727 and the Lch determination unit 728 and outputs the determination result to the underwater characteristic correction unit 73.

  The Rch determination unit 727 and the Lch determination unit 728 can be the same as the frequency characteristic determination unit 721 and the propagation characteristic determination unit 722 of the above-described embodiment. Further, only when the Rch determination unit 727 and the Lch determination unit 728 determine that the overall determination unit 729 is an audio signal collected in water, the overall determination unit 729 receives the audio signal input to the sound collection environment determination unit 72e. It is determined that the sound is collected at.

  When the overall determination unit 729 determines that the audio signal input to the sound collection environment determination unit 72e is collected in water, the underwater characteristic correction unit 73 performs underwater characteristic correction processing on the input audio signal. To output. In addition, when the Rch determination unit 727 and the Lch determination unit 728 make different determinations or determine that both are not collected in water, the underwater characteristic correction unit 73 adds an underwater characteristic to the input audio signal. Output without performing characteristic correction processing.

  Furthermore, when the determination results of the Rch determination unit 727 and the Lch determination unit 728 are different, the overall determination unit 729 outputs a warning signal to notify the user to that effect. For example, the warning signal may be used to display character information such as “sound collection environment determination error” on a display device such as a display provided in the imaging device 1. It may be provided with an LED and lighted up for notification.

  If such a warning signal is output and notified to the user, for example, when the imaging device 1 is moved from the water to the air, there is a case where one of the microphones of the stereo microphone 5 is insufficiently drained. Even if it does, it can be notified to the user. Thereby, the user can confirm that sound collection environment determination is performed appropriately. In addition, since it is possible to notify the user that there is a microphone with insufficient draining, it is possible to suppress that each microphone continuously collects sound in different sound collection environments. Become.

  If the sound collection environment determination unit 72e is configured as in the present embodiment, it is possible to determine the sound collection environment based on the collected sound signal as in the first to fourth embodiments. Therefore, it is possible to determine the sound collection environment without using a special device or a dedicated part for determining the sound collection environment.

  In addition, in the sound collection environment determination units 72a to 72e of the first to fifth embodiments, a multi-channel microphone array (for example, a 5.1ch surround recording compatible microphone) is used, and each of the input from a plurality of microphones is used. The determination may be made based on the audio signal. Further, the determination may be made based on an audio signal input from a predetermined microphone. However, the sound collection environment determination units 72d and 72e of the fourth embodiment and the fifth embodiment perform determination based on audio signals input from at least two microphones.

  In addition, the sound collection environment determination units 72a to 72c of the first to third embodiments may be determined based on an audio signal input from one channel microphone, or may be a monaural microphone. I do not care.

  Further, in the case where the determination is made using the self-generated drive sound as in the sound collection environment determination units 72b and 72c of the second and third embodiments, the drive sound of the motor provided in the lens unit 3 is used. Although described as an example, other sounds may be used.

(Underwater characteristic correction unit)
Hereinafter, each example of the underwater characteristic correction unit 73 shown in FIG. 2 will be described with reference to the drawings. In the following, for the sake of simplicity, the audio signals collected by the stereo microphone 5 are collectively described as one audio signal without distinguishing the channels, but underwater characteristic correction processing by the underwater characteristic correction processing unit 73 is performed for each channel. It does not matter if it is done.

<First Example of Underwater Characteristic Correction Unit>
First, a first embodiment of the underwater characteristic correction unit 73 shown in FIG. 2 will be described with reference to the drawings. FIG. 12 is a block diagram illustrating a first example of the underwater characteristic correction unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 12, the audio processing unit 7f includes an A / D conversion unit 71, a sound collection environment determination unit 72, and an underwater characteristic correction unit 73a.

  The underwater characteristic correction unit 73a also extracts an LPF (Low Pass Filter) 7301 that extracts and outputs a low frequency band component that is equal to or lower than a predetermined frequency in the input audio signal, and a predetermined frequency in the input audio signal. An HPF (High Pass Filter) 7302 that extracts and outputs a high frequency band component that is equal to or higher than the frequency, an attenuation unit 7303 that attenuates a low frequency band component output from the LPF 7301 based on the determination result of the sound collection environment determination unit 72, and A synthesizing unit 7304 for synthesizing the low frequency band component output from the attenuating unit 7303 and the high frequency band component output from the HPF 7302 to output an audio signal.

  The sound collection environment determination unit 72 determines whether or not the input audio signal is collected in water by the method described above. On the other hand, the underwater characteristic correction unit 73 a extracts a low frequency band component of the audio signal input in the LPF 7301 and extracts a high frequency band component of the audio signal input in the HPF 7302. The attenuation unit 7303 attenuates the low frequency band component output from the LPF 7301 when the sound collection environment determination unit 72 determines that the input audio signal is collected in water. When the sound collection environment determination unit 72 determines that the input audio signal is not collected in water, the low frequency band component is output without being attenuated. Then, the synthesis unit 7304 synthesizes the low frequency band component and the high frequency band component and outputs an audio signal.

  As shown in FIGS. 4 and 5, the frequency characteristic of the sound signal collected in the air is different from the frequency characteristic of the sound signal collected in the water. In particular, the intensity of an audio signal collected in water is concentrated in a low frequency band as compared to an audio signal collected in air. For this reason, it may be very difficult to hear at the time of reproduction, or it may become annoying, and may be far from the audio signal intended by the user.

  However, when the underwater characteristic correction unit 73a is configured as in the present embodiment, it is possible to attenuate the low frequency band component of the audio signal collected in water. Therefore, it is possible to reduce the influence of the sound collection characteristics of the sound in water from the sound signal collected in water. That is, it is possible to effectively approximate the audio signal intended by the user.

Note that the cutoff frequency of the LPF 7301 and the HPF 7302 may be set to a certain frequency λ ₁ . Further, the frequency λ ₁ may be set to 2 kHz, for example. In addition, the amount of gain attenuation by the attenuation unit 7303 may be set to 20 dB, for example.

In this example, the LPF 7301 and the HPF 7302 are used to attenuate all components having a frequency of λ ₁ or less, but a configuration in which components in a predetermined frequency band are attenuated may be used. In order to achieve such a configuration, for example, the LPF 7301 is replaced with a BPF (Band Pass Filter) that passes a component of a frequency band having an upper limit frequency λ ₁ and a lower limit frequency λ _a, and the component that has passed this BPF A configuration in which the attenuation unit 7303 attenuates may be used. Further, in this case, for example, the HPF 7302 may be replaced with a BEF (Band Elimination Filter) that passes components having _a frequency of λ ₁ or more and a frequency λ _a or less.

<Second Example of Underwater Characteristic Correction Unit>
Next, a second embodiment of the underwater characteristic correction unit 73 shown in FIG. 2 will be described with reference to the drawings. FIG. 13 is a block diagram illustrating a second example of the underwater characteristic correction unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 13, the audio processing unit 7g includes an A / D conversion unit 71, a sound collection environment determination unit 72, and an underwater characteristic correction unit 73b.

  The underwater characteristic correction unit 73b extracts an LPF 7305 that extracts and outputs a low frequency band component that is equal to or lower than a predetermined frequency in the input audio signal, and a high frequency band that is equal to or higher than the predetermined frequency in the input audio signal. The HPF 7306 that extracts and outputs the components, the amplifying unit 7307 that amplifies the high frequency band component output from the HPF 7306 based on the determination result of the sound collection environment determining unit 72, the high frequency band component that is output from the amplifying unit 7307, and the LPF 7305 And a synthesizing unit 7308 that synthesizes the low frequency band component output from the synthesizer and outputs an audio signal.

  The sound collection environment determination unit 72 determines whether or not the input audio signal is collected in water by the method described above. On the other hand, the underwater characteristic correction unit 73b extracts the low frequency band component of the audio signal input in the LPF 7305 and extracts the high frequency band component of the audio signal input in the HPF 7306. The amplification unit 7307 then amplifies the high frequency band component output from the HPF 7306 when the sound collection environment determination unit 72 determines that the input audio signal is collected in water. Further, when the sound collection environment determination unit 72 does not determine that the input audio signal is collected in water, the high frequency band component is output without being amplified. Then, the synthesis unit 7308 synthesizes the low frequency band component and the high frequency band component and outputs an audio signal.

  As described above, the frequency characteristic of an audio signal collected in air is different from the frequency characteristic of an audio signal collected in water. In particular, an audio signal collected in water has a high attenuation in the high frequency band compared to an audio signal collected in air. For this reason, it may be very difficult to hear at the time of reproduction, or it may become annoying, and may be far from the audio signal intended by the user.

  However, if the underwater characteristic correction unit 73b is configured as in the present embodiment, it is possible to amplify the high frequency band component of the audio signal collected in water. Therefore, it is possible to reduce the influence of the sound collection characteristics of the sound in water from the sound signal collected in water. That is, it is possible to effectively approximate the audio signal intended by the user.

Note that the cutoff frequency of the LPF 7305 and the HPF 7306 may be a certain frequency λ ₂ . Further, the frequency λ ₂ may be set to 4 kHz, for example. Further, the gain amplification amount by the amplifying unit 7307 may be set to 15 dB, for example.

In this example, the LPF 7305 and the HPF 7306 are used to amplify all components having a frequency of λ ₂ or more. However, a configuration in which components in a predetermined frequency band are amplified may be used. For example, the HPF 7306 may be replaced with a BPF that passes a component of a frequency band having a lower limit frequency λ ₂ and an upper limit frequency λ _b, and the amplification unit 7307 may amplify the component that has passed this BPF. Further, in this case, for example, the LPF 7305 may be replaced with a BEF that passes components having a frequency of λ ₂ or more and a frequency λ _b or less.

<Third Example of Underwater Characteristic Correction Unit>
Next, a third embodiment of the underwater characteristic correction unit 73 shown in FIG. 2 will be described with reference to the drawings. FIG. 14 is a block diagram showing a third example of the underwater characteristic correction unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 14, the sound processing unit 7h includes an A / D conversion unit 71, a sound collection environment determination unit 72, and an underwater characteristic correction unit 73c.

  Further, the underwater characteristic correction unit 73c attenuates a predetermined signal band that attenuates a component of a predetermined frequency band in the input audio signal based on the determination result of the sound collection environment determination unit 72 and the control information output from the CPU 14. Part 7309. The predetermined signal attenuating unit 7309 may be provided with a filter that attenuates components in a predetermined frequency band such as a notch filter.

  The sound collection environment determination unit 72 determines whether or not the input sound signal is collected in water by the method described above, and inputs the sound signal to the predetermined signal attenuation unit 7309. Further, the CPU 14 outputs a motor drive control signal for driving the motor that changes the optical zoom magnification of the lens unit 3 described above to the lens unit 3, and outputs control information indicating the drive information to the predetermined signal attenuating unit 7309. . The predetermined signal attenuation unit 7309 detects the driving of the motor of the lens unit 3 based on the control information.

  Then, the sound collection environment determination unit 72 determines that the input audio signal is collected underwater, and a predetermined signal when control information indicating that the motor is driven is output from the CPU 14. The attenuator 7309 operates and attenuates a component of a predetermined frequency band of the input audio signal.

  If the underwater characteristic correction unit 73c is configured as in the present embodiment, it is possible to attenuate self-generated driving noise generated when, for example, a motor is driven. This self-generated driving sound noise is shown in FIG. FIG. 15 is a graph showing self-generated driving sound noise included in an audio signal collected in water. FIG. 15 is a graph similar to FIG. 4 and FIG. 5 and shows the low frequency band in an enlarged manner.

  FIG. 15 shows a case where self-generated driving noise (portion surrounded by a broken line in the figure) is generated around 0.5 kHz. The frequency of the self-generated drive sound like noise because, depending on the device used in the imaging apparatus 1, the self-generating drive sound noise can know generated in advance what frequency band. Therefore, by setting the predetermined signal attenuating unit 7309 to attenuate the frequency band component (in the case of FIG. 15, a signal around 0.5 kHz), the self-generated driving sound noise can be attenuated.

  In particular, when sound is collected in water, the sound collection sensitivity to noise signals such as self-generated drive sound noise is increased and the sound is easily emphasized. For this reason, the self-generated driving sound noise does not become harsh when collecting sound in the air, but it becomes harsh when sound is collected in water.

  However, if the self-generated driving sound noise is attenuated when it is determined that the sound signal is collected underwater as in the present embodiment, it is possible to effectively reduce a noise signal that is easily emphasized. . Therefore, it is possible to reduce the influence of the sound collection characteristics of the sound in water from the sound signal collected in water. That is, it is possible to effectively approximate the audio signal intended by the user.

  Further, when the imaging apparatus 1 is driven, the self-generated driving sound noise can be selectively attenuated by attenuating the frequency band component of the self-generating driving sound noise. Therefore, it is possible to suppress deterioration of the audio signal due to attenuation as much as possible.

  If the control information output from the CPU 14 is the same as the time information described above, it is possible to perform sound collection environment determination and underwater characteristic correction processing based on one piece of information. Therefore, it is possible to simplify the configuration and operation of the CPU 14 and the sound processing unit 7h. At this time, as the control information from the CPU 14 is input to the sound collection environment determination unit 72, the predetermined signal attenuation unit 7309 determines whether or not the operation is necessary based on only the signal from the sound collection environment determination unit 72. You may comprise as follows.

  Further, the self-generated driving noise is not necessarily caused by driving of a motor provided in the lens unit 3. That is, even self-generated driving noise generated due to other devices can be reduced.

<Fourth Example of Underwater Characteristic Correction Unit>
Next, a fourth embodiment of the underwater characteristic correction unit shown in FIG. 2 will be described with reference to the drawings. FIG. 16 is a block diagram illustrating a fourth example of the underwater characteristic correction unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 16, the audio processing unit 7k includes an A / D conversion unit 71, a sound collection environment determination unit 72, and an underwater characteristic correction unit 73d.

  Further, the underwater characteristic correction unit 73d includes a smoothing unit 7310 that performs smoothing in the input audio signal based on the determination result of the sound collection environment determination unit 72. Note that the smoothing unit 7310 may include, for example, a filter that performs a smoothing process such as a smoothing filter.

  The sound collection environment determination unit 72 determines whether or not the input audio signal is collected underwater by the method as described above, and inputs the sound signal to the smoothing unit 7310. When the sound collection environment determination unit 72 determines that the input audio signal is collected in water, the smoothing unit 7310 operates to smooth the input audio signal.

  If the underwater characteristic correction unit 73d is configured as in the present embodiment, it is possible to reduce fraying noise. This fraying noise is shown in FIG. FIG. 17 is a graph showing fraying noise and smoothing processing included in an audio signal collected in water. FIG. 17 is a graph similar to FIG. 4, FIG. 5 and FIG.

  As shown in FIG. 17A, when fray noise (a portion surrounded by a broken line in the figure) is generated, the intensity of a signal having an unspecified frequency is locally attenuated. Therefore, the smoothing unit 7310 performs smoothing on the input audio signal to suppress fraying noise. FIG. 17 (b) also shows the audio signal after smoothing (solid line) and the audio signal for comparison before smoothing (broken line). As shown in FIG. 17B, it is possible to reduce the attenuation amount of the locally attenuated portion by performing smoothing. That is, it becomes possible to reduce fraying noise.

  As described above, when sound is collected in water, the sound collection sensitivity with respect to the noise signal is increased and the sound signal is easily emphasized. That is, the fray noise is particularly easily emphasized by an audio signal collected in water. For this reason, fraying noise is not harsh when collecting sound in the air, but it becomes harsh when collecting sound in water.

  However, when the sound collection environment determination unit 72 determines that the sound signal is collected underwater as in this embodiment, the noise signal that is easily emphasized is effectively reduced. It becomes possible to reduce. Therefore, it is possible to reduce the influence of the sound collection characteristics of the sound in water from the sound signal collected in water. That is, it is possible to effectively approximate the audio signal intended by the user.

  It should be noted that this embodiment can be applied not only to fray noise but also to noise that appears in the frequency axis direction in general. For example, noise of a predetermined frequency (self-generated driving sound noise) associated with focus, zoom, or the like can be dealt with using the smoothing unit 7310 of this embodiment instead of using the above-described predetermined signal attenuation unit 7309. it can.

  Further, as a smoothing process performed by the smoothing unit 7310, a method using an addition average value can be used. The addition average value x ′ [f] used in this method is obtained by adding the signal value x [i] of each frequency i (Hz) included in a certain frequency width Fa (Hz) with the frequency f (Hz) as the center. Then, the total value is obtained by dividing by Fa. Specifically, it is obtained by the calculation formula shown in the following formula (1).

  When the obtained average value x ′ [f] is smaller than the signal value x [f], the value of the average value x ′ [f] is employed as the signal value at the frequency f. On the other hand, when the addition average value x ′ [f] is larger than the signal value x [f], the value of the signal value x [f] is adopted as it is. FIG. 18 shows a schematic diagram when the smoothing unit 7310 performs such smoothing processing. FIG. 18 is a graph showing the noise due to the focus sound and the processing of the smoothing unit. A broken line represents a signal value before the smoothing process, and a solid line represents a signal value after the smoothing process.

  As shown in FIG. 18, in the signal before the smoothing process, the intensity rapidly increases in some bands. In addition, since such noise is often a signal in a low frequency band, it becomes a signal that is easily emphasized in water as described above. Therefore, a signal in such a band is heard as noise on hearing. However, if smoothing processing is performed as in the present embodiment, a signal as indicated by a solid line can be obtained. That is, it is possible to reduce the noise by reducing the signal value in the band where the intensity suddenly increases.

  In formula (1), when calculating x ′ [f], each value of Fa (pieces) of x [f− (Fa / 2) +1] to x [f + (Fa / 2)] is summed. However, the values of x [f− (Fa / 2)] to x [f + (Fa / 2) −1] may be summed. Moreover, although the example mentioned above is a thing when Fa is an even number, when Fa is an odd number, x [f- (Fa / 2) +1/2] to x [f + (Fa / 2) -1/2]. These values may be summed and divided by Fa to calculate the addition average value x ′ [f].

  Furthermore, not only noise appearing on the frequency axis but also noise appearing in the time axis direction can be dealt with by performing a smoothing process. In this case, it is possible to apply a process using the addition average value similar to the above-described equation (1). In this case, however, the average value is taken in the time axis direction.

  The calculated average value x ′ [t] is obtained by adding the signal values x [k] of each time k (sec) included in a certain time width Ta (sec) with the time t (sec) as the center, It is obtained by dividing by Ta. Specifically, it is obtained by the calculation formula shown in the following formula (2).

  As in the case of reducing the noise appearing in the frequency axis direction described above, when the obtained average value x ′ [t] is smaller than the signal value x [t], the average value is obtained as the signal value at time t. The value of x ′ [t] is adopted. On the other hand, when the addition average value x ′ [t] is larger than the signal value x [t], the value of the signal value x [t] is adopted as it is. By performing such smoothing processing, for example, even if sudden and large noise occurs due to contact of the housing of the imaging apparatus with something, it is possible to reduce the generated noise.

  In equation (2), when calculating x ′ [t], the respective values of Ta (pieces) of x [t− (Ta / 2) +1] to x [t + (Ta / 2)] are summed. However, the values of x [t− (Ta / 2)] to x [t + (Ta / 2) −1] may be summed. Moreover, although the example mentioned above is a thing when Ta is an even number, Ta is an odd number, and x [t− (Ta / 2) +1/2] to x [t + (Ta / 2) −1/2]. These values may be summed and divided by Ta to calculate the addition average value x ′ [t].

  In addition, as an example of the smoothing process performed in the frequency axis direction and the time axis direction, the case where the addition average value is used has been described, but the present embodiment is not limited to this method. In particular, other methods may be used as long as smoothing processing can be performed.

<Fifth Example of Underwater Characteristic Correction Unit>
Next, a fifth embodiment of the underwater characteristic correction unit 73 shown in FIG. 2 will be described with reference to the drawings. FIG. 19 is a block diagram illustrating a fifth example of the underwater characteristic correction unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 19, the audio processing unit 7m has an AGC (Automatic Gain Control) unit 74 that automatically controls the gain of the input audio signal, and an A signal to which the audio signal output from the AGC unit 74 is input. / D conversion part 71, the sound collection environment determination part 72, and the underwater characteristic correction | amendment part 73e are provided. In addition, the underwater characteristic correction unit 73 e includes a response speed control unit 7311 that controls the response speed of the AGC unit 74 based on the determination result of the sound collection environment determination unit 72. Although only the AGC unit 74 is shown in FIG. 19 showing the present embodiment, the AGC unit 74 may be provided in other embodiments.

  The AGC unit 74 monitors the intensity of the input audio signal, attenuates when the intensity of the audio signal is greater than a set value, and amplifies when the intensity of the audio signal is less than the set value. By operating in this way, the AGC unit 74 controls the intensity of the audio signal to approach the intensity suitable for subsequent processing.

  A specific operation example of the AGC unit 74 will be described with reference to FIG. FIG. 20 is a graph showing a specific example of the operation of the AGC unit. The wave in the figure schematically shows an audio signal. Moreover, the broken lines described above and below the wave indicate the time change of the amplitude (intensity) of the audio signal. In FIG. 20, the intensity of the audio signal in the section I is equal to the set value, the intensity in the section II is smaller than the intensity in the section I, and the intensity in the section III is larger than the intensity in the section I. To do.

  As shown in FIG. 20, when the AGC unit 74 confirms that the intensity of the audio signal has become smaller than the set value in the section II, the rate of increase for making the intensity equal to the set value is in the section (one). Is set. In the section (2), the audio signal is amplified at the rate of increase set in the section (1), and the intensity of the audio signal is adjusted to be equal to the set value at the end of the section (2). On the other hand, when the AGC unit 74 confirms that the intensity of the audio signal has become larger than the set value in the section III, a decrease rate for setting the intensity equal to the set value is set in the section (3). . In the section (4), the audio signal is attenuated at the rate of decrease set in the section (3), and the intensity of the audio signal is adjusted to be equal to the set value at the end of the section (4).

  In addition, the sound collection environment determination unit 72 determines whether or not the input audio signal is collected underwater by the method as described above, and inputs it to the response speed control unit 7311. When the sound collection environment determination unit 72 determines that the input audio signal is collected in water, the response speed control unit 7311 does not determine that the response speed in the AGC unit 74 is underwater. Control to make faster than. For example, when the AGC unit 74 performs control as shown in FIG. 20, the response control unit 7311 performs control to shorten the sections (1), (2), (3), and (4).

  When the underwater characteristic correction unit 73e is configured as in the present embodiment, it is possible to reduce sudden noise such as click noise. This sudden noise is shown in FIG. FIG. 21 is a graph showing sudden noise included in an audio signal collected in water and processing of the response speed control unit. In the graph shown in FIG. 21, the horizontal axis represents time and the vertical axis represents gain. FIG. 21A is a graph showing a state in which the response speed control unit 7311 does not perform control to increase the response speed of the AGC unit 74 (a state in which the response speed is the same as that in the case where it is not determined to be underwater). FIG. 21B is a graph showing a state in which control is performed to increase the response speed.

  As shown in FIG. 21A, when sudden noise (portion surrounded by a broken line in the figure) occurs, a portion where the signal intensity locally increases in the time axis direction is generated. In order to suppress this local increase in signal strength, the response speed control unit 7311 controls the AGC unit 74 to increase the response speed. FIG. 21B shows a graph of the audio signal (solid line) when the response speed is increased and a graph of the audio signal (dashed line) before increasing the response speed for comparison. As shown in FIG. 21B, when the response speed is increased, AGC is quickly performed against noise. Therefore, it is possible to suppress an increase in local signal strength. That is, sudden noise can be reduced.

  As described above, when sound is collected in water, the sound collection sensitivity with respect to the noise signal is increased and the sound signal is easily emphasized. That is, sudden noise is particularly easily emphasized by an audio signal collected in water. For this reason, sudden noise does not become harsh when collecting sound in the air, but it becomes harsh when collected in water.

  However, in the case where the sound collection environment determination unit 72 determines that the sound signal is collected underwater as in the present embodiment, if the sudden noise is reduced, a noise signal that is easily emphasized can be effectively used. It becomes possible to reduce. Therefore, it is possible to reduce the influence of the sound collection characteristics of the sound in water from the sound signal collected in water. That is, it is possible to effectively approximate the audio signal intended by the user.

  In addition, the case where the signal intensity locally decreases due to sudden noise can be similarly reduced. In this case, the signal is rapidly amplified by the AGC unit 74, thereby suppressing a decrease in signal strength.

<Sixth Example of Underwater Characteristic Correction Unit>
Next, a sixth embodiment of the underwater characteristic correction unit 73 shown in FIG. 2 will be described with reference to the drawings. FIG. 22 is a block diagram showing a sixth example of the underwater characteristic correction unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 22, the audio processing unit 7n includes an A / D conversion unit 71, a sound collection environment determination unit 72, and an underwater characteristic correction unit 73f.

  The underwater characteristic correction unit 73f also extracts an LPF 7312 that extracts and outputs a low frequency band component that is equal to or lower than a predetermined frequency in the input audio signal, and an intermediate frequency that is a predetermined frequency band in the input audio signal. A BPF 7313 that extracts and outputs a band component, an HPF 7314 that extracts and outputs a high-frequency band component that is equal to or higher than a predetermined frequency in the input audio signal, and a mixing coefficient α that is output from the sound collection environment determination unit 72. From the coefficient integrating unit 7315 that integrates the low frequency band component output from the LPF 7312, the restoring unit 7316 that restores the audio signal based on the medium frequency band component output from the BPF 7313, and outputs the restored signal, and the restoring unit 7316 An LPF 7317 that extracts a low frequency band from an output restoration signal and outputs a restored low frequency band component, and an output from the sound collection environment determination unit 72 The coefficient integration unit 7318 for integrating the mixed coefficient (1-α) to be output to the restored low frequency band component output from the LPF 7317, and the low frequency band component output from the coefficient integration unit 7315 and the restoration output from the coefficient integration unit 7318 A synthesis unit 7319 that synthesizes the low frequency band component and outputs a synthesized low frequency band component, a synthesized low frequency band component output from the synthesis unit 7319, an intermediate frequency band component output from the BPF 7313, and an HPF 7314. And a synthesizing unit 7320 that synthesizes the high frequency band component and outputs an audio signal.

  The sound collection environment determination unit 72 determines whether or not the input audio signal is collected in water by the method described above. On the other hand, the underwater characteristic correction unit 73f extracts a low frequency band component of the audio signal input in the LPF 7312. Further, the medium frequency band component of the audio signal input by the BPF 7313 is extracted, and the high frequency band component of the audio signal input by the HPF 7314 is extracted.

  When the sound collection environment determination unit 72 determines that the input audio signal is collected underwater, the coefficient integration unit 7315 adds the mixing coefficient α to the low frequency band component output from the LPF 7312. Accumulate and output. Further, the medium frequency band component output from the BPF 7313 is restored by the restoration unit 7316 and output as a restoration signal. At this time, the restoration unit 7316 performs nonlinear processing on the inputted intermediate frequency band component. Non-linear processing includes, for example, squaring processing, full-wave rectification processing (absolute value processing), half-wave rectification processing, and the like.

  When the squaring process is used, the restoration unit 7316 squares the medium frequency band component output from the BPF 7313. At this time, since the harmonic component (overtone component of the fundamental frequency) of the pitch signal of the sound collected by the stereo microphone 5 is included in the medium frequency band component, the difference and sum of each harmonic component is obtained by squaring. A signal having a frequency corresponding to is generated. Therefore, harmonic components (fundamental harmonic components and fundamental wave components that are fundamental frequencies) are generated in the lower frequency band and the higher frequency band than the medium frequency band component. Therefore, it is possible to restore the low frequency band component and the high frequency band component from the medium frequency band component. When the squaring process is performed, the amplitude of the generated harmonic component becomes the square of the harmonic component originally desired. On the other hand, normalization may be performed on the squared signal to thereby adjust the amplitude.

  The full wave rectification process (absolute value process) and the half wave rectification process can be performed in the same manner. For example, when using full-wave rectification, the absolute value of the medium frequency band component is calculated and restored. As the above restoration method, for example, the methods described in JP-A-8-130494, JP-A-8-278800, JP-A-9-55778 may be used.

  The restoration unit 7316 performs restoration by using the method as described above, and outputs a restoration signal. The LPF 7317 generates and outputs a restored low frequency band component by extracting the low frequency band component from the inputted restored signal. Then, the restored low frequency band component is input to the coefficient integration unit 7318, and the mixing coefficient (1-α) is integrated. Further, the restored low frequency band component integrated with the mixing coefficient (1-α) and the low frequency band component integrated with the mixing coefficient α are combined by the combining unit 7319 and output as one combined low frequency band component. Is done.

  Then, the synthesized low frequency band component output from the synthesizing unit 7319, the intermediate frequency band component output from the BPF 7313, and the high frequency band component output from the HPF 7314 are synthesized by the synthesizing unit 7320 as one audio signal. Is output.

  When the underwater characteristic correction unit 73f is configured as in the present embodiment, it is possible to improve the signal of the low frequency band component. This improvement method will be described with reference to FIG. FIG. 23 is a schematic graph showing both an audio signal collected in water and an audio signal collected in the air, and is a graph similar to FIGS. 4, 5, 15, and 17. It is.

As described above, the sound signal collected in water (solid line) concentrates in the low frequency band and attenuates the intensity in the high frequency band greatly compared to the sound signal collected in the air (broken line). Therefore, an audio signal collected in water is different from an audio signal collected in air. However, as shown in FIG. 23, in the middle frequency band between the frequency λ ₃ and the frequency λ ₄ , the sound signal collected in water and the sound signal collected in air are the same. That is, this band is not easily affected by the sound collection environment, and even if it is a sound signal collected in water, it is likely to be a good signal. Therefore, a signal in another band restored based on the signal in this band is likely to be a better signal than an intact signal collected in water.

  Therefore, when the sound collection environment determination unit 72 determines that the sound signal is collected in water as in the present embodiment, it is restored to the synthesized low frequency band component output from the synthesis unit 7319. Is included, it is possible to improve the audio signal output from the synthesis unit 7320. Therefore, it is possible to reduce the influence of the sound collection characteristics of the sound in water from the sound signal collected in water. That is, it is possible to effectively approximate the audio signal intended by the user.

In order to enable the above-described synthesis, the cutoff frequency of the LPF 7312 and the LPF 7317 may be set to λ ₃ and the cutoff frequency of the HPF may be set to λ ₄ . Further, the cutoff frequency on the low frequency side of the BPF 7313 may be λ ₃ and the cutoff frequency on the high frequency side may be λ ₄ . Further, λ ₃ may be set to 2 kHz, for example, and λ ₄ may be set to 6 kHz, for example.

  In addition, when the sound collection environment determination unit 72 determines that the sound is collected in water, the mixing coefficient α is decreased, and when it is determined that the sound is not collected in water, the mixing coefficient α is increased. It doesn't matter. Furthermore, the mixing coefficient α may be set to 1 when the sound collection environment determination unit 72 determines that the sound is not collected in water. When the mixing coefficient α is 1, the low frequency band component output from the LPF 7312 passes through the coefficient accumulating unit 7315 and the combining unit 7319 as it is and is input to the combining unit 7320. That is, the combined low frequency band component output from the combining unit 7319 is equal to the low frequency band component. At this time, the restoration unit 7316 may not perform the restoration process.

  Further, the degree of deterioration of the input audio signal may be determined based on the intensity of the input audio signal, and the value of the mixing coefficient α may be changed. Further, when it is determined that the degree of deterioration is large, the mixing coefficient α may be reduced. Further, the mixing coefficient α may be 0.

  Further, the underwater characteristic correction unit 73f shown in FIG. 22 is configured to restore the low frequency band component from the medium frequency band component, but similarly, restores the high frequency band component to create a restored high frequency band component, which is output from the HPF 7314. It is also possible to synthesize the high frequency band component to be synthesized at a predetermined ratio.

This configuration is shown in FIG. FIG. 24 is a block diagram illustrating another example of the sixth example of the underwater characteristic correction unit provided in the sound processing unit according to the embodiment of the present invention. In the underwater characteristic correction unit 73fa shown in FIG. 24, an HPF 7321 that extracts a high-frequency band component from the restoration signal output from the LPF 7312, the BPF 7313, the HPF 7314, the restoration unit 7316, and the restoration unit 7316 and outputs the restored high-frequency band component. A coefficient integration unit 7322 that integrates the mixing coefficient α ₁ output from the sound collection environment determination unit 72 with the high frequency band component output from the HPF 7314, and a mixing coefficient (1-α that is output from the sound collection environment determination unit 72 ₁ ) a coefficient integrating unit 7323 for integrating the restored high frequency band component output from the HPF 7321, a high frequency band component output from the coefficient integrating unit 7322, and a restored high frequency band component output from the coefficient integrating unit 7323 A synthesizing unit 7324 that outputs a synthetic high frequency band component, and a synthetic high frequency output from the synthesizing unit 7324 Comprises a combining unit 7325 for outputting an audio signal by synthesizing the low frequency band component output from the frequency band component and LPF7312 in output from the frequency components and BPF7313.

  The underwater characteristic correction unit 73fa is different from the underwater characteristic correction unit 73f in that the high frequency band component is restored and combined instead of the low frequency band component. However, since other operations are the same, description thereof is omitted.

  Further, the low frequency band component and the high frequency band component are restored to create the restored low frequency band component and the restored high frequency band component, respectively, and the low frequency band component output from the LPF 7312 and the high frequency band component output from the HPF 7314 Each of them may be combined at a predetermined ratio.

  This configuration is shown in FIG. FIG. 25 is a block diagram illustrating another example of the sixth example of the underwater characteristic correction unit provided in the sound processing unit according to the embodiment of the present invention. 25, the LPF 7312, the BPF 7313, the HPF 7314, the restoration unit 7316, the LPF 7317, the HPF 7321, the coefficient integration unit 7315, the coefficient integration unit 7322, the coefficient integration unit 7318, and the coefficient Integration unit 7323, synthesis unit 7319, synthesis unit 7324, synthesized low frequency band component output from synthesis unit 7319, medium frequency band component output from BPF 7313, and synthesized high frequency band component output from synthesis unit 7324 And a synthesizing unit 7326 that outputs an audio signal.

  The underwater characteristic correction unit 73fb is different from the underwater characteristic correction unit 73f and the underwater characteristic correction unit 73fa in that both low frequency band components and high frequency band components are restored and combined. However, since other operations are the same, description thereof is omitted.

  The sound signal collected in the water is greatly influenced by the sound collection characteristics of the sound in the water, unlike the sound signal collected in the air, both in the low frequency band component and the high frequency band component. . For this reason, the underwater characteristic correction units 73f, 73fa, and 73fb are configured to synthesize a component restored to one or both of the high-frequency band component and the low-frequency band component of the audio signal. The influence can be reduced. That is, it is possible to effectively approximate the audio signal intended by the user.

<Seventh Example of Underwater Characteristic Correction Unit>
Next, a seventh embodiment of the underwater characteristic correction unit 73 shown in FIG. 2 will be described with reference to the drawings. FIG. 26 is a block diagram showing a seventh example of the underwater characteristic correction unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 26, the audio processing unit 7p includes an A / D conversion unit 71, a sound collection environment determination unit 72, and an underwater characteristic correction unit 73g.

  The underwater characteristic correction unit 73g extracts a predetermined frequency component from the input audio signal and outputs a predetermined frequency band component, and the predetermined component extraction unit 7327 extracts the input audio signal. A predetermined component reduction unit 7328 that outputs a non-predetermined frequency band component by reducing frequency components, an alternative component supply unit 7329 that outputs an alternative component, and a predetermined frequency band component output from the predetermined component extraction unit 7327 collects sound. A coefficient integration unit 7330 that integrates the mixing coefficient β output from the environment determination unit 72, and a mixing coefficient (1-β) output from the sound collection environment determination unit 72 is integrated into the alternative component output from the alternative component supply unit And combining the predetermined frequency band component output from the coefficient integration unit 7330 and the alternative component output from the coefficient integration unit 7331 to generate a combined predetermined frequency band component. A synthesizing unit 7332 for outputting the minute, a synthesizing unit 7333 for synthesizing the synthesized predetermined frequency band component output from the synthesizing unit 7332 and the non-predetermined frequency band component output from the predetermined component reducing unit, and outputting an audio signal; Is provided.

  The sound collection environment determination unit 72 determines whether or not the input audio signal is collected in water by the method described above. On the other hand, the underwater characteristic correction unit 73g extracts a predetermined frequency band component of the input audio signal in the predetermined component extraction unit 7327. Further, the predetermined component reduction unit 7328 reduces the predetermined frequency band component of the input audio signal to obtain a non-predetermined frequency band component. Further, the substitute component supply unit 7329 outputs a substitute component that is a component in the same frequency band as the predetermined frequency band component. The predetermined frequency band component extracted by the predetermined component extraction unit 7327 is input to the coefficient integration unit 7330, and the alternative component output from the alternative component supply unit 7329 is input to the coefficient integration unit 7331.

  When the sound collection environment determination unit 72 determines that the input audio signal is collected in water, the coefficient integration unit 7330 adds the mixing coefficient β to the input predetermined frequency band component. Output. In addition, the coefficient accumulating unit 7331 accumulates and outputs the mixing coefficient (1-β) to the input substitute component. The predetermined frequency band component integrated with the mixing coefficient β and the alternative component integrated with the mixing coefficient (1-β) are combined by the combining unit 7332 and output as one combined predetermined frequency band component.

  The synthesized predetermined frequency band component output from the synthesizing unit 7332 and the non-predetermined frequency band component output from the predetermined component reducing unit 7328 are synthesized by the synthesizing unit 7333 and output as one audio signal.

  The substitute component output from the substitute component supply unit 7329 is obtained, for example, by extracting a component having the same frequency as the predetermined frequency band component from the substitute signal obtained by collecting sound in advance. This substitute component is a signal that the user wants to substitute and is arbitrary. For example, it may be obtained by collecting sound during silence in the air. Further, the sound may be such that a water bubble can be played, or may be an artificial sound or silence.

  If the underwater characteristic correction unit 73g is configured as in the present embodiment, it is possible to improve the signal of the predetermined frequency band component. In particular, low-frequency band components that concentrate when collecting sound in water, high-frequency band components that attenuate greatly, and predetermined frequency bands that contain self-generated drive sound that is sensitive and easily harsh. Alternative components that are not affected by the sound collection characteristics of the underwater sound (including those that are not significantly affected by the sound collection characteristics of the underwater sound such as low noise) can be synthesized. Therefore, it is possible to reduce the influence of the sound collection characteristics of the sound in water from the sound signal collected in water. That is, it is possible to effectively approximate the audio signal intended by the user.

  Note that the combination of the predetermined component extraction unit 7327 and the predetermined component reduction unit 7328 may be a combination of LPF and HPF having the same cutoff frequency. Further, the predetermined component extraction unit 7327 may be a BPF, and the predetermined component reduction unit 7328 may be a BEF that blocks a frequency band component extracted by the BPF.

  As in the third embodiment, the alternative component supply unit 7329 and the sound collection environment determination unit 72 detect that a self-generated driving sound (for example, a motor driving sound) has occurred, and respond to the generation of the self-generated driving sound. It is also possible to synthesize alternative components. Further, the substitute component at this time may be self-generated driving sound that is suppressed to such an extent that it does not become harsh.

  Further, when creating an alternative component from an alternative signal obtained by collecting sound in advance, a portion that is suitable for use as an alternative component from an audio signal obtained by collecting sound in air for a certain period of time (for example, A substitute signal may be created by taking out and combining a portion that does not include a signal such as noise and is stable at a constant intensity. When such a substitute signal is created and synthesized from such a substitute signal, a voice signal intended by the user can be obtained more effectively.

  In addition, a plurality of alternative components may be recorded, and these may be randomly output and combined. With such a configuration, it is possible to shorten the total time of the alternative components to be stored. Therefore, the data amount of the substitute component is reduced, and the apparatus for recording the substitute component can be downsized. In addition, since the substitute component is not a repetition of a predetermined pattern, a natural audio signal can be obtained.

  Further, the alternative component may be stored in the alternative component supply unit 7329, or the alternative component may be stored in the memory 15 or the external memory 9 shown in FIG. Further, the alternative signal may be stored, and an alternative component that is appropriately required by the alternative component supply unit 7329 may be extracted from the alternative signal and output to the coefficient integration unit 7331.

  Further, when the sound collection environment determination unit 72 determines that the sound is collected in water, the mixing coefficient β is decreased, and when it is determined that the sound is not collected in water, the mixing coefficient β is increased. It doesn't matter. Furthermore, the mixing coefficient β may be set to 1 when the sound collection environment determination unit 72 determines that the sound is not collected in water. When the mixing coefficient β is 1, the predetermined frequency band component output from the predetermined component extraction unit 7327 passes through the coefficient integration unit 7330 and the synthesis unit 7332 as it is and is input to the synthesis unit 7333. That is, the combined predetermined frequency band component output from the combining unit 7332 is equal to the predetermined frequency band component. At this time, the substitute component may not be output from the substitute component supply unit 7329.

  Also, the degree of deterioration of the input audio signal may be determined based on the intensity of the input audio signal, and the value of the mixing coefficient β may be changed. Further, when it is determined that the degree of deterioration is large, the mixing coefficient β may be reduced. Further, the mixing coefficient β may be 0.

  In this example, the alternative component is synthesized with the component of the predetermined frequency extracted by the predetermined component extraction unit 7327. However, the configuration may be such that the alternative component or the alternative signal is synthesized in all frequency bands.

<Eighth Example of Underwater Characteristic Correction Unit>
Next, an eighth embodiment of the underwater characteristic correction unit 73 shown in FIG. 2 will be described with reference to the drawings. FIG. 27 is a block diagram showing an eighth example of the underwater characteristic correction unit provided in the sound processing unit according to the embodiment of the present invention. As shown in FIG. 27, the audio processing unit 7q includes an A / D conversion unit 71, a sound collection environment determination unit 72, and an underwater characteristic correction unit 73h. Also, it is assumed that n + 1 channel audio signals of 0ch to nch (where n is a natural number) are input to the audio processing unit 7q.

  The underwater characteristic correction unit 73h selects a minimum-signal selection unit 7334 that selects and outputs the audio signal having the minimum intensity from the input multiple-channel audio signals, and the input multiple-channel audio signals. A coefficient integration unit 7335-7337 that integrates the mixing coefficient (1-γ), a coefficient integration unit 7338 that integrates the mixing coefficient γ into the audio signal output from the minimum signal selection unit 7334, and a coefficient integration unit 7335-7337, respectively. Synthesizer 7339 to 7341 for synthesizing the audio signal output from the audio signal and the audio signal output from the coefficient accumulating unit 7338 and outputting them as audio signals of the respective channels.

  The sound collection environment determination unit 72 determines whether or not the input audio signal is collected in water by the method described above. On the other hand, the underwater characteristic correction unit 73h selects the audio signal having the lowest intensity from the plurality of input audio signals in the minimum signal selection unit 7334. The selected audio signal is input to the coefficient integration unit 7338. The audio signals of the respective channels input to the underwater characteristic correction unit 73h are also input to the coefficient integration units 7335 to 7337, respectively.

  When the sound collection environment determination unit 72 determines that the input audio signal is collected in water, the coefficient integration unit 7338 adds the mixing coefficient to the audio signal output from the minimum signal selection unit 7334. γ is integrated and output. In addition, the coefficient accumulating units 7335 to 7337 accumulate and output the mixing coefficient (1-γ) to the input audio signals of the respective channels. Then, the audio signal integrated with the mixing coefficient γ and the audio signal of each channel integrated with the mixing coefficient (1−γ) are synthesized by the synthesizing units 7339 to 7341, respectively, as the audio signal of each channel. Is output.

  When the underwater characteristic correction unit 73h is configured as in the present embodiment, even if a high-intensity noise is included in an audio signal collected by a microphone of any channel, the noise can be reduced. In particular, since the water pressure is applied to the microphone in the water, noise with a high intensity may be randomly generated in the audio signal obtained by collecting the sound. However, as in this embodiment, when the audio signal with the lowest intensity is synthesized with the audio signal of each channel at a predetermined ratio (mixing coefficient γ and (1-γ)), To reduce the high-intensity noise included in the audio signal by synthesizing the audio signal of the other channel (the audio signal of the lowest intensity, that is, the audio signal of the channel expected to contain no noise of high intensity). Can do. Therefore, it is possible to reduce the influence of the sound collection characteristics of the sound in water from the sound signal collected in water. That is, it is possible to effectively approximate the audio signal intended by the user.

  Note that γ = 1 may be used. If γ = 1, the audio signals of all channels are unified to the audio signal having the lowest intensity. In other words, it is possible to obtain a monaural audio signal, and it is possible to eliminate an audio signal of a channel that is expected to have a high intensity and include noise.

  Further, when the sound collection environment determination unit 72 determines that the sound is collected in water, the mixing coefficient γ is increased, and when it is determined that the sound is not collected in water, the mixing coefficient γ is decreased. It doesn't matter. Furthermore, the mixing coefficient γ may be set to 0 when the sound collection environment determination unit 72 determines that the sound is not collected in water. When the mixing coefficient γ is set to 0, the input audio signal of each channel passes through the coefficient integrating units 7335 to 7337 and the combining units 7339 to 7341 and is output as it is. At this time, the minimum signal selection unit 7334 may not select and output the audio signal.

  Further, the value of the mixing coefficient γ may be changed based on the intensity of the input audio signal. Furthermore, a mixing coefficient may be set for each coefficient integration unit 7335 to 7337. In this case, the ratio of synthesis for each audio signal of each channel is adjusted.

  In the above-described example, the synthesis of the entire audio signal has been described. However, the synthesis may be performed only on the components of the predetermined frequency band of the audio signal. When configured in this manner, components of a predetermined frequency band of the audio signal of each channel are input to the minimum signal selection unit 7334 and the coefficient integration units 7335 to 7337, respectively. In addition, the synthesis units 7339 to 7341 output components of predetermined frequency bands of the respective audio signals. At this time, a predetermined component extraction unit that extracts a component of a predetermined frequency band may be provided before the minimum signal selection unit 7334 and the coefficient integration units 7335 to 7337. Further, a synthesis unit that synthesizes a component of a predetermined frequency band after synthesis into the audio signal of each channel may be provided in the subsequent stage of the synthesis units 7339 to 7341.

<Combination of Examples of Underwater Characteristic Correction Unit>
About the underwater characteristic correction | amendment parts 73a-73h of the Example mentioned above, it is possible to implement in combination. When combined, the effects of the respective embodiments can be obtained. As for the combination method, a combination method performed simultaneously (for example, when the first embodiment and the second embodiment are combined, the amplification unit 7303 in FIG. 13 is provided between the HPF 7302 and the synthesis unit 7304 in FIG. 12, etc.) may be used. Further, it may be combined in multiple stages (for example, when the first embodiment and the fourth embodiment are combined, the smoothing unit 7310 in FIG. 16 is connected to the output side of the combining unit 7304 in FIG. 12).

<< Application example during playback >>
The above-described embodiment relates to a sound processing device and an electronic device that determine a sound collection environment during sound collection and perform underwater characteristic correction processing on a sound signal. However, the present invention is not limited to this, and can also be applied to an electronic apparatus having a reproduction function for determining a sound collection environment during reproduction and performing underwater characteristic correction processing.

  FIG. 28 shows an imaging apparatus 1a capable of determining the sound collection environment during reproduction and performing underwater characteristic correction processing. FIG. 28 is a block diagram showing a basic configuration of an imaging apparatus according to another embodiment of the present invention, and corresponds to FIG. The same parts as those in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted.

  The imaging apparatus 1a illustrated in FIG. 28 includes a voice processing unit 7r instead of the voice processing unit 7, and a voice processing unit 7s that processes a signal input from the expansion processing unit 11 and outputs the processed signal to the voice output circuit unit 13. The configuration is the same as that of the imaging device 1 of FIG.

  The sound processing unit 7r has the same configuration as the sound processing unit 7 shown in FIG. 2 except that the sound collection environment determination unit 72 and the underwater characteristic correction unit 73 are not provided. The audio processing unit 7s has the same configuration as the audio processing unit 7 shown in FIG. 2 except that the audio processing unit 7s does not include the A / D conversion unit 71. In this example, in the sound collection environment determination unit and the underwater characteristic correction unit provided in the sound processing unit 7s, it is determined whether or not the sound signal is collected in water, and underwater based on the determination. A characteristic correction process is performed. Since the configuration of the sound collection environment determination unit and the underwater characteristic correction unit, the determination method, and the correction method are the same as those in the above-described embodiments, detailed description thereof is omitted.

  With this configuration, it is determined whether or not the sound collection environment is underwater based on the audio signal to be reproduced, and when it is determined that the sound collection environment is underwater, the underwater characteristic correction is performed on the audio signal. Can be played. Therefore, it is possible to perform the sound collection environment determination and the underwater characteristic correction process even for the audio signal recorded without performing the underwater characteristic correction process.

  Note that the sound processing unit 7r includes a sound collection environment determination unit and an underwater characteristic correction unit, and both the sound processing units 7r and 7s perform sound collection environment determination and underwater characteristic correction processing. It does not matter as a possible configuration. With this configuration, it is possible to determine the sound collection environment and perform underwater characteristic correction processing at any time during sound collection and playback.

  Further, the present example may be applied to a playback apparatus having only a playback function (for example, a playback apparatus that plays back an image signal and an audio signal recorded on an optical disk or the like). Even with such a configuration, it is possible to perform sound collection environment determination and underwater characteristic correction processing based on the audio signal.

  In addition, when the sound collection environment determination units 72b and 72c shown in FIGS. 7 and 8 are used as the sound collection environment determination unit provided in the sound processing unit 7s, the underwater characteristic correction unit shown in FIG. 14 is used as the underwater characteristic correction unit. In the case of using 73c, the sound collection environment determination units 72b and 72c and the underwater characteristic correction unit 73c are not configured to receive time information from the CPU 14, and the generation of self-generated driving sound is detected from an index or the like added to the audio signal. It doesn't matter. Furthermore, in this case, the audio processing unit 7r may acquire time information from the CPU 14 and add it to the audio signal.

  With this configuration, it is possible to perform sound collection environment determination and underwater characteristic correction processing even when a sound signal that has not been collected and recorded by the imaging device 1a is reproduced using the imaging device 1a.

<< Modification >>
Note that the audio signal processed in the sound collection environment determination unit 72 and the underwater characteristic correction unit 73 of the audio processing unit 7 shown in FIG. 2 may be a time axis signal or a frequency axis signal.

  In addition, regarding the imaging devices 1 and 1a in the embodiment of the present invention, a control device such as a microcomputer performs the operations of the sound collection environment determination unit 72 and the underwater characteristic correction unit 73 of the sound processing units 7 and 7s. It doesn't matter. Further, all or part of the functions realized by such a control device is described as a program, and the program is executed on a program execution device (for example, a computer) to realize all or part of the functions. It doesn't matter if you do.

  In addition to the case described above, the imaging devices 1 and 1a in FIGS. 1 and 28 and the audio processing unit 7 in FIG. 2 can be realized by hardware or a combination of hardware and software. When the imaging devices 1 and 1a and the sound processing unit 7 are configured using software, a block diagram of a part realized by software represents a functional block diagram of the part.

  As mentioned above, although embodiment in this invention was described, the range of this invention is not limited to this, It can add and implement various changes in the range which does not deviate from the main point of invention.

  The present invention relates to an audio processing device, an audio processing method, and an electronic apparatus equipped with an audio processing device that processes an audio signal collected in water. In particular, the present invention is preferably applied to an electronic device capable of collecting sound such as an imaging device or an IC recorder.

These are block diagrams shown about the basic composition of the imaging device in the embodiment of the present invention. These are block diagrams shown about the basic composition of the voice processing part of the imaging device in the embodiment of the present invention. These are block diagrams shown about the 1st Example of the sound collection environment determination part with which the audio | voice processing part in embodiment of this invention is equipped. These are graphs which show an example of the frequency characteristic of the audio | voice signal collected in the air. These are graphs which show an example of the frequency characteristic of the audio | voice signal collected underwater. These are figures which show the difference in the frequency characteristic of the sound in the air and underwater. These are block diagrams shown about the 2nd Example of the sound collection environment determination part with which the audio | voice processing part in embodiment of this invention is equipped. These are block diagrams shown about the 3rd Example of the sound collection environment determination part with which the audio | voice processing part in embodiment of this invention is equipped. These are block diagrams shown about the 4th Example of the sound collection environment determination part with which the audio | voice processing part in embodiment of this invention is equipped. These are schematic diagrams of a stereo microphone. These are block diagrams shown about the 5th Example of the sound collection environment determination part with which the audio | voice processing part in embodiment of this invention is equipped. These are block diagrams shown about the 1st Example of the underwater characteristic correction | amendment part with which the audio | voice processing part in embodiment of this invention is equipped. These are block diagrams shown about the 2nd Example of the underwater characteristic correction | amendment part with which the audio | voice processing part in embodiment of this invention is equipped. These are block diagrams shown about the 3rd Example of the underwater characteristic correction | amendment part with which the audio | voice processing part in embodiment of this invention is equipped. These are the graphs shown about the self-generated drive sound noise contained in the audio | voice signal collected underwater. These are block diagrams shown about the 4th Example of the underwater characteristic correction | amendment part with which the audio | voice processing part in embodiment of this invention is equipped. These are graphs showing the fray noise noise included in the audio signal collected in water and the processing of the smoothing unit. These are the graphs shown about the noise contained in the audio | voice signal collected underwater, and the process of a smooth part. These are block diagrams shown about the 5th Example of the underwater characteristic correction | amendment part with which the audio | voice processing part in embodiment of this invention is equipped. These are graphs showing a specific example of the operation of the AGC unit. These are the graphs which showed about the sudden noise contained in the audio | voice signal collected underwater, and the process of a response speed control part. These are block diagrams shown about the 6th Example of the underwater characteristic correction | amendment part with which the audio | voice processing part in embodiment of this invention is equipped. [Fig. 4] is a schematic graph showing an audio signal collected in water and an audio signal collected in air. These are block diagrams shown about another example of 6th Example of the underwater characteristic correction | amendment part with which the audio | voice processing part in embodiment of this invention is equipped. These are block diagrams shown about another example of 6th Example of the underwater characteristic correction | amendment part with which the audio | voice processing part in embodiment of this invention is equipped. These are block diagrams shown about the 7th Example of the underwater characteristic correction | amendment part with which the audio | voice processing part in embodiment of this invention is equipped. These are block diagrams shown about the 8th Example of the underwater characteristic correction | amendment part with which the audio | voice processing part in embodiment of this invention is equipped. These are block diagrams shown about the basic composition of the imaging device in another embodiment of the present invention.

Explanation of symbols

1, 1a Imaging device 2 Image sensor 3 Lens unit 4 AFE
5 Stereo microphone 6 Image processing unit 7, 7a to 7h, 7k, 7m, 7n, 7p to 7s Audio processing unit 71 A / D conversion unit 72, 72a to 72e Sound collection environment determination unit 721 Frequency characteristic determination unit 722, 726 Propagation Characteristic determination unit 723 First determination unit 724 Second determination unit 725, 729 Total determination unit 727 Rch determination unit 728 Lch determination unit 73, 73a to 73g, 73fa, 73fb Underwater characteristic correction unit 7301, 7305, 7312, 7317 LPF
7302, 7306, 7314, 7321 HPF
7303 Attenuator 7304, 7308, 7319, 7320, 7324-7326 Synthesizer 7332, 7333, 7339-7341 Synthesizer 7307 Amplifier 7309 Predetermined signal attenuator 7310 Smoother 7311 Response speed controller 7313 BPF
7315, 7318, 7322, 7323, 7330, 7331 Coefficient accumulating unit 7335-7338 Coefficient accumulating unit 7316 Restoring unit 7327 Predetermined component extracting unit 7328 Predetermined component reducing unit 7329 Alternative component supplying unit 7334 Minimum signal selecting unit 74 AGC unit 8 Compression processing unit 9 External memory 10 Driver unit 11 Decompression processing unit 12 Image output circuit unit 13 Audio output circuit unit 14 CPU
15 Memory 16 Operation section 17 TG section 18 Bus 19 Bus

Claims (9)

In an audio processing device that processes input audio signals,
A sound collection environment determination unit that determines whether the sound signal is collected in water based on the input sound signal;
An underwater characteristic correction unit that performs an underwater characteristic correction process on the input audio signal to reduce the influence of the sound collection characteristic of the underwater sound from the audio signal;
With
When the sound signal input by the sound collection environment determination unit is determined to be collected in water, the underwater characteristic correction unit performs the underwater characteristic correction processing on the input sound signal. A voice processing apparatus characterized by the above. The underwater characteristic correction unit includes an attenuation unit that attenuates an input audio signal,
The audio processing apparatus according to claim 1, wherein a process of attenuating a component having a frequency equal to or lower than the first frequency of the audio signal by the attenuation unit is included in the underwater characteristic correction process. The underwater characteristic correction unit includes an amplification unit that amplifies an input audio signal,
The audio processing apparatus according to claim 1 or 2, wherein the amplifying unit includes a process of amplifying a component having a frequency equal to or higher than a second frequency of the audio signal in the underwater characteristic correction process. The underwater characteristic correction unit includes a smoothing unit that smoothes an input audio signal,
The audio processing apparatus according to claim 1, wherein a process of smoothing an audio signal by the smoothing unit is included in the underwater characteristic correction process. An AGC unit for monitoring and adjusting the intensity of the input audio signal in a predetermined time unit is further provided.
The underwater characteristic correction unit includes a response speed control unit that controls a predetermined length of time of the AGC unit,
The voice according to any one of claims 1 to 4, wherein the underwater characteristic correction process includes a process of shortening the predetermined time length of the AGC unit by the response speed control unit. Processing equipment. The underwater characteristic correction unit includes a synthesis unit that synthesizes a predetermined signal with an input audio signal,
The audio processing apparatus according to claim 1, wherein a process of synthesizing the predetermined signal into an audio signal by the synthesizing unit is included in the underwater characteristic correction process. In an electronic device having a correction function for correcting an audio signal obtained by collecting sound,
An electronic apparatus comprising the voice processing device according to claim 1 as a voice processing unit that realizes the correction function. While further comprising a control unit for controlling the operation of the device itself,
The underwater characteristic correction unit provided in the audio processing unit operates based on control information output when the control unit operates its own device, and attenuates a component in a predetermined frequency band of an input audio signal. A predetermined signal attenuation unit;
The electronic device according to claim 7, wherein a process of attenuating the component of the predetermined frequency band by the predetermined signal attenuating unit is included in the underwater characteristic correction process. A first step for determining whether or not the audio signal is collected in water based on the input audio signal;
In the first step, when it is determined that the input audio signal is collected underwater, an underwater characteristic correction process for reducing the influence of the sound collection characteristic of the underwater sound from the audio signal is input. A second step applied to the audio signal to be performed;
An audio processing method comprising:

Images