JP2010081395A - Electronic apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an electronic apparatus in which when sound signals collected under water are played back, the sound signals can be suppressed from becoming against user intention. <P>SOLUTION: The electronic apparatus including a noise influence reduction section (processing sections 64R, 64L) which has a function for recording and/or playing back collected sound signals, in which an underwater mode and a normal mode are switched in accordance with a determination result by a sound collection environment determination section 63 and which reduces the influence of noise coming from a recorder body during sound collection. The electronic apparatus is characterized in that, when recording collected sound signals under the underwater mode or when playing back recorded sound signals under the underwater mode, the noise influence reduction section uses information from a noise determination information generation section 62 to reduce the influence of noise coming from the recorder body during sound collection. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an electronic device (for example, an imaging device or an IC recorder) capable of recording and / or reproducing an audio signal.

  In recent years, various electronic devices such as an imaging device and an IC recorder that can collect sound not only in the air but also in the water have been proposed. These electronic devices can collect sound not only in the air but also in water by adopting a waterproof structure or a structure that can be stored in a waterproof housing.

  However, since the sound collection characteristic in water is significantly different from the sound collection characteristic in air, an audio signal collected in water using an electronic device is significantly different from an audio signal collected in air. For this reason, when an audio signal collected in water is reproduced, it becomes a problem that the audio signal is contrary to the user's intention, such as being very difficult to hear or annoying.

  In response to this problem, it is determined whether or not the environment in which sound is collected is underwater, and the gain control unit and variable filter unit are controlled according to the determination result, and the gain and frequency characteristics of the sound collection signal from the microphone are determined. A video camera to be changed is proposed in Patent Document 1.

Japanese Patent Laid-Open No. 7-30790

  In the above-mentioned Patent Document 1, it is only described that the gain and frequency characteristics are optimally adjusted according to the air and the water, and what kind of adjustment should be made when reproducing the sound signal collected in the water, There was no disclosure about whether it would be possible to solve problems such as being extremely difficult to hear or becoming annoying.

  In view of the above situation, the present invention provides an electronic device capable of suppressing an audio signal that is contrary to a user's intention when an audio signal collected in water is reproduced. Objective.

  The inventors of the present invention have made extensive studies to achieve the above object. As a result, when the audio signal collected in water is reproduced, the audio signal is contrary to the user's intention, at the boundary between the air layer surrounding the microphone inside the electronic device and the water outside the electric device. Sound that travels from the water and reflects from the outside is greatly reflected due to the difference in the medium, making it difficult to detect the sound that is necessary from the outside by using a microphone. The knowledge that it was because it was collected as a loud sound was collected.

  The present invention has been made on the basis of such knowledge, and the electronic apparatus according to the present invention has a recording and / or reproduction function of a collected sound signal, has an underwater mode and a normal mode, and records at the time of sound collection. A noise effect reduction unit that reduces the effect of noise coming from the device main body, and when recording the collected audio signal in underwater mode or when reproducing the recorded audio signal in underwater mode The noise influence reducing unit is configured to reduce the influence of noise coming from the recording device main body during sound collection.

  According to such a configuration, it is possible to perform sound processing for reducing the influence of noise coming from the recording device main body at the time of sound collection on the sound signal collected underwater, and the sound signal collected underwater It is possible to suppress an audio signal that is contrary to the user's intention during reproduction. In normal mode, there is little need to reduce the effect of noise coming from the recording device during sound collection. In normal mode, it is difficult to reduce the effect of noise coming from the recording device during sound collection. Therefore, if you have a recording and playback function for the collected audio signal, either when recording the collected audio signal in the normal mode or when reproducing the recorded audio signal in the normal mode However, if the noise effect reduction unit does not reduce the effect of noise coming from the recording device main body during sound collection and has a function of recording or reproducing the collected sound signal, the collected sound signal is converted to the normal mode. When recording at the time of recording, or when reproducing the recorded audio signal in the normal mode, the noise effect reducing unit is affected by the noise coming from the recording device main body during sound collection. It is preferable to avoid reduced.

  Further, the noise influence reducing unit uses a relative phase difference and / or a relative level difference of audio signals collected by a plurality of microphones to determine noise generated by the recording device body, and A processing unit that performs audio processing for reducing the influence of noise coming from the recording device main body based on the determination result may be provided. In this case, a conversion unit that converts audio signals collected by a plurality of microphones into a signal in the frequency domain is provided, and the determination unit has a relative phase difference and / or relative frequency for each predetermined frequency band of the signal in the frequency domain. The level difference may be analyzed, and a frequency band component having a relative phase difference and / or a relative level difference equal to or greater than a preset threshold value may be determined as a noise component generated by the device body. Further, the processing unit attenuates a signal in a plurality of bands including a signal in a frequency band determined as noise by the determination unit or a signal in a frequency band determined as noise by the determination unit, and / or , Noise arriving from the recording device main body by amplifying a signal other than the frequency band determined as noise by the determination unit or a signal other than a plurality of bands including a signal in the frequency band determined as noise by the determination unit You may make it reduce the influence of this.

  Further, in each of the above configurations, a sound collection environment determination unit that determines whether or not the audio signal is collected in water, and depending on a determination result of the sound collection environment determination unit, underwater mode and normal mode And may be switched. Thereby, automatic switching between the underwater mode and the normal mode becomes possible.

  Moreover, as an example of the electronic device having each configuration described above, an imaging apparatus including a camera that captures an image can be given.

  According to the configuration of the present invention, an audio signal collected underwater can be subjected to audio processing that reduces the influence of noise coming from the recording device main body at the time of sound collection. It is possible to suppress an audio signal that is contrary to the user's intention during reproduction.

  Embodiments of the present invention will be described below with reference to the drawings. Here, as an example of an electronic apparatus according to the present invention, an imaging apparatus capable of recording / reproducing an image signal as well as recording / reproducing an audio signal will be described.

<< Imaging device >>
(Basic configuration of imaging device)
First, the basic configuration of the imaging apparatus will be described with reference to FIG. FIG. 1 is a block diagram illustrating an internal configuration example of an imaging apparatus according to the present invention.

  The image pickup apparatus shown in FIG. 1 has a solid-state image pickup device (image sensor) 1 such as a CCD (Charge Coupled Device) or CMOS (Complimentary Metal Oxide Semiconductor) sensor that converts incident light into an electric signal, and an optical image of a subject. The image sensor 1 outputs a zoom lens that forms an image on the image sensor 1, a motor that changes the focal length of the zoom lens, that is, a motor that changes the optical zoom magnification, and a motor that focuses the zoom lens on the subject. AFE (Analog Front End) 3 that converts an image signal, which is an analog signal, into a digital signal, a stereo microphone 4 that independently converts audio input from the left and right directions in front of the imaging device into an electrical signal, An image processing unit 5 that performs various image processing such as gradation correction on an image signal that is a digital signal; The audio signal that is an analog signal from the audio signal 4 is converted into a digital signal and subjected to audio correction processing, the image signal output from the image processor 5 and the audio output from the audio processor 6 A compression processing unit 7 that performs compression encoding processing such as an MPEG (Moving Picture Experts Group) compression method on each of the signals, and an external memory such as an SD card that stores the compression encoded signal compressed by the compression processing unit 7 22, the driver unit 8 that records in the memory 22, the decompression processing unit 9 that decompresses and decodes the compression-coded signal read from the external memory 22 by the driver unit 8, and the image signal obtained by decoding by the decompression processing unit 9 is analog Based on a video output circuit unit 10 for converting to a signal, a video output terminal 11 for outputting a signal converted by the video output circuit unit 10, and a signal from the video output circuit unit 10 A display unit 12 having an LCD (Liquid Crystal Display) or the like for displaying an image, an audio output circuit unit 13 for converting an audio signal from the decompression processing unit 9 into an analog signal, and a signal converted by the audio output circuit unit 13 A sound output terminal 14 that outputs the sound, a speaker unit 15 that reproduces and outputs sound based on the sound signal from the sound output circuit unit 13, and a timing generator that outputs a timing control signal for matching the operation timing of each block ( TG) 16, a CPU (Central Processing Unit) 17 that controls the drive operation of the entire imaging apparatus, a memory 18 that stores each program for each operation and temporarily stores data at the time of program execution, and a user A bus for exchanging data between the operation unit 19 to which an instruction from the CPU 17 is input and the CPU 17 and each block It comprises a line 20, a bus line 21 for exchanging data between the memory 18 and each block. The CPU 17 controls the focus and the diaphragm by driving each motor in the lens unit 2 in accordance with the image signal detected by the image processing unit 5.

(Basic operation of the imaging device)
Next, the basic operation of the image pickup apparatus shown in FIG. 1 during moving image shooting will be described with reference to FIG. First, the imaging apparatus acquires an image signal that is an electrical signal by photoelectrically converting light incident from the lens unit 2 in the image sensor 1. Then, in synchronization with the timing control signal input from the timing generator 16, the image sensor 1 sequentially outputs image signals to the AFE 3 at a predetermined frame period (for example, 1/60 seconds).

  Then, the image signal converted from the analog signal to the digital signal by the AFE 3 is input to the image processing unit 5. The image processing unit 5 converts the input image signal into an image signal composed of a luminance signal and a color difference signal, and performs various image processing such as gradation correction and contour enhancement. The memory 18 operates as a frame memory, and temporarily holds an image signal when the image processing unit 5 performs processing.

  At this time, based on the image signal input to the image processing unit 5, the lens unit 2 adjusts the position of various lenses to adjust the focus, or adjusts the opening of the diaphragm to adjust the exposure. It is done. This adjustment of focus and exposure is automatically performed based on a predetermined program so as to be in an optimum state, or manually performed based on a user instruction.

  On the other hand, an audio signal converted into an electrical signal in the stereo microphone 4 is input to the audio processing unit 6. The sound processing unit 6 converts an input sound signal into a digital signal and performs sound correction processing such as noise removal and sound signal intensity control. The configuration of the audio processing unit 6 will be described later.

  Then, both the image signal output from the image processing unit 5 and the audio signal output from the audio processing unit 6 are input to the compression processing unit 7 and compressed by the compression processing unit 7 using a predetermined compression method. At this time, the image signal and the audio signal are associated with each other in time, and the image and the sound are not shifted during reproduction. The compressed image signal and audio signal are recorded in the external memory 22 via the driver unit 8.

  If only audio is recorded, the audio signal is compressed by the compression processing unit 7 using a predetermined compression method and recorded in the external memory 22.

  The compression-coded signal recorded in the external memory 22 is read out to the decompression processing unit 9 in accordance with the output signal of the operation unit 19 based on a user instruction. The decompression processing unit 9 decompresses and decodes the compressed encoded signal, and generates an image signal and an audio signal. The image signal is output to the video output circuit unit 10 and the audio signal is output to the audio output circuit unit 13. Then, in the video output circuit unit 10 and the audio output circuit unit 13, the image signal and the audio signal are converted into a format reproducible on the display unit 12 and the speaker unit 15 and output.

  Further, in a so-called preview mode in which the user confirms an image displayed on the display unit 12 without recording an image signal, the compression processing unit 7 is prevented from performing compression processing, and the image processing unit 5 The image signal may be output not to the compression processing unit 7 but to the video output circuit unit 10. Further, when recording an image signal, the image signal may be output to the display unit 12 via the video output circuit 10 in parallel with the operation of recording in the external memory 22 via the driver unit 8. .

  In the configuration shown in FIG. 1, the display unit 12 and the speaker unit 15 are mounted on the imaging device. However, the display unit 12 and the speaker unit 15 are separated from the imaging device, and terminals (video output) provided in the imaging device. The terminal 11 and the audio output terminal 14) may be connected using a cable or the like.

(Stereo microphone placement)
Next, an arrangement example of the stereo microphone 4 included in the imaging apparatus illustrated in FIG. 1 will be described with reference to the drawings. FIG. 2 is a schematic external view of the image pickup apparatus shown in FIG. The monitor unit 23 is provided with a display unit 12 and a right microphone 4R and a left microphone 4L constituting the stereo microphone 4. The right microphone 4R and the left microphone 4L are provided on the back surface of the display unit 12 with a microphone interval of 2 cm.

(Basic configuration of the audio processing unit)
Next, the basic configuration of the audio processing unit 6 included in the imaging apparatus shown in FIG. 1 will be described with reference to FIG. FIG. 3 is a block diagram showing a basic configuration of the audio processing unit 6.

  As shown in FIG. 3, the audio processing unit 6 includes FFT (Fast Fourier Transform) units 61R and 61L, a noise determination information generation unit 62, a sound collection environment determination unit 63, processing units 64R and 64L, and IFFT ( Inverse Fast Fourier Transform) sections 65R and 65L.

  The FFT unit 61R samples the Rch audio signal input from the right microphone 4R (see FIG. 2) of the stereo microphone 4 at 48 kHz and converts it to a digital signal, and then performs frequency domain signal SR by FFT processing every 2048 samples. Convert to [F]. Further, the FFT unit 61L samples the Lch audio signal input from the left microphone 4L (see FIG. 2) of the stereo microphone 4 at 48 kHz and converts it to a digital signal, and then performs FFT processing for each 2048 sample in the frequency domain. Conversion to signal SL [F].

  The noise determination information generation unit 62 is noise generated by the imaging apparatus main body using the frequency domain signal SR [F] output from the FFT unit 61R and the frequency domain signal SL [F] output from the FFT unit 61L. Information necessary to determine whether or not there is generated. The sound collection environment determination unit 63 determines whether or not the sound signal input to the sound processing unit 6 is collected in water.

  When the sound collection environment determination unit 63 determines that the two sound signals input to the sound processing unit 6 are collected underwater, the processing unit 64R is information output from the noise determination information generation unit 62. Is used for the frequency domain signal SR [F] to reduce the influence of noise arriving from the imaging apparatus main body during sound collection, and the processing unit 64L is output from the noise determination information generation unit 62. Is used for the frequency domain signal SL [F] to reduce the influence of noise coming from the imaging apparatus body during sound collection. On the other hand, when the sound collection environment determination unit 63 determines that the two sound signals input to the sound processing unit 6 are not collected in water, the processing units 64R and 64L arrive from the imaging apparatus main body during sound collection. Audio processing that reduces the effect of noise is not performed.

  With such a configuration, it is possible to perform audio processing to reduce the effect of noise coming from the imaging device main body during sound collection on sound signals collected in water, and reproduce sound signals collected in water When it does, it can suppress that it becomes an audio | voice signal contrary to a user's intention.

  In addition, the determination result of the sound collection environment determination unit 63 is displayed on the display unit 12, and a display LED (Light Emitting Diode) is lit and blinks in accordance with the determination result of the sound collection environment determination unit 63. The determination result of the sound environment determination unit 63 may be notified to the user. According to such a configuration, the user can easily confirm whether or not the sound collection environment determination unit 63 makes a correct determination.

  Further, the user can operate the operation unit 19 to select a normal shooting mode that is a shooting mode suitable when the imaging device is in the air and an underwater shooting mode that is a shooting mode suitable when the imaging device is in water. A configuration that can be manually switched may be used. Also in this case, the sound collection environment determination unit 63 may determine whether or not the sound signal input to the sound processing unit 6 is collected in water. Either the shooting mode automatically set based on the determination result 63 or the shooting mode manually set by operating the operation unit 19 may be prioritized. In addition, it is desirable that it is possible to change by the operation of the operation unit 19 whether to give priority to the automatically set shooting mode or the manually set shooting mode.

  Further, the audio processing unit 6 may include a processing unit that performs audio correction processing in addition to the processing units 64R and 64L. In addition, the processing units other than the processing units 64R and 64L may perform a sound correction process on an audio signal determined not to be collected underwater, or all input audio You may perform an audio | voice correction process with respect to a signal.

(Examples of noise determination information generation unit, sound collection environment determination unit, and processing unit)
Examples of the noise determination information generation unit 62, the sound collection environment determination unit 63, and the processing units 64R and 64L described above will be described in order with reference to the drawings. In the following description, the same parts as those in the basic configuration described above are denoted by the same reference numerals, and detailed description thereof is omitted.

<First Example of Noise Determination Information Generation Unit>
First, a first example of the noise determination information generation unit 62 will be described with reference to FIGS. 4 and 5. FIG. 4 is a block diagram showing the configuration of the audio processing device 6 when the first embodiment of the noise determination information generating unit 62 is adopted. In the first embodiment of the noise determination information generation unit 62, the noise determination information generation unit 62 includes a relative phase difference information generation unit 621. FIG. 5 is a diagram illustrating a state of sound propagation from the noise source of the imaging apparatus main body and the sound source that is the original sound collection target.

  In order to uniquely determine the relative phase difference between the two audio signals collected by the two microphones, the interval between the two microphones needs to be an audio signal having a frequency equal to or lower than a half wavelength. Therefore, when the distance between the two microphones is 2 cm as shown in FIG. 5, the relative phase difference information generation unit 621 is configured to process the audio signal in the band of 8.5 kHz or less if the speed of sound in the air is 340 m / s. Only relative phase difference information can be generated.

Noise such as motor sound emitted from the imaging apparatus main body propagates through a cavity (in the air) in the casing of the imaging apparatus and reaches each of the microphones 4R and 4L. At this time, the relative phase difference Δφ0 that is the difference between the phase of the noise reaching the right microphone 4R and the phase of the noise reaching the left microphone 4L can be expressed by the following equation (1). Here, Freq is the frequency of the noise for which the relative phase difference is obtained.
Δφ0 = 2π × (Freq × 20/340000) (1)

On the other hand, the difference (relative phase difference) between the phase of the sound that has propagated through the water and reached the right microphone 4R and the phase of the sound that has reached the left microphone 4L is as follows. When it comes from the side of the imaging device, it becomes the largest, and the relative phase difference Δφ1 in this case is expressed by the following equation (2) because the sound speed in water is about five times the sound speed in air. it can. Where Freq is the frequency of the sound whose relative phase difference is being calculated. The sound propagated in the water then propagates in the air until it enters the monitor unit 23 and reaches the microphones 4R and 4L, but it enters the monitor unit 23 until it reaches the microphones 4R and 4L. The propagation path length is almost the same for the two paths, and the propagation path length in the monitor unit 23 (in the air) is extremely short compared to the propagation path length in water. In considering the relative phase difference, the propagation path in the monitor unit 23 (in the air) may be ignored. In addition, as shown in FIG. 5A, the sound source that is the original sound collection target may be in the air, but the propagation path length from the sound source to the air-water boundary surface is almost equal to the two routes. Since they are the same, the propagation path from the sound source to the air-water interface may be ignored when considering the relative phase difference of the sound propagating in the water.
Δφ1 = 2π × {Freq × 20 / (340000 × 5)} (2)

  The relative phase difference information generation unit 621 compares the phase difference between the frequency domain signal SR [F] and the frequency domain signal SL [F], and the phase of the sound reaching the right microphone 4R and the left microphone 4L. Relative phase difference information that is a difference from the phase of the sound is generated. Note that the relative phase difference comparison unit 621 obtains a relative phase difference for each 2048/48000 [Hz], which is the resolution of the FFT units 61R and 61L.

  Since the relative phase difference of the sound propagating through the water is Δφ1 or less and the relative phase difference of the noise generated by the imaging apparatus main body is Δφ0 (= 5 × Δφ1), the relative phase difference information generation unit 621 has obtained. A frequency component having a relative phase difference of Δφ1 or less can be determined to be a frequency component of sound transmitted through water.

<Second Example of Noise Determination Information Generation Unit>
Next, a second embodiment of the noise determination information generation unit 62 will be described with reference to FIG. FIG. 6 is a block diagram showing a configuration of the audio processing device 6 when the second embodiment of the noise determination information generation unit 62 is adopted. In the second embodiment of the noise determination information generation unit 62, the noise determination information generation unit 62 includes a relative level difference information generation unit 622.

  It is known that sound attenuation is very small underwater. In general, it is known that the closer the sound source is, the greater the distance attenuation of the sound. Therefore, the attenuation with respect to the sound from the outside which propagates in water and reaches each of the microphones 4R and 4L is small, and the signal level difference between the right microphone 4R and the left microphone 4L hardly occurs. On the other hand, noise that propagates through the cavity (in the air) in the housing of the imaging device and reaches each of the microphones 4R and 4L propagates in the air, and the noise source and the microphones 4R and 4L are in a short distance. Since there is attenuation due to sound absorption during reflection inside the housing, the signal level difference between the right microphone 4R and the left microphone 4L increases.

  The relative level difference information generation unit 622 compares the level difference between the frequency domain signal SR [F] and the frequency domain signal SL [F] to reach the right microphone 4R and the left microphone 4L. Relative level difference information that is a difference from the sound level is generated. The relative level difference information generation unit 622 obtains a relative level difference for each of 2048/48000 [Hz], which is the resolution of the FFT units 61R and 61L.

  Since the relative level difference of the sound propagating through the water is large and the relative level difference of the noise generated by the imaging apparatus main body is small, the relative level difference obtained by the relative level difference information generation unit 622 is equal to or greater than a preset threshold value. It can be determined that the frequency component is the frequency component of the sound transmitted through the water.

  A combination of the first embodiment and the second embodiment of the noise determination information generation unit 62 is also possible. That is, the noise determination information generation unit 62 can also generate information on the relative phase difference and the relative level difference. By performing determination using both the relative phase difference and the relative level difference, the determination accuracy can be improved.

<First Example of Sound Collection Environment Determination Unit>
Next, a first embodiment of the sound collection environment determination unit 63 will be described with reference to FIG. FIG. 7 is a block diagram showing the configuration of the sound processing device 6 when the first embodiment of the sound collection environment determination unit 63 is adopted. In the first embodiment of the sound collection environment determination unit 63, the sound collection environment determination unit 63 includes a pressure determination unit 631.

  When the first embodiment of the sound collection environment determination unit 63 is adopted, a pressure sensor is newly provided in the imaging apparatus shown in FIG. The pressure determination unit 631 receives the detection signal of the pressure sensor, and when the pressure outside the imaging device is equal to or higher than a preset threshold, the imaging device is used in water and the sound input to the audio processing unit 6 When it is determined that the signal is collected in water and the pressure outside the imaging device is less than a preset threshold, the imaging device is used in the air and is input to the audio processing unit 6. It is determined that the audio signal is collected in the air.

<Second Example of Sound Collection Environment Determination Unit>
Next, a second embodiment of the sound collection environment determination unit 63 will be described with reference to FIG. FIG. 8 is a block diagram showing the configuration of the sound processing device 6 when the second embodiment of the sound collection environment determination unit 63 is adopted. In the second embodiment of the sound collection environment determination unit 63, the sound collection environment determination unit 63 includes a frequency characteristic determination unit 632.

  Here, frequency characteristics when white noise is reproduced in the air and collected in the air are shown in FIG. FIG. 10 shows frequency characteristics when white noise is reproduced in the air and collected in water.

  When the sound is collected in the air, the frequency characteristic is substantially flat as shown in FIG. On the other hand, as for the frequency characteristics when sound is collected in water, generally, if the signal level is large, the signal in the high frequency band is greatly attenuated as shown in FIG. This is because the propagated sound is attenuated by reflection at the two boundaries in the air-underwater, underwater-inside the housing of the sound collecting device (in the air), and the sound of the waves newly generated in the water and the case This is because generally low sounds such as newly generated sounds remain inside.

  In this way, when the imaging device is used underwater, low-band sounds, medium-band sounds, and high-band sounds that cannot occur when the imaging device is used in air, Therefore, the determination is performed using the level difference.

  Hereinafter, a determination method executed by the frequency characteristic determination unit 632 will be described. Targeting Rch audio signals and Lch audio signals, signals in each of a low band (for example, several tens (70) Hz to 3 kHz), a middle band (for example, 6 kHz to 9 kHz), and a high band (for example, 12 kHz to 15 kHz) Calculate the average level. In addition, the specific numerical value of each band is not limited to the above example, and there is no problem if the magnitude relationship between the bands is correct. Further, the low band and the middle band may partially overlap, and the middle band and the high band may partially overlap.

  The signal level ratio of the low band to the high band (low band / high band) R1 and the signal level ratio of the low band to the medium band (low band / medium band) that can be calculated from the average value of the signal level in each band ) R2 and the signal level ratio (medium band / high band) R3 of the medium band to the high band change with time as shown in FIG. 11 when the stereo microphone 4 is inserted from the air into the water and returned to the air again. Indicates. Periods T1 and T3 in FIG. 11 are periods in which the stereo microphone 4 is located in the air, and periods T2 in FIG. 11 are periods in which the stereo microphone 4 is located in water. The signal level ratio (medium band / high band) R3 of the medium band to the high band is a substantially constant value regardless of whether in the air or underwater. On the other hand, the low-band signal level ratio (low band / high band) R1 to the high band and the low-band signal level ratio (low band / middle band) R2 to the medium band are small values in the air. The sensitivity of sound reception changes in water, which is a much larger value than in air.

  By utilizing this, the frequency characteristic determination unit 632 obtains a low-band signal level ratio (low band / high band) R1 for the high band and a low-band signal level ratio (low band / middle band) R2 for the medium band. Calculated from the average value of the signal level in each band, the low band signal level ratio (low band / high band) R1 to the high band and the low band signal level ratio (low band / middle band) R2 to the medium band are preset. When it becomes larger than the threshold value, it is determined that the audio signal input to the audio processing unit 6 is collected in water. Although the determination accuracy is inferior, the average value of the signal level in the medium band and the signal level ratio of the low band to the medium band (low band / medium band) R2 are not calculated, and the signal level ratio of the low band to the high band ( When the low band / high band (R1) is greater than or equal to a preset threshold value, it is determined that the audio signal input to the audio processing unit 6 is collected in water, or the signal in the high band The average value of the level and the signal level ratio of the low band to the high band (low band / high band) R1 are not calculated, and the signal level ratio of the low band to the medium band (low band / medium band) R2 is equal to or greater than a preset threshold value. It is also possible to determine that the audio signal input to the audio processing unit 6 is collected in water when the signal becomes larger.

  Even underwater, sudden noise occurs due to the sound of bubbles and the rubbing sound of the housing, and the signal level of the middle band and the high band increases instantaneously, and the signal level ratio of the low band to the high band (low There is a possibility that the signal level ratio (low band / medium band) R2 of the low band with respect to the band (high band) R1 and the medium band instantaneously becomes a small value. Therefore, the low-band signal level ratio (low band / high band) R1 to the high band and the low-band signal level ratio (low band / middle band) R2 to the high band used for the determination by the frequency characteristic determination unit 632 are constant. It is desirable to use an averaged value over time.

  Further, regarding the threshold value, it is desirable to provide a hysteresis characteristic and set the threshold value high while determining that it is in the air, and setting the threshold value low while determining that it is underwater.

<First Example of Processing Unit>
Next, a first embodiment of the processing units 64R and 64L will be described with reference to FIG. FIG. 12 is a block diagram showing the configuration of the audio processing device 6 when the first embodiment of the processing units 64R and 64L is employed. In the first embodiment of the processing units 64R and 64L, the processing unit 64R includes a reduction processing unit 641R, and the processing unit 64L includes a reduction processing unit 641L.

  When the sound collection environment determination unit 63 determines that the sound signal input to the sound processing unit 6 is collected in water, the reduction processing units 641R and 641L perform noise determination from the noise determination information generation unit 62. The signals SR [F] and SL [F] in the frequency domain are imaged by comparing the information with a threshold value (for example, Δφ1 in the above-described equation (2) when the first embodiment of the noise determination information generation unit 62 is adopted). It is determined for each 2048/48000 [Hz], which is the resolution of the FFT units 61R and 61L, whether or not the noise is generated by the apparatus main body, and the frequency component determined to be noise generated by the imaging apparatus main body is reduced by -20 dB. Then, processing that does not reduce the frequency component that has not been determined to be noise generated by the imaging apparatus main body is performed.

  When the first embodiment of the processing units 64R and 64L is adopted, if the first embodiment of the noise determination information generation unit 62 is adopted, the frequency having a large phase difference between the signals SR [F] and SL [F] in the frequency domain. Since the reduction process is performed only on the components, even if the sound collection environment determination unit 63 makes an erroneous determination, the adverse effect of the sound collection environment erroneous determination is small without reducing the sound in the front direction of shooting. There is an advantage.

<Second Embodiment of Processing Unit>
Next, a second embodiment of the processing units 64R and 64L will be described with reference to FIG. FIG. 13 is a block diagram showing the configuration of the audio processing device 6 when the second embodiment of the processing units 64R and 64L is adopted. In the second embodiment of the processing units 64R and 64L, the processing unit 64R includes an enhancement processing unit 642R, and the processing unit 64L includes an enhancement processing unit 642L.

  When the sound collection environment determination unit 63 determines that the sound signal input to the sound processing unit 6 is collected underwater, the enhancement processing units 642R and 642L perform noise determination from the noise determination information generation unit 62. The signals SR [F] and SL [F] in the frequency domain are imaged by comparing the information with a threshold value (for example, Δφ1 in the above-described equation (2) when the first embodiment of the noise determination information generation unit 62 is adopted). It is determined for each 2048/48000 [Hz], which is the resolution of the FFT units 61R and 61L, whether or not the noise is generated by the apparatus main body, and frequency components that are not determined to be noise generated by the imaging apparatus main body are emphasized ( The frequency component determined to be noise generated by the imaging apparatus body is not emphasized. Note that the degree of emphasis may be uniform regardless of the frequency, and varies depending on the frequency (in consideration of the frequency characteristics in FIG. May be.

  The frequency components other than the frequency components determined by the processing units 64R and 64L as noise are the frequency components of the original underwater sound that has propagated through the water. The original underwater sound that has propagated in the water is reflected at the boundary between water and air, and the sound is greatly attenuated. Therefore, by adopting the second embodiment of the processing units 64R and 64L and emphasizing (amplifying) components other than the frequency components determined to be noise, the original underwater sound can be brought close to a desired level.

  A combination of the first embodiment and the second embodiment of the processing unit 64 is also possible. In other words, it is possible to reduce the frequency component determined to be noise generated by the imaging device main body and to perform processing to emphasize (amplify) the frequency component not determined to be noise generated by the imaging device main body. It is.

<Third Example of Processing Unit>
Next, a third embodiment of the processing units 64R and 64L will be described with reference to FIG. FIG. 14 is a block diagram showing the configuration of the audio processing device 6 when the third embodiment of the processing units 64R and 64L is adopted. In the third embodiment of the processing units 64R and 64L, the processing unit 64R includes a threshold variable processing unit 643R, and the processing unit 64L includes a threshold variable processing unit 643L. In addition, when employ | adopting 3rd Example of the process parts 64R and 64L, as shown in FIG. 14, 1st Example of the noise determination information generation part 62 is employ | adopted.

  When the sound collection environment determination unit 63 determines that the sound signal input to the sound processing unit 6 is collected in water, the threshold variable processing units 643R and 643L are configured to receive noise from the noise determination information generation unit 62. By comparing the determination information with the first threshold value (Δφ1 in the above-described equation (2)), it is determined whether the frequency domain signals SR [F] and SL [F] are noises generated by the imaging apparatus main body. It is determined every 2048/48000 [Hz] that is the resolution of the units 61R and 61L, the frequency component determined to be noise generated by the imaging device main body is reduced by -20 dB, and is determined to be noise generated by the imaging device main body. Processing that does not reduce the frequency component that has not been performed is performed.

Further, when the sound collection environment determination unit 63 determines that the sound signal input to the sound processing unit 6 has been collected in the air, the threshold variable processing units 643R and 643L have the FFT units 61R and 61L. For each resolution of 2048/48000 [Hz], the relative phase difference information from the relative phase difference information generation unit 621 is compared with the second threshold value (Δφ2 in the expression (3) shown below) to determine the relative phase difference. If the relative phase difference information from the information generation unit 621 is larger than the second threshold value (Δφ2 in the expression (3) shown below), the signals SR [F] and SL [F] in the frequency domain are the sound source direction angles θR or θL. It is determined that the sound source (see FIG. 15) is due to a sound source (unnecessary sound) greater than 30 °, and the frequency component determined to be due to a sound source (unnecessary sound) whose sound source direction angle θR or θL is greater than 30 ° is determined. -20dB lower, relative If the relative phase difference information from the phase difference information generation unit 621 is equal to or less than the second threshold value (Δφ2 in the following equation (3)), the frequency domain signals SR [F] and SL [F] are the sound source direction angle θR. Alternatively, it is determined that θL (see FIG. 15) is due to a sound source within 30 °, and processing that does not reduce the frequency component determined that the sound source direction angle θR or θL is due to a sound source within 30 ° is performed. Thereby, directivity can be given to directions within 30 ° on the left and right sides from the front direction of the imaging apparatus. However, Freq in the equation (3) is a frequency to be compared with the relative phase difference information.
Δφ2 = 2π × (Freq × 20 × sin 30 ° / 340000) (3)

  When the third embodiment of the processing unit 64 is adopted, switching between the noise reduction processing in the underwater shooting mode and the directivity processing in the normal shooting mode can be performed without increasing the circuit scale or the software scale. This can be realized only by switching the second threshold).

<< Application example during playback >>
The above-described imaging apparatus shown in FIG. 1 executes processing for reducing the influence of noise coming from the imaging apparatus main body during sound collection when recording the collected audio signal. However, the present invention is not limited to this, and when reproducing the collected audio signal, a process for reducing the influence of noise coming from the imaging apparatus main body during sound collection may be executed.

  FIG. 16 shows an imaging apparatus that executes processing for reducing the influence of noise coming from the imaging apparatus main body during sound collection when reproducing the collected audio signal. In FIG. 16, parts that are substantially the same as those in FIG.

  The imaging apparatus shown in FIG. 16 differs from the imaging apparatus shown in FIG. 1 in that an audio processing unit 6a is provided instead of the audio processing unit 6, and audio processing is performed between the expansion processing unit 9 and the audio output circuit unit 13. The point is that a portion 6b is provided.

  Unlike the audio processing unit 6, the audio processing unit 6 a performs A / D conversion on an audio signal that is an analog signal from the stereo microphone 4, but performs FFT processing, noise determination information generation processing, sound collection environment determination processing, This is a configuration that does not perform processing for reducing the influence of noise coming from the imaging apparatus main body during sound collection and IFFT processing.

  The audio processing unit 6b has the same configuration as the audio processing unit 6 except that the FFT unit does not perform A / D conversion processing. Since the audio processing performed in the audio processing unit 6b is basically the same as the audio processing performed in the audio processing unit 6, the description thereof is omitted here.

  When the sound collection environment determination unit of the sound processing unit 6b determines that the sound is collected in water, the sound processing unit 6b enters the underwater reproduction mode, which is a reproduction mode suitable for reproduction of the sound signal collected in water. When it is determined that the sound is collected in the air by the sound collection environment determining unit, the normal playback mode is selected as a playback mode suitable for playing back the sound signal collected in the air. In addition to this, a configuration may be adopted in which the user can manually switch between the underwater reproduction mode and the normal reproduction mode by operating the operation unit 19. When the manual switching configuration is added as in the latter case, either the automatically set playback mode or the manually set playback mode by operating the operation unit 19 may be prioritized. Note that it is preferable that the operation unit 19 can be changed to give priority to the automatically set reproduction mode or the manually set reproduction mode.

  Note that there is a possibility that the sound collection environment (the environment in which the imaging device shown in FIG. 16 is placed at the time of sound collection) and the reproduction environment (the environment in which the imaging device shown in FIG. 16 is placed at the time of reproduction) are different. For this reason, the sound collection environment determination unit of the sound processing unit 6b is not adapted to the first embodiment of the sound collection environment determination unit described above.

<< Modification >>
In the imaging apparatus shown in FIG. 1 or FIG. 16 described above, when it is determined that the sound is collected in water, the sound collection is performed on the collected sound signal (including the recorded and reproduced sound signal). Audio processing that sometimes reduces the effects of noise coming from the device body was performed, but in addition, smoothing processing, suppression of gain during zoom processing, use of audio data or pre-frame data stored in advance in the memory 18 In addition, at least one of suppression of click noise by increasing the response speed of auto gain control may be performed.

  In the imaging apparatus shown in FIG. 1 or FIG. 16 described above, when it is determined that the sound is collected in water, the collected sound signal (including the recorded and reproduced sound signal) is collected. In addition to performing audio processing to reduce the effects of noise coming from the recording device during sound collection, the video signal that was shot when it was determined that the sound was collected in water (recorded and played video signals were recorded) Characteristic correction suitable for the underwater environment and characteristic correction suitable for the underwater environment may be performed for camera control during shooting.

  Moreover, in the imaging device shown in FIG. 1 or FIG. 16 described above, the stereo microphone 4 is used, but another microphone (for example, a 5.1ch surround recording-compatible microphone) including a plurality of microphones may be used. .

  In addition, since the electronic apparatus according to the present invention only needs to be able to record and / or reproduce an audio signal, a block related to an image is not particularly necessary. Therefore, the present invention can also be applied to electronic devices other than the imaging device, for example, an audio recording device, an audio reproducing device, an audio recording / reproducing device (for example, an IC recorder), and the like.

  In addition, the electronic device equipped with the sound collection environment determination device according to the present invention preferably has a waterproof structure, but even if it is not waterproof, for example, an audio signal stored in a waterproof housing and collected by a waterproof external microphone It is possible to adopt a usage such as inputting.

  For example, the electronic device according to the present invention can be realized by operating reproduction software including a program for causing a computer to function as each unit of the audio processing unit 6b on a personal computer.

  In the first embodiment of the processing unit, the frequency component determined to be noise generated by the imaging apparatus main body is reduced by -20 dB, and the frequency component not determined to be noise generated by the imaging apparatus main body is not reduced. However, instead of this processing, as shown in FIG. 17, the frequency component determined to be noise generated by the imaging apparatus main body is greatly reduced (−12 dB in the example shown in FIG. 17), and the imaging apparatus main body emits. A process may be performed in which the frequency components before and after the frequency component determined to be noise are slightly reduced (−6 dB in the example illustrated in FIG. 17) and other frequency components are not reduced. As a result, it is possible to suppress the occurrence of unnatural sound or musical noise due to a large difference in attenuation amount between consecutive frequencies.

  Similarly, in the second embodiment of the processing unit, frequency components that are not determined to be noise generated by the imaging apparatus main body are emphasized (amplified), and frequency components that are determined to be noise generated by the imaging apparatus main body are emphasized. However, instead of this processing, the frequency component that was not determined to be noise generated by the imaging device main body was greatly emphasized (amplified), and the frequency that was not determined to be noise generated by the imaging device main body Processing may be performed in which the frequency components before and after the component are slightly emphasized (amplified) and the other frequency components are not emphasized (amplified).

These are block diagrams which show the example of 1 internal structure of the imaging device which concerns on this invention. FIG. 2 is a schematic external view of the image pickup apparatus shown in FIG. 1 viewed from the upper surface of the apparatus. These are block diagrams which show the basic composition of the audio | voice processing part with which the imaging device shown in FIG. 1 is provided. These are block diagrams which show the structure of the audio | voice processing apparatus at the time of employ | adopting 1st Example of a noise determination information generation part. These are figures which show the mode of the propagation of the sound from the noise source of an imaging device main body, and the sound source which is the original sound collection object. These are block diagrams which show the structure of the audio | voice processing apparatus at the time of employ | adopting 2nd Example of a noise determination information generation part. These are block diagrams which show the structure of the audio | voice processing apparatus at the time of employ | adopting 1st Example of a sound collection environment determination part. These are block diagrams which show the structure of the audio processing apparatus at the time of employ | adopting 2nd Example of the sound collection environment determination part. These are figures which show the frequency characteristic of the sound in the air. FIG. 4 is a diagram illustrating frequency characteristics of sound in water. These are figures which show the difference in the frequency characteristic of the sound in the air and underwater. These are block diagrams which show the structure of the audio | voice processing apparatus at the time of employ | adopting 1st Example of a process part. These are block diagrams which show the structure of the audio | voice processing apparatus at the time of employ | adopting 2nd Example of a process part. These are block diagrams which show the structure of the audio | voice processing apparatus at the time of employ | adopting 3rd Example of a process part. These are figures which show a sound source direction angle. These are the block diagrams which show the other internal structural example of the imaging device which concerns on this invention. FIG. 10 is a diagram illustrating an example of attenuation processing of a processing unit.

Explanation of symbols

1 Solid-state image sensor (image sensor)
2 Lens part 3 AFE
4 stereo microphone 4L, 4R microphone 5 image processing unit 6, 6a, 6b audio processing unit 7 compression processing unit 8 driver unit 9 expansion processing unit 10 video output circuit unit 11 video output terminal 12 display unit 13 audio output circuit unit 14 audio output Terminal 15 Speaker unit 16 Timing generator (TG)
17 CPU
18 memory 19 operation unit 20, 21 bus line 22 external memory 23 monitor unit 61R, 61L FFT unit 62 noise determination information generation unit 63 sound collection environment determination unit 64R, 64L processing unit 65R, 65L IFFT unit 621 relative phase difference information generation unit 622 Relative level difference information generation unit 631 Pressure determination unit 632 Frequency characteristic determination unit 641R, 641L Reduction processing unit 642R, 642L Enhancement processing unit 643R, 643L Threshold variable processing unit

Claims (6)

It has a recording and / or playback function for collected audio signals
It has an underwater mode and a normal mode,
Equipped with a noise effect reduction unit that reduces the effect of noise coming from the recording device body during sound collection,
When the collected audio signal is recorded in the underwater mode, or when the recorded audio signal is played back in the underwater mode, the noise effect reducing unit receives noise from the recording device main body at the time of collecting the sound. Electronic devices characterized by reducing the influence of The noise influence reducing unit is
A determination unit that determines noise generated by the recording device main body using a relative phase difference and / or a relative level difference of audio signals collected by a plurality of microphones;
The electronic device according to claim 1, further comprising: a processing unit that performs audio processing that reduces an influence of noise coming from the recording device main body based on a determination result of the determination unit. A conversion unit that converts audio signals collected by a plurality of microphones into signals in the frequency domain,
The determination unit analyzes a relative phase difference and / or a relative level difference for each predetermined frequency band of the signal in the frequency domain, and has a relative phase difference and / or a relative level difference equal to or greater than a preset threshold. The electronic device according to claim 2, wherein the component is determined to be a noise component emitted by the device main body.   The processing unit attenuates a signal in a plurality of bands including a signal in a frequency band determined as noise by the determination unit or a signal in a frequency band determined as noise by the determination unit, and / or Influence of noise coming from the recording device main body by amplifying signals other than the frequency band determined as noise by the determination unit or signals other than a plurality of bands including the signal in the frequency band determined as noise by the determination unit The electronic device according to claim 3, wherein the electronic device is reduced.   A sound collection environment determination unit that determines whether or not an audio signal is collected in water, and switches between an underwater mode and a normal mode according to a determination result of the sound collection environment determination unit. 5. The electronic device according to any one of 4 above.   The electronic apparatus according to claim 1, wherein the electronic apparatus includes a camera that captures a video.