JP2010134260A - Electronic apparatus and voice processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an electronic apparatus reducing a sense of incongruity in reproduction of voice collected underwater. <P>SOLUTION: The electronic apparatus includes a voice processing part 7 having a recording and/or reproducing function of gathered voice signals, and in recording voice signals gathered underwater and/or reproducing the voice signals, the voice signals are processed according to acoustic sense characteristics of a human in the water. The voice processing part 7 includes a correction processing part 72 correcting frequency characteristics of the voice signals in accordance with human acoustic sense characteristics in the water. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an electronic device (for example, an imaging apparatus or an IC recorder) capable of recording and / or reproducing an audio signal, and an audio signal processing method performed when recording and / or reproducing an audio signal.

  With the development of technology in recent years, it is possible to record underwater sounds. Recording of sound in water is performed using, for example, a hydrophone (underwater microphone). Hydrophones are specially waterproofed and can be used directly in the water. In addition, general hydrophones have sound collection characteristics that are flat and audible in water, and some hydrophones can collect even high-frequency sounds that humans cannot hear. . In addition, waterproof digital cameras and movie cameras that can shoot and collect sound at a depth of several to several tens of meters have been developed.

  On the other hand, human auditory characteristics differ greatly between water and air, and it is known that the audibility in the vicinity of 3 kHz to 8 kHz is very poor in water (see FIG. 1). FIG. 1 is a diagram illustrating a level difference between an underwater equivalent feeling level corresponding to 40 phon in air and an equal feeling level in air, which is disclosed in Non-Patent Document 1. Here, the contents of the graph shown in FIG. 1 will be described in detail. “Phon” means “volume” and indicates the volume of sound felt by humans. Further, the “equivalence level” means a level at which the same “volume” is felt in all frequency bands. The level of equivalence may vary with frequency. That is, the level that feels 40 phon at a certain frequency may be different from the level that feels 40 phon at another frequency. Further, the unit dB on the vertical axis of the graph shown in FIG. 1 is a unit of “sound pressure level” and indicates the signal level itself. Therefore, the graph shown in FIG. 1 shows that when an acoustic signal having an isosensitivity level of 40 phon in air is radiated in water, the audibility characteristics in the vicinity of 3 kHz to 8 kHz are worse in water than in air. Yes. For example, in the vicinity of a frequency of 6 kHz, the signal level that a human feels as 40 phon is about 28 dB higher in water than in air.

  Due to the above-mentioned human auditory characteristics, when submerged in the water, it feels like it was muffled, making it difficult to hear mid-range sounds, but such muffled sounds are “underwater” for humans. It feels like a sound.

Nobuya Kuwabara and 3 others, “Improvement of intelligibility by correcting frequency characteristics based on underwater hearing”, Ocean Acoustics Society Proceedings, 2004, 5, p. 89-92

  However, the sound recorded underwater may be different from the sound heard when the photographer actually submersed in the water, which is uncomfortable. Such a sense of incongruity occurs when the sound collection characteristics of the sound collection device (microphone) in water are different from the human auditory characteristics in water. For example, the sound collected by a general hydrophone is collected with the same sensitivity for all the audible bands, but the sound that the photographer dive in the water is difficult to hear the midrange. Therefore, when the sound is collected and recorded underwater using a general hydrophone, the above-mentioned uncomfortable feeling occurs.

  In view of the above situation, an object of the present invention is to provide an electronic device and an audio signal processing method capable of reducing a sense of incongruity when audio recorded in water is reproduced.

  In order to achieve the above object, an electronic device according to the present invention has a recording and / or reproduction function of a collected audio signal, and records an audio signal collected in water and / or underwater. When reproducing the sound signal collected and recorded in step S1, the sound processing unit is configured to process the sound signal in accordance with the human auditory characteristics in water.

  Further, as an example of the audio processing unit, when recording an audio signal collected in water and / or reproducing an audio signal collected and recorded in water, the audio processing unit is matched with human auditory characteristics in water. An audio correction processing unit that corrects the frequency characteristics of the audio signal.

  As another example of the sound processing unit, when recording a sound signal collected underwater, the weighting of code bit allocation in sound compression coding according to human auditory characteristics in water is performed outside of water. An audio processing unit that changes from the case of recording the audio signal is included.

  Still another example of the voice processing unit is a voice processing unit that combines the two examples described above. The audio processing unit combining the above two examples corrects the frequency characteristics of the audio signal in accordance with the human auditory characteristics in the water and records the human auditory characteristics in the water when recording the audio signal collected in the water. In accordance with the audio correction processing unit for changing the weighting of the code bit allocation in the audio compression coding from when recording the audio signal collected outside the water, or when recording the audio signal collected underwater, When reproducing audio signals collected and recorded underwater, the weighting of code bit allocation in audio compression coding is changed from when recording audio signals collected outside the water according to human auditory characteristics The sound correction processing unit corrects the frequency characteristics of the sound signal in accordance with the human auditory characteristics in water.

  A sound collecting environment determining unit that determines whether or not the sound signal is collected in water, and the sound collecting environment determining unit determines that the sound signal is collected in water; In this case, the audio processing unit may process the audio signal in accordance with the human auditory characteristics in water.

  Moreover, as an example of the electronic device having each configuration described above, an imaging apparatus including a camera that captures an image can be given.

  In order to achieve the above object, the audio signal processing method according to the present invention records an audio signal collected in water and / or reproduces an audio signal collected and recorded in water. In any case, a step of processing an audio signal in accordance with human auditory characteristics in water is provided.

  According to the configuration of the present invention, when recording an audio signal collected in water and / or reproducing an audio signal collected and recorded in water, it is matched with the human auditory characteristics in water. Since the audio signal is processed, it can be brought close to the sound heard when the photographer actually dive in the water, and the uncomfortable feeling when reproducing the sound recorded in the water can be reduced.

  Embodiments of the present invention will be described below with reference to the drawings. Here, an imaging apparatus will be described as an example of the electronic apparatus according to the invention.

<Basic configuration of imaging device>
First, the basic configuration of the imaging apparatus will be described with reference to FIG. FIG. 2 is a block diagram illustrating an internal configuration example of the imaging apparatus according to the present invention.

  The imaging apparatus shown in FIG. 2 is a solid-state imaging device (image sensor) 1 such as a CCD (Charge Coupled Device) or CMOS (Complimentary Metal Oxide Semiconductor) sensor that converts incident light into an electrical signal, and an optical image of a subject. The image sensor 1 outputs a zoom lens that forms an image on the image sensor 1, a motor that changes the focal length of the zoom lens, that is, a motor that changes the optical zoom magnification, and a motor that focuses the zoom lens on the subject. An AFE (Analog Front End) 3 that converts an image signal that is an analog signal into a digital signal; an image processing unit 4 that performs various image processing such as gradation correction on the image signal that is a digital signal from the AFE 3; A compression coding process such as MPEG (Moving Picture Experts Group) compression method is applied to the image signal output from the image processing unit 4. An image compression processing unit 5 that performs the processing, a hydrophone (underwater microphone) 6 that converts sound into an electric signal, and sound processing (at least analog-to-digital conversion processing and MPEG) on the sound signal that is an analog signal from the hydrophone 6 A sound processing unit 7 that performs compression processing such as a compression method), a compression-encoded image signal that is compression-encoded by the image compression processing unit 5, and a compression that is compression-encoded by the sound processing unit 7. A driver unit 8 that records a compressed encoded signal composed of an encoded audio signal in an external memory 22 such as an SD card, and an expansion processing unit 9 that expands and decodes the compressed encoded signal read from the external memory 22 by the driver unit 8 A video output circuit unit 10 for converting an image signal obtained by decoding by the decompression processing unit 9 into an analog signal, and a signal converted by the video output circuit unit 10. A video output terminal 11, a display unit 12 having an LCD (Liquid Crystal Display) or the like that displays an image based on a signal from the video output circuit unit 10, and an audio signal from the expansion processing unit 9 is converted into an analog signal. Audio output circuit unit 13, audio output terminal 14 that outputs a signal converted by audio output circuit unit 13, speaker unit 15 that reproduces and outputs audio based on the audio signal from audio output circuit unit 13, and each block Stores a timing generator (TG) 16 that outputs a timing control signal for matching the operation timings of the two, a CPU (Central Processing Unit) 17 that controls the drive operation of the entire imaging apparatus, and a program for each operation. And a memory 18 for temporarily storing data during program execution, an operation unit 19 for inputting an instruction from the photographer, It comprises a PU17 a bus line 20 for exchanging data with each block, and a bus line 21 for exchanging data between the memory 18 and each of the blocks. The CPU 17 controls the focus and the diaphragm by driving each motor in the lens unit 2 according to the image signal detected by the image processing unit 4.

<Basic operation of imaging device>
Next, a basic operation of the imaging apparatus shown in FIG. 2 when shooting a moving image will be described with reference to FIG. First, the imaging apparatus acquires an image signal that is an electrical signal by photoelectrically converting light incident from the lens unit 2 in the image sensor 1. Then, in synchronization with the timing control signal input from the timing generator 16, the image sensor 1 sequentially outputs image signals to the AFE 3 at a predetermined frame period (for example, 1/60 seconds). The CPU 17 performs camera control (AF, AE, ISO sensitivity, etc.) on the image sensor 1 and the lens unit 2 in accordance with the set shooting mode.

  Then, the image signal converted from the analog signal to the digital signal by the AFE 3 is input to the image processing unit 4. The image processing unit 4 converts the input image signal into an image signal composed of a luminance signal and a color difference signal, and performs various image processing such as gradation correction and contour enhancement. The memory 18 operates as a frame memory, and temporarily holds an image signal when the image processing unit 4 performs processing. Then, the image signal output from the image processing unit 4 is input to the image compression processing unit 5 and is compressed by the predetermined compression method in the image compression processing unit 5.

  At this time, based on the image signal input to the image processing unit 4, the lens unit 2 adjusts the position of various lenses to adjust the focus, or adjusts the opening of the diaphragm to adjust the exposure. It is done. This adjustment of focus and exposure is automatically performed based on a predetermined program so as to be in an optimum state, or manually performed based on a photographer's instruction.

  On the other hand, an audio signal converted into an electric signal in the hydrophone 6 is input to the audio processing unit 7. The sound processing unit 7 converts an input sound signal into a digital signal, and in some cases, performs sound correction processing such as noise removal and sound signal intensity control, and then compressed by a predetermined compression method.

  The compression-encoded image signal output from the image compression processing unit 5 and the compression-encoded audio signal output from the audio processing unit 7 are temporally related so that the image and sound do not shift during reproduction. Composed. The compressed image signal and audio signal are recorded in the external memory 22 via the driver unit 8.

  The compression-coded signal recorded in the external memory 22 is read by the decompression processing unit 9 in accordance with the output signal from the operation unit 19 based on the photographer's instruction. The decompression processing unit 9 decompresses and decodes the compressed encoded signal, and generates an image signal and an audio signal. The image signal is output to the video output circuit unit 10 and the audio signal is output to the audio output circuit unit 13. Then, in the video output circuit unit 10 and the audio output circuit unit 13, the image signal and the audio signal are converted into a format reproducible on the display unit 12 and the speaker unit 15 and output.

  Further, in a so-called preview mode in which the photographer confirms an image displayed on the display unit 12 without recording an image signal, the image compression processing unit 5 is prevented from performing the compression processing, and the image processing unit 4 may output the image signal to the video output circuit unit 10 instead of the image compression processing unit 5. Further, when recording an image signal, the image signal may be output to the display unit 12 via the video output circuit 10 in parallel with the operation of recording in the external memory 22 via the driver unit 8. .

  In the configuration shown in FIG. 2, the display unit 12 and the speaker unit 15 are mounted on the imaging device. However, the display unit 12 and the speaker unit 15 are separated from the imaging device, and terminals (video output) provided in the imaging device. The terminal 11 and the audio output terminal 14) may be connected using a cable or the like.

<First Example of Audio Processing Unit>
First, a first embodiment of the voice processing unit 7 will be described with reference to FIGS. FIG. 3 is a block diagram showing the configuration of the voice processing unit 7 when the first embodiment of the voice processing unit 7 is adopted. In the first embodiment of the audio processing unit 7, the audio processing unit 7 converts an analog audio signal output from the hydrophone 6 into a digital audio signal, and outputs from the A / D conversion unit 71. A correction processing unit 72 that performs correction processing on the digital audio signal to be processed, and an audio compression processing unit 73 that performs compression encoding processing such as MPEG compression on the audio signal output from the correction processing unit 72. ing. 4A is a diagram showing the frequency characteristics of the input signal of the correction processing unit 72 when white noise is reproduced in water, and FIG. 4B is a correction process when white noise is reproduced in water. 7 is a diagram illustrating frequency characteristics of an output signal of a unit 72. FIG.

  As described above, a general hydrophone has a flat sound collecting characteristic in water, whereas a human auditory characteristic in water has a characteristic as shown in FIG. Therefore, the correction processing unit 72 reduces the signal level of 3 kHz to 8 kHz by about several dB to 30 dB with respect to the digital audio signal obtained by A / D converting the analog audio signal output from the hydrophone 6, thereby allowing human hearing underwater. Correct to a frequency characteristic close to the characteristic. Thereby, the frequency characteristic of the input / output signal of the correction processing unit 72 as shown in FIG. 4 can be obtained, and the uncomfortable feeling when reproducing the sound recorded in water can be reduced.

  A summary of operations continuously performed by the voice processing unit 7 in the case where the first embodiment of the voice processing unit 7 is adopted is summarized as a flowchart shown in FIG. That is, the audio processing unit 7 when the first example of the audio processing unit 7 is adopted performs A / D conversion processing for converting the analog audio signal output from the hydrophone 6 into a digital audio signal (step # 10). ) The digital audio signal obtained by the A / D conversion process is subjected to a correction process for reducing the signal level from 3 kHz to 8 kHz by about several dB to 30 dB and correcting the frequency characteristic close to human auditory characteristics in water. (Step # 20) A series of operations of performing compression encoding processing such as MPEG compression on the audio signal obtained by the correction processing (Step # 30) is continuously performed.

  In addition, although the hydrophone 6 is used in the imaging device shown in FIG. 2, the sound collecting device is not limited to the hydrophone. The correction processing setting of the correction processing unit 72 is changed so that the sound collection characteristic of the sound collection device in water can be regarded as the characteristic shown in FIG. As a result, even when a sound collecting device other than the hydrophone is used, the correction processing unit 72 corrects the frequency characteristics close to human auditory characteristics in water.

  In addition, when the hydrophone 6 can collect a high-band sound that cannot be heard by humans, the correction processing unit 72 reduces the signal level of the high-band sound that cannot be heard by humans. Correction processing may also be performed.

<Second Embodiment of Audio Processing Unit>
Next, a second embodiment of the audio processing unit 7 will be described with reference to FIGS. FIG. 6 is a block diagram showing the configuration of the audio processing unit 7 when the second embodiment of the audio processing unit 7 is adopted. In the second embodiment of the audio processing unit 7, the audio processing unit 7 converts an analog audio signal output from the hydrophone 6 into a digital audio signal, and an output from the A / D conversion unit 71. An audio compression processing unit 73 that performs compression encoding processing such as an MPEG compression method on the audio signal to be recorded, and a sound collection environment determination unit 74 that determines whether or not the audio signal is collected in water It has.

  In general, speech compression coding considers human auditory psychology, weights so that many code bits are assigned to the band with the best hearing sensitivity, and compresses the speech signal so that the quantization error is reduced. ing. In the air, since 3 kHz to 4 kHz is the band with the highest audibility, generally, the weight is assigned so that the most sign bit is assigned to 3 kHz to 4 kHz. On the other hand, in water, the auditory characteristics shown in FIG. 1 are obtained, so that the auditory sensitivity particularly in the vicinity of 3 kHz to 8 kHz is deteriorated.

  Therefore, in the second embodiment of the audio processing unit 7, when the audio compression processing unit 73 records an audio signal collected underwater, code bit allocation to 3 kHz to 8 kHz according to human auditory characteristics in water is performed. And the weighting of the code bit assignment is changed from the case of recording a sound signal collected in the air so that the code bit assignment to other bands is increased.

  In order to realize the above-described change in the weighting of the code bit assignment, in the second embodiment of the sound processing unit 7, the sound compression processing unit 73 includes a compression coding unit 731 and an underwater code bit weighting table storage unit 732. And a code bit weighting table storage unit 733 for the air.

  When the sound collection environment determination unit 74 determines that the sound signal collected by the hydrophone 6 is collected underwater, the compression encoding unit 731 stores the code signal in the underwater code bit weighting table storage unit 732. Compression coding processing is performed using the underwater code bit weighting table (a table in which code bit allocation to 3 kHz to 8 kHz is small and code bit allocation to other bands is large). On the other hand, when the sound collection environment determination unit 74 determines that the sound signal collected by the hydrophone 6 is collected in the air, the compression coding unit 731 stores the code bit weighting table for air. The compression coding process is performed using a code bit weighting table for air stored in the unit 733 (a table in which code bits are most allocated to 3 kHz to 4 kHz).

  In FIG. 6, an embodiment in which the sound collection environment determination unit 74 inputs a digital audio signal output from the A / D conversion unit 71, that is, the sound collection environment determination unit 74 is output from the A / D conversion unit 71. Although the embodiment for determining the sound collection environment based on the digital audio signal is illustrated, the sound collection environment determination unit 74 is not limited to this embodiment. Details of the embodiment of the sound collection environment determination unit 74 will be described later.

  A summary of operations continuously performed by the voice processing unit 7 when the second embodiment of the voice processing unit 7 is adopted is summarized as a flowchart shown in FIG. That is, the sound processing unit 7 when the second embodiment of the sound processing unit 7 is adopted performs A / D conversion processing for converting the analog sound signal output from the hydrophone 6 into a digital sound signal (step # 110). ), It is determined whether or not the sound signal collected by the hydrophone 6 is collected in water (step # 120), and the sound signal collected by the hydrophone 6 is collected in water. If it is determined that the digital audio signal is determined (YES in step # 120), the digital audio signal obtained by the A / D conversion process is subjected to the compression encoding process using the underwater code bit weighting table (step S120). # 130) If it is not determined that the sound signal collected by the hydrophone 6 is collected in water (NO in step # 120), the code bit for air is used. Using weighting table is performed continuously a series of operations that performs compression coding processing on the digital audio signal obtained by the A / D conversion processing (step # 140).

  In addition, although the hydrophone 6 is used in the imaging device shown in FIG. 2, the sound collecting device is not limited to the hydrophone. Even when a sound collecting device other than the hydrophone is used, if the sound collecting environment determining unit 74 determines that the sound signal collected by the sound collecting device is collected in water, the compression encoding unit 731 is an underwater code bit weighting table stored in the underwater code bit weighting table storage unit 732 (a table in which code bit allocation to 3 kHz to 8 kHz is small and code bit allocation to other bands is large). A compression encoding process may be performed.

<Third embodiment of voice processing unit>
When shooting with a waterproof digital camera or movie camera, the location where the sound is collected may be in the air or in the water. In particular, when shooting near the surface of the water, it is expected to frequently go back and forth between the water (underwater) and the outside (in the air) during the shooting. The second embodiment of the voice processing unit described above can cope with such a scene, but the first embodiment of the voice processing unit described above cannot cope with such a scene.

  Therefore, in the third embodiment of the sound processing unit 7, the first embodiment of the sound processing unit described above is modified so that the place where the sound is collected is air as in the second embodiment of the sound processing unit described above. Be able to cope with the situation in the middle or underwater. Hereinafter, a third embodiment of the voice processing unit 7 will be described with reference to FIGS.

  FIG. 8 is a block diagram showing the configuration of the voice processing unit 7 when the third embodiment of the voice processing unit 7 is adopted. In the third embodiment of the sound processing unit 7, the sound processing unit 7 has an A / D conversion unit 71 that converts an analog sound signal output from the hydrophone 6 into a digital sound signal, and the sound signal is collected in water. From the A / D conversion unit 71 when the sound collection environment determination unit 74 determines whether the sound signal is collected underwater or not. A correction processing unit 72 that performs correction processing on the output digital audio signal, and a sound that is output from the A / D conversion unit 71 and that is input without passing through the correction processing unit 72 or through the correction processing unit 72 And an audio compression processing unit 73 that performs compression encoding processing such as MPEG compression on the signal.

  In FIG. 8, an embodiment in which the sound collection environment determination unit 74 inputs a digital audio signal output from the A / D conversion unit 71, that is, the sound collection environment determination unit 74 is output from the A / D conversion unit 71. Although the embodiment for determining the sound collection environment based on the digital audio signal is illustrated, the sound collection environment determination unit 74 is not limited to this embodiment. Details of the embodiment of the sound collection environment determination unit 74 will be described later.

  A summary of operations continuously performed by the voice processing unit 7 in the case where the third embodiment of the voice processing unit 7 is adopted is summarized as a flowchart shown in FIG. That is, the sound processing unit 7 when the third embodiment of the sound processing unit 7 is adopted performs A / D conversion processing for converting the analog sound signal output from the hydrophone 6 into a digital sound signal (step # 210). ), It is determined whether or not the sound signal collected by the hydrophone 6 is collected in water (step # 220), and the sound signal collected by the hydrophone 6 is collected in water. If it is determined that the signal level is YES (YES in step # 220), the signal level from 3 kHz to 8 kHz is reduced by several dB to 30 dB with respect to the digital audio signal obtained by the A / D conversion process in step # 210. Then, a correction process for correcting the frequency characteristics close to human auditory characteristics in water is performed (step # 230), and then the audio signal obtained by the correction process is subjected to M When compression encoding processing such as the EG compression method is performed (step # 240), on the other hand, when it is not determined that the audio signal collected by the hydrophone 6 is collected in water (step # 220) NO) continuously performs a series of operations of performing compression encoding processing such as MPEG compression on the digital audio signal obtained by the A / D conversion processing in step # 210 (step # 240).

  In addition, although the hydrophone 6 is used in the imaging device shown in FIG. 2, the sound collecting device is not limited to the hydrophone. The correction processing setting of the correction processing unit 72 is changed so that the sound collection characteristic of the sound collection device in water can be regarded as the characteristic shown in FIG. As a result, even when a sound collecting device other than the hydrophone is used, the correction processing unit 72 corrects the frequency characteristics close to human auditory characteristics in water.

  In addition, when the hydrophone 6 can collect a high-band sound that cannot be heard by humans, the correction processing unit 72 reduces the signal level of the high-band sound that cannot be heard by humans. Correction processing may also be performed.

  Further, the correction is performed only when it is determined that the sound signal collected by the hydrophone 6 is collected in the air, and the correction process suitable for the sound signal collected in the air is performed. A processing unit may be provided in the sound processing unit 7. When it is determined that the sound signal collected by the hydrophone 6 is collected in water, and the sound signal collected by the hydrophone 6. Correction to perform correction processing suitable for both the sound signal collected in water and the sound signal collected in the air, which are executed both when it is determined that the sound is collected in the air A processing unit may be provided in the audio processing unit 7.

  The third embodiment of the voice processing unit can be executed in combination with the second embodiment of the voice processing unit.

<First Example of Sound Collection Environment Determination Unit>
Next, a first example of the sound collection environment determination unit 74 used in the second and third examples of the sound processing unit 7 will be described with reference to FIG. FIG. 10 is a block diagram showing the configuration of the sound collection environment determination unit 74 when the first embodiment of the sound collection environment determination unit 74 is adopted. In the first embodiment of the sound collection environment determination unit 74, the sound collection environment determination unit 74 includes a frequency characteristic determination unit 741.

  Here, FIG. 11 shows frequency characteristics when white noise is reproduced in the air and collected in the air. FIG. 12 shows frequency characteristics when white noise is reproduced in the air and collected in water. The frequency characteristics shown in FIG. 11 and FIG. 12 are sound collections in which a waterproof structure is applied to a housing such as an electret condenser microphone (Electret Condenser Microphone) or a MEMS (Micro Electro Mechanical System) microphone generally used in the air. It was obtained by collecting sound with the device.

  When the sound is collected in the air, the frequency characteristic is substantially flat as shown in FIG. On the other hand, as for the frequency characteristics when sound is collected in water, generally, if the signal level is large, the signal in the high frequency band is greatly attenuated as shown in FIG. This is because the propagated sound is attenuated by reflection at the two boundaries in the air-underwater, underwater-inside the housing of the sound collecting device (in the air), and the sound of the waves newly generated in the water and the case This is because generally low sounds such as newly generated sounds remain inside.

  In this way, when the imaging device is used underwater, low-band sounds, medium-band sounds, and high-band sounds that cannot occur when the imaging device is used in air, Therefore, the determination is performed using the level difference. Note that when the sound is collected underwater using a hydrophone microphone, there is no boundary between the underwater and the housing of the sound collecting device (in the air). The level difference is smaller than in the case of FIGS. However, as long as the sound source is in the air, the transmitted sound is attenuated by reflection at the air-water boundary, so the difference in level between the low-band sound, the mid-band sound, and the high-band sound is used. Sound collection environment judgment is possible.

  The frequency characteristic determination unit 741 targets each of an audio signal as a low band (for example, several tens (70) Hz to 3 kHz), a medium band (for example, 6 kHz to 9 kHz), and a high band (for example, 12 kHz to 15 kHz). To calculate the average signal level. In addition, the specific numerical value of each band is not limited to the above example, and there is no problem if the magnitude relationship between the bands is correct. Further, the low band and the middle band may partially overlap, and the middle band and the high band may partially overlap.

  The signal level ratio of the low band to the high band (low band / high band) R1 and the signal level ratio of the low band to the medium band (low band / medium band) that can be calculated from the average value of the signal level in each band ) R2 and the signal level ratio (medium band / high band) R3 of the medium band to the high band show time changes as shown in FIG. 13 when the microphone is inserted from the air into the water and returned to the air again. (FIG. 13 shows data when a microphone other than the hydrophone is used). Periods T1 and T3 in FIG. 13 are periods in which the microphone is located in the air, and periods T2 in FIG. 13 are periods in which the microphone is located in the water. The signal level ratio (medium band / high band) R3 of the medium band to the high band is a substantially constant value regardless of whether in the air or underwater. On the other hand, the low-band signal level ratio (low band / high band) R1 to the high band and the low-band signal level ratio (low band / middle band) R2 to the medium band are small values in the air. The sensitivity of sound reception changes in water, which is a much larger value than in air.

  By utilizing this, the frequency characteristic determination unit 741 obtains a low-band signal level ratio (low band / high band) R1 for the high band and a low-band signal level ratio (low band / middle band) R2 for the medium band. Calculated from the average value of the signal level in each band, the low band signal level ratio (low band / high band) R1 to the high band and the low band signal level ratio (low band / middle band) R2 to the medium band are preset. When it becomes larger than the above threshold, it is determined that the audio signal input to the audio processing unit 7 is collected in water. Although the determination accuracy is inferior, the average value of the signal level in the medium band and the signal level ratio of the low band to the medium band (low band / medium band) R2 are not calculated, and the signal level ratio of the low band to the high band ( When the low band / high band (R1) is greater than or equal to a preset threshold value, it is determined that the audio signal input to the audio processing unit 7 is collected in water, or the signal in the high band The average value of the level and the signal level ratio of the low band to the high band (low band / high band) R1 are not calculated, and the signal level ratio of the low band to the medium band (low band / medium band) R2 is equal to or greater than a preset threshold It is also possible to determine that the audio signal input to the audio processing unit 7 has been collected underwater.

  Even underwater, sudden noise occurs due to the sound of bubbles and the rubbing sound of the housing, and the signal level of the middle band and the high band increases instantaneously, and the signal level ratio of the low band to the high band (low There is a possibility that the signal level ratio (low band / medium band) R2 of the low band with respect to the band (high band) R1 and the medium band instantaneously becomes a small value. Therefore, the low band signal level ratio (low band / high band) R1 to the high band and the low band signal level ratio (low band / middle band) R2 to the high band used by the frequency characteristic determination unit 741 are constant. It is desirable to use an averaged value over time.

  Further, regarding the threshold value, it is desirable to provide a hysteresis characteristic and set the threshold value high while determining that it is in the air, and setting the threshold value low while determining that it is underwater.

<Second Example of Sound Collection Environment Determination Unit>
Next, a second example of the sound collection environment determination unit 74 used in the second and third examples of the sound processing unit 7 will be described with reference to FIG. FIG. 14 is a block diagram showing a configuration of the sound collection environment determination unit 74 when the second embodiment of the sound collection environment determination unit 74 is adopted. In the second embodiment of the sound collection environment determination unit 74, the sound collection environment determination unit 74 includes a propagation characteristic determination unit 742. The propagation characteristic determination unit 742 is an environment in which an input audio signal is collected based on a difference between the propagation characteristic of an audio signal collected in the air and the propagation characteristic of an audio signal collected in water. Determine.

  As a propagation characteristic of an audio signal used for the determination by the propagation characteristic determination unit 742, for example, there is a difference in sound speed. The speed of sound in the air is about 344 m / s, and the speed of sound in water is about 1500 m / s. In this example, the sound collection environment is determined based on the difference in sound speed. Further, when making such a determination, for example, a driving sound of a motor that changes the optical zoom magnification provided in the lens unit 2 can be used. The case where determination is performed in this way will be described below.

  The CPU 17 controls the driving of the motor that changes the optical zoom magnification in the lens unit 2 according to the output of the operation unit 19. Further, the CPU 17 performs time management on the driving timing of the motor that changes the optical zoom magnification in the lens unit 2, and provides the information to the propagation characteristic determination unit 742. The drive sound of the motor that changes the optical zoom magnification in the lens unit 2 is collected by the hydrophone 6 as drive sound (self-generated drive sound) in the imaging apparatus shown in FIG. Then, in the propagation characteristic determination unit 742, the time information regarding the drive timing of the motor that changes the optical zoom magnification in the lens unit 2 provided from the CPU 17, and the self-generated driving sound collected by the hydrophone 6 (the lens unit 2). The transmission speed of the self-generated driving sound is measured based on the sound collecting time of the driving sound of the motor that changes the optical zoom magnification. In the imaging apparatus shown in FIG. 2, since the transmission speed of the self-generated driving sound in the air is obtained in advance, it is possible to determine whether the sound is collected in the air or in the water.

<Third Example of Sound Collection Environment Determination Unit>
Next, a third example of the sound collection environment determination unit 74 used in the second and third examples of the sound processing unit 7 will be described with reference to FIG. FIG. 15 is a block diagram showing the configuration of the sound collection environment determination unit 74 when the third embodiment of the sound collection environment determination unit 74 is adopted. In the third embodiment of the sound collection environment determination unit 74, the sound collection environment determination unit 74 includes a propagation characteristic determination unit 743. The propagation characteristic determination unit 743 determines the environment where the input audio signal is collected based on the difference in the propagation characteristics of the audio signal in air and underwater. However, the propagation characteristic determination unit 743 performs determination using a method different from the propagation characteristic determination unit 742 of the second embodiment illustrated in FIG.

  A determination method of the propagation characteristic determination unit 743 will be described with reference to FIG. When the third embodiment of the underwater determination unit is implemented, the hydrophone 6 is a stereo microphone including a microphone 6L and a microphone 6R as shown in FIG. 16, for example. As shown in FIG. 16, when the sound emitted by the sound source is collected by the stereo microphone, a path difference d is generated between the microphone 6L and the microphone 6R. Then, due to this path difference d, a difference in arrival time (a value obtained by dividing the path difference d by the speed of sound) occurs. In addition, the difference in arrival time is different between the audio signal (Rch) input from the microphone 6R to the audio processing unit 7 and the audio signal (Lch) input from the microphone 6L to the audio processing unit 7. A phase difference occurs.

  As described above, the speed of sound differs between air and water. Therefore, the phase difference generated between the audio signal (Rch) and the audio signal (Lch) differs depending on the audio signal collected in the air and the audio signal collected in the water. Therefore, the propagation characteristic determination unit 743 determines whether or not the sound signal input to the sound collection environment determination unit 74 is collected in water based on the difference in phase difference.

  It should be noted that the present embodiment can be implemented even if the hydrophone 6 is replaced with a stereo microphone instead of a microphone array of another channel (for example, a 5.1ch surround recording compatible microphone).

<Fourth Example of Sound Collection Environment Determination Unit>
Next, a fourth embodiment of the sound collection environment determination unit 74 will be described with reference to FIG. FIG. 17 is a block diagram showing the configuration of the sound collection environment determination unit 74 when the fourth embodiment of the sound collection environment determination unit 74 is adopted. In the fourth embodiment of the sound collection environment determination unit 74, the sound collection environment determination unit 74 includes a pressure determination unit 744.

  When the fourth embodiment of the sound collection environment determination unit 74 is employed, a pressure sensor is newly provided in the imaging apparatus shown in FIG. The pressure determination unit 744 receives the detection signal of the pressure sensor, and when the pressure outside the imaging device is equal to or higher than a preset threshold, the imaging device is used in water, and the audio input to the audio processing unit 7 When it is determined that the signal is collected in water and the pressure outside the imaging device is less than a preset threshold, the imaging device is used in the air and is input to the audio processing unit 7 It is determined that the audio signal is collected in the air.

<Fifth Example of Sound Collection Environment Determination Unit>
Next, a fifth embodiment of the sound collection environment determination unit 74 will be described with reference to FIG. FIG. 18 is a block diagram showing the configuration of the sound collection environment determination unit 74 when the fifth embodiment of the sound collection environment determination unit 74 is adopted. In the fifth embodiment of the sound collection environment determination unit 74, the sound collection environment determination unit 74 includes a video determination unit 745.

  The video determination unit 745 analyzes the video signal input from the image processing unit 4 and determines whether the imaging device is used in water. The video determination unit 745 receives, for example, the video signal input from the image processing unit 4 and before white balance adjustment from the image processing unit 4, analyzes the color distribution of the video signal before white balance adjustment, and the imaging apparatus Determine if it is used underwater.

  Note that the video determination unit 745 may be provided not in the audio processing unit 7 but in the image processing unit 4, and the sound collection environment determination unit 74 may input the determination result of the video determination unit 745.

<Modification>
The above-described imaging device shown in FIG. 2 includes a processing unit that can process an audio signal in accordance with human auditory characteristics in water when recording the collected audio signal. However, the present invention is not limited to this, and when a collected audio signal is reproduced, a processing unit that can process the audio signal in accordance with human auditory characteristics in water may be provided.

  FIG. 19 illustrates an imaging apparatus including a processing unit that can process an audio signal in accordance with human auditory characteristics in water when reproducing the collected audio signal when reproducing the collected audio signal. Shown in In FIG. 19, the same reference numerals are given to substantially the same parts as those in FIG.

  The imaging apparatus shown in FIG. 19 is different from the imaging apparatus shown in FIG. 2 in that an audio processing unit 7a is provided instead of the audio processing unit 7, and audio processing is performed between the expansion processing unit 9 and the audio output circuit unit 13. The point is that a portion 7b is provided.

  Unlike the audio processing unit 7, the audio processing unit 7 a does not include a processing unit that can process an audio signal in accordance with human auditory characteristics in water when recording an audio signal collected in water. It is.

  The audio processing unit 7b has the same configuration as the audio processing unit 7 except that the A / D conversion process and the compression encoding process are not performed. Since the audio processing performed in the audio processing unit 7b is basically the same as the audio processing performed in the audio processing unit 7, the description thereof is omitted here. In addition, since the audio | voice processing part 7b does not perform a compression encoding process, 2nd Example of the audio | voice processing part mentioned above cannot be implemented.

  Note that there is a possibility that the sound collection environment (the environment in which the imaging device shown in FIG. 19 is placed at the time of sound collection) and the reproduction environment (the environment in which the imaging device shown in FIG. 19 is placed at the time of reproduction) are different. Therefore, the fourth embodiment of the sound collection environment determination unit described above is not adopted in the sound collection environment determination unit of the sound processing unit 7b.

  Further, in the imaging apparatus shown in FIG. 2 or FIG. 19 described above, it is possible to match the human auditory characteristics in water only when recording the collected audio signal or reproducing the collected audio signal. The sound signal may not be processed, but the sound signal may be processed in accordance with the human auditory characteristics in water both when recording the collected sound signal and when reproducing the collected and recorded sound signal. . In this case, for example, when recording the collected audio signal, the weighting of the code bit allocation in the audio compression encoding is changed from the case of recording the audio signal collected outside the underwater according to the human auditory characteristics in the water, When playing back the collected and recorded audio signal, the frequency characteristics of the audio signal may be corrected according to the human auditory characteristics underwater, or the frequency characteristics of the audio signal are corrected according to the human auditory characteristics underwater. May be divided into two stages, and the first stage correction may be performed when recording the collected audio signal, and the second stage correction may be performed when reproducing the collected and recorded audio signal.

  In addition, since the electronic apparatus according to the present invention only needs to be able to record and / or reproduce an audio signal, a block related to an image is not particularly necessary. Therefore, the present invention can also be applied to electronic devices other than the imaging device, for example, an audio recording device, an audio reproducing device, an audio recording / reproducing device (for example, an IC recorder), and the like.

  In addition, it is desirable that the electronic device equipped with the sound collection environment determination device according to the present invention has a waterproof structure, but even if the electronic device is not waterproof, for example, an audio signal stored in an waterproof housing and collected by an external hydrophone It is possible to adopt a usage such as inputting.

  In addition, for example, the electronic device according to the present invention can be realized by operating reproduction software including a program for causing a computer to function as each unit of the audio processing unit 7b on a personal computer.

  In addition to the above-described case, the imaging apparatus and the sound processing units 7, 7 a, and 7 b shown in FIGS. 2 and 19 can be realized by hardware or a combination of hardware and software. In addition, when the image capturing apparatus and the sound processing units 7, 7 a, and 7 b of FIGS. 2 and 19 are configured using software, the block diagram of the part realized by the software represents a functional block diagram of the part. To do.

  As mentioned above, although embodiment which concerns on this invention was described, the range of this invention is not limited to this, A various change can be added and implemented in the range which does not deviate from the main point of invention.

These are figures which show the level difference of the isosensitive level in water, and the isosensitive level in air. These are block diagrams which show the example of 1 internal structure of the imaging device which concerns on this invention. These are block diagrams which show the structure of 1st Example of an audio | voice processing part. These are figures which show the frequency characteristic of the input-output signal of a correction process part at the time of reproducing | regenerating white noise in water. These are the flowcharts which show the outline of the operation | movement which the audio | voice processing part of 1st Example performs continuously. These are block diagrams which show the structure of 2nd Example of an audio | voice processing part. These are the flowcharts which show the outline of the operation | movement which the audio | voice processing part of 2nd Example performs continuously. These are block diagrams which show the structure of 3rd Example of an audio | voice processing part. These are the flowcharts which show the outline of the operation | movement which the audio | voice processing part of 3rd Example performs continuously. These are block diagrams which show the structure of 1st Example of a sound collection environment determination part. These are figures which show the frequency characteristic of the sound in the air. FIG. 4 is a diagram illustrating frequency characteristics of sound in water. These are figures which show the difference in the frequency characteristic of the sound in the air and underwater. These are block diagrams which show the structure of 2nd Example of a sound collection environment determination part. These are block diagrams which show the structure of 3rd Example of a sound collection environment determination part. These are schematic diagrams of a stereo microphone. These are block diagrams which show the structure of 4th Example of a sound collection environment determination part. These are block diagrams which show the structure of 5th Example of a sound collection environment determination part. These are the block diagrams which show the other internal structural example of the imaging device which concerns on this invention.

Explanation of symbols

1 Solid-state image sensor (image sensor)
2 Lens part 3 AFE
4 Image processing unit 5 Image compression processing unit 6 Hydrophone 7, 7a, 7b Audio processing unit 8 Driver unit 9 Decompression processing unit 10 Video output circuit unit 11 Video output terminal 12 Display unit 13 Audio output circuit unit 14 Audio output terminal 15 Speaker Part 16 Timing generator (TG)
17 CPU
DESCRIPTION OF SYMBOLS 18 Memory 19 Operation part 20, 21 Bus line 22 External memory 71 A / D conversion part 72 Correction processing part 73 Voice compression processing part 731 Compression encoding part 732 Underwater code bit weighting table storage part 733 In-air code bit weighting table Storage unit 74 Sound collection environment determination unit 741 Frequency characteristic determination unit 742, 743 Propagation characteristic determination unit 744 Pressure determination unit 745 Video determination unit

Claims (6)

It has a recording and / or playback function for collected audio signals
An audio processing unit for processing an audio signal in accordance with human auditory characteristics in water when recording an audio signal collected in water and / or reproducing an audio signal collected and recorded in water; An electronic device comprising the electronic device.   When the sound processing unit records a sound signal collected in water and / or reproduces a sound signal collected and recorded in water, the sound processing unit The electronic device according to claim 1, wherein the frequency characteristic is corrected.   When recording the sound signal collected underwater, the sound processing unit records the sound signal collected outside the water with weighting of code bit allocation in sound compression coding in accordance with human auditory characteristics in water. The electronic device according to claim 1 or 2, which is changed from the case.   A sound collection environment determination unit that determines whether or not the sound signal is collected in water, and the sound collection environment determination unit determines that the sound signal is collected in water The electronic device according to claim 1, wherein the audio processing unit processes an audio signal in accordance with human auditory characteristics in water.   The electronic apparatus according to claim 1, wherein the electronic apparatus includes an imaging device that captures an image.   When recording an audio signal collected in water or reproducing an audio signal collected and recorded in water, the audio signal is processed according to human auditory characteristics in water An audio signal processing method.