JP2016051917A - Voice processing device, imaging device, and voice processing device control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent a voice signal without sound from being recorded for a long period in a voice processing device that receives voice via radio communication.SOLUTION: A voice processing device comprises: a microphone mounted on a device body; voice input means capable of accepting voice input via radio communication; silence detection means for detecting silence at the input level into the voice input means; mix rate determination means for determining a mix rate of mixing the voice from the voice input means and the voice from the microphone on the basis of the detection result by the silence detection means; and mixing means for mixing the voice from the voice input means and the voice from the microphone at the mixed rate determined by the mix rate determination means.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an audio processing device, an imaging device, and a control method for the audio processing device.

  2. Description of the Related Art Conventionally, as an apparatus that processes an audio signal, an imaging apparatus that records an image signal and an audio signal is known. Some of such imaging devices can use an external microphone that acquires an audio signal by wireless communication, in addition to the built-in microphone provided in the imaging device. When an external microphone is used, a device that cuts off and records the sound of the built-in microphone is generally used. When using an external microphone that acquires an audio signal by wireless communication, it often occurs that the wireless communication is interrupted due to the deterioration of the radio wave environment and the audio data is interrupted and silence is recorded.

  A technique for complementing such a problem as much as possible so that the voice on wireless communication is not interrupted has been proposed (Patent Document 1).

JP 2003-298541 A

  However, it is difficult to continue to compensate for the interruption of sound when the interruption time becomes long or when many disturbing waves enter. Therefore, there is a problem that the audio signal becomes silent for a long period of time, and the user is uncomfortable.

  Accordingly, an object of the present invention is to prevent a silent audio signal from being recorded for a long period of time in an audio processing device that receives audio via wireless communication.

  According to a first aspect of the present invention, there is provided a microphone provided in the apparatus main body, a voice input unit capable of receiving voice input via wireless communication, and a silence detecting that the input level of the voice input unit is silent. Based on a detection result by the detection means, the silence detection means, a mix ratio determination means for determining a mix ratio for mixing the sound of the sound input means and the sound of the microphone, and the mix ratio determination means An audio processing apparatus comprising: mixing means for mixing the sound of the sound input means and the sound of the microphone at a mix ratio.

  According to a second aspect of the present invention, there is provided a microphone provided in the apparatus main body, a voice input unit capable of accepting voice input via wireless communication, a reception status acquisition unit that acquires the reception status of the wireless communication, Based on the reception status acquired by the reception status acquisition means, a mix ratio determination means for determining a mix ratio for mixing the sound of the voice input means and the sound of the microphone, and the mix ratio determined by the mix ratio determination means And a mixing means for mixing the sound of the sound input means and the sound of the microphone.

According to a third aspect of the present invention, there is provided a method for controlling a voice processing apparatus, comprising: a microphone provided in the apparatus main body; and voice input means capable of receiving voice input via wireless communication, wherein the voice input means A silence detection step for detecting that the input level of the sound acquired from the sound is silence, and a mix ratio for mixing the sound of the sound input means and the sound of the microphone is determined based on the detection result in the silence detection step. And a mixing step of mixing the sound of the sound input means and the sound of the microphone with the mix ratio determined in the mix ratio determination step. .

  According to a fourth aspect of the present invention, there is provided a method for controlling a voice processing apparatus comprising: a microphone provided in the apparatus main body; and a voice input unit capable of receiving voice input via radio communication. A reception status acquisition step for acquiring a reception status; a mix ratio determination step for determining a mix ratio for mixing the voice of the voice input unit and the voice of the microphone based on the reception status acquired in the reception status acquisition step; And a mixing step of mixing the sound of the sound input means and the sound of the microphone at the mix ratio determined in the mix ratio determination step.

  ADVANTAGE OF THE INVENTION According to this invention, it can suppress that a silence audio | voice signal is recorded for a long period in the audio processing apparatus which receives an audio | voice via wireless communication.

1 is an overall block diagram of an imaging apparatus according to a first embodiment. The block diagram of the audio | voice processing part in 1st Embodiment. The flowchart of the audio | voice mix process in 1st Embodiment. The figure which shows the example of the mix ratio determination process in 1st Embodiment. FIG. 6 is an overall block diagram of an imaging apparatus according to a second embodiment. The block diagram of the audio | voice processing part in 2nd Embodiment. The flowchart of the audio | voice mix process in 2nd Embodiment. The figure which shows the example of the provisional mix ratio determination process in 1st Embodiment.

  Hereinafter, examples of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiments.

<First Embodiment>
[Constitution]
The imaging apparatus 100 according to the present embodiment will be described with reference to FIGS. 1, 2, 3, and 4. The imaging apparatus 100 is a digital video camera that acquires moving images and audio, stores them in association with each other. The imaging apparatus 100 can use an internal microphone and an optional external microphone, as will be described later, and the sound to be recorded is an audio signal after being subjected to a mix process by the audio processing apparatus.

FIG. 1 is a block diagram illustrating an overall configuration of the imaging apparatus 100 according to the first embodiment. The imaging unit 101 includes an imaging lens (imaging optical system) and an imaging element such as a CCD sensor or a CMOS sensor. The imaging unit 101 converts the optical image of the subject captured by the photographing lens into an image signal by the imaging element, performs analog-digital conversion, and transmits the image signal to the image processing unit 102.
The image processing unit 102 performs image quality adjustment processing for adjusting white balance, color, brightness, and the like on the digital image signal input from the imaging unit 101 based on setting values, and the memory 105 and a display control unit described later. 109 and transmitted to the control unit 111.

The voice input unit 103 includes a built-in microphone unit 114 built in the apparatus main body and an external voice input unit 116 that is a connection interface with the external microphone 115.
The built-in microphone unit 114 is built in the main body of the imaging apparatus 100, collects surrounding sounds, performs analog-digital conversion, and outputs them to the sound processing unit 104.
The external audio input unit 116 can be connected to an external microphone (external microphone) 115 via an audio input terminal, an accessory shoe having an electrical contact, a USB connector, and the like. The data is output to the processing unit 104. The external audio input unit 116 can accept input of audio from various external microphones, and can also accept audio input from a wireless microphone (wireless microphone) that acquires audio via wireless communication, for example. The external audio input unit 116 is also an external microphone information signal indicating whether or not the external microphone 115 is connected and whether or not the connected external microphone 115 acquires an audio signal via wireless communication. Is also output to the audio processing unit 104.

  The external microphone 115 may be any microphone as long as it is a microphone that collects ambient sound and outputs it as an electrical signal. For example, the external microphone 115 includes a microphone main body that acquires sound and transmits a sound signal by wireless communication, and a receiver (radio device) that receives the sound signal by wireless communication from the microphone main body and outputs the sound signal to the external sound input unit 116. It is a wireless microphone. The wireless communication method in the wireless microphone may be arbitrary, and for example, Bluetooth (registered trademark), wireless LAN, Wireless USB, or the like can be used. The external microphone 115 outputs information related to the type of the external microphone 115 (whether it is a wireless microphone or the like) to the external audio input unit 116 in addition to the collected audio signal.

The sound processing unit 104 performs sound-related processing such as processing for optimizing the level of the input digital sound signal and processing for reducing a specific frequency, and transmits the processing to the memory 105.
The memory 105 temporarily stores the image signal and the audio signal obtained by the image processing unit 102 and the audio processing unit 104.
The encoding processing unit 106 reads the image signal and the audio signal temporarily stored in the memory 105, performs the encoding of the image signal and the encoding of the audio signal, and generates compressed image data, compressed audio data, and the like. Then, the data is transmitted to the recording / reproducing unit 107.
The recording / reproducing unit 107 controls the recording by transferring the compressed image data, the compressed audio data, and other control data related to photographing generated by the encoding processing unit 106 to the recording medium 108. The recording / playback unit 107 performs recording control so that the compressed image data and the compressed audio data are associated with each other as a moving image file and recorded on the recording medium 108. The recording / playback unit 107 also performs playback control for reading compressed image data, compressed audio data, various data, and programs recorded on the recording medium 108.
The recording medium 108 is a recording medium that records and saves data transferred from the recording / reproducing unit 107. As long as compressed image data, compressed audio data, and the like can be recorded, the recording medium may be any type of recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, or may be a plurality of recording media.

The display control unit 109 displays an image signal transmitted from the encoding processing unit 106, an image related to the image signal transmitted from the image processing unit 102, an operation screen (menu screen) for operating the imaging device 100, and the like. Control to be displayed on the display unit 110 is performed.
The display unit 110 is a display unit that displays data from the display control unit 109. The display unit 110 is, for example, a display device such as a liquid crystal display, an organic EL display, or electronic paper. The operation unit 112 includes buttons, a dial, and the like, and transmits an instruction signal to the control unit 111 according to a user operation.
The operation unit 112 includes, for example, a power button, a recording start button, a menu display button, a determination button, a cursor key, a pointing device for designating an arbitrary point on the display unit 110, a touch panel, and the like.

  The control unit 111 controls each block by transmitting a control signal to each block of the imaging apparatus 100 based on the instruction signal transmitted from the operation unit 112. The control unit 111 includes, for example, a processor (CPU, MPU), a memory (DRAM, SRAM), and the like for executing various processes (programs). The bus 113 is a communication path that sends various data, control signals, and the like to each block of the imaging apparatus 100.

  The control unit 111, the image processing unit 102, the audio processing unit 104, the encoding processing unit 106, and the display control unit 109 may be realized in software by a computer having a processor, a memory, and the like executing a program. Further, these processing units may be realized by a dedicated hardware circuit. When these processing units are realized by software, one processor may provide the functions of a plurality of processing units, or the functions of one processing unit may be provided by a plurality of processors. These mounting methods may be combined arbitrarily.

[Overall processing]
Here, the overall operation at the time of moving image shooting of the imaging apparatus 100 according to the present embodiment will be described.
In the imaging apparatus 100 according to the present embodiment, when the user operates the power button of the operation unit 112, a start instruction is issued from the operation unit 112 to the control unit 111. Upon receiving this instruction, the control unit 111 controls a power supply unit (not shown) to supply power to each block of the imaging apparatus 100.
When the power is supplied, the control unit 111 confirms, for example, which mode the mode change switch of the operation unit 112 is in, for example, a shooting mode, a reproduction mode, or the like by an instruction signal from the operation unit 112. At the start of the shooting mode, the imaging apparatus 100 is in a shooting standby state. When the user operates the recording start button of the operation unit 112, shooting starts. During that time, compressed image data and compressed audio data are recorded on the recording medium 108. Is done. Then, when the user operates the photographing end button of the operation unit 112, the photographing is finished and the photographing standby state is entered again. In the reproduction mode, the compressed image data and the compressed audio data relating to the file selected by the user are reproduced from the recording medium 108 and the video is displayed on the display unit 110.

First, the shooting mode will be described. When the shooting mode is set, the shooting standby state is first set as described above. When a shooting start instruction signal is transmitted when the user operates the recording start button of the operation unit 112 in the shooting standby state, the control unit 111 transmits a shooting start control signal to each block of the imaging apparatus 100. Then, control is performed so as to perform the following operation.
The imaging unit 101 converts the optical image of the subject captured by the photographing lens into an image signal by the imaging element, performs analog-digital conversion, and transmits the image signal to the image processing unit 102. The image processing unit 102 processes the image quality adjustment processing (white balance, color, brightness, etc.) of the input digital image signal based on the set value, and transmits it to the display control unit 109. The display control unit 109 causes the display unit 110 to display a video related to the received image signal. Further, the image signal is also output to the memory 105 and temporarily stored in the memory 105.
The audio input unit 103 transmits a digital audio signal obtained from the built-in microphone unit 114 or the external microphone 115 to the audio processing unit 104. The sound processing unit 104 generates a sound signal by performing a process for optimizing the level of the input digital sound signal, a process for reducing a specific frequency, and the like, and temporarily stores the sound signal in the memory 105.
Then, the encoding processing unit 106 reads the image signal and audio signal temporarily stored in the memory 105 and performs predetermined encoding to generate compressed image data, compressed audio data, and the like. Then, the control unit 111 synthesizes the compressed image data and the compressed audio data, forms a data stream, and outputs the data stream to the recording / reproducing unit 107. The recording / playback unit 107 writes the data stream to the recording medium 108 as one moving image file under the management of a file system such as UDF or FAT.
The above operation is continued during shooting.

When the user operates the recording button of the operation unit 112 and an instruction signal to end shooting is transmitted to the control unit 111, the control unit 111 transmits a control signal to end shooting to each block of the imaging apparatus 100. Then, control is performed so as to perform the following operation.
The image processing unit 102 and the audio processing unit 104 stop transmitting the image signal and the audio signal to the memory 105, respectively. Then, the encoding processing unit 106 reads the remaining image signal and audio signal stored in the memory 105, performs predetermined encoding, generates compressed image data, compressed audio data, and the like. Stop operation.
The control unit 111 synthesizes these last compressed image data and compressed audio data, forms a data stream, and outputs the data stream to the recording / reproducing unit 107.
The recording / playback unit 107 writes the data stream to the recording medium 108 as one moving image file under the management of a file system such as UDF or FAT. When the supply of the data stream is stopped, the moving image file is completed and the recording operation is stopped.
When the recording operation stops, the control unit 111 transmits a control signal to each block of the imaging apparatus 100 so as to shift to the shooting standby state, and returns to the shooting standby state.
In the shooting standby state, the control unit 111 controls each block of the imaging apparatus 100 to perform the following operation. The image processing unit 102 transmits the image signal to the display control unit 109 and causes the display unit 110 to display a video related to the image signal. The user prepares for shooting while viewing the screen displayed in this way.

  Next, in the playback mode, the control unit 111 transmits a control signal to each block of the imaging apparatus 100 so as to shift to the playback state, and performs the following operation. The recording / playback unit 107 reads out a moving image file composed of compressed image data and compressed audio data recorded on the recording medium 108, and sends the read compressed image data and compressed audio data to the encoding processing unit 106. The encoding processing unit 106 temporarily stores the compressed image data and the compressed audio data in the memory 105, decodes them according to a predetermined procedure, and transmits the decoded image signal to the display control unit 109. The display control unit 109 causes the display unit 110 to display a video related to the input image signal.

[Audio mix processing]
Here, among the operations of the imaging apparatus 100 according to the present embodiment, the audio processing operation will be described in more detail. The imaging apparatus 100 according to the present embodiment can use an internal microphone and an external microphone, and when the external microphone is connected, the sound of the internal microphone and the external microphone is mixed and recorded.

  FIG. 2 is a diagram illustrating a control block related to the process of mixing the sounds of the built-in microphone and the external microphone in the sound processing unit 104 of FIG. That is, in FIG. 2, control blocks relating to the sound quality adjustment processing and the like are omitted.

  Based on the external microphone information signal S3 output from the external audio input unit 116, the wireless microphone connection detection unit 201 receives audio from the external microphone inserted into the external audio input unit 116 of the audio input unit 103 via wireless communication. It is determined whether or not it is to be acquired. That is, the wireless microphone connection detection unit 201 determines whether or not the audio signal S2 input from the external audio input unit 116 to the audio processing unit 104 is a signal from the wireless receiver of the wireless microphone.

The silence detection unit 202 detects whether or not the audio signal S2 input from the external audio input unit 116 (hereinafter referred to as an external audio signal) is silent. The silence detection unit 202 detects the sound signal S2 when the state where the input level of the sound signal S2 is equal to or lower than a predetermined threshold value continues for a predetermined period.
Is determined to be silent.

  Based on the detection result from the wireless microphone connection detection unit 201, the mix ratio determination unit 203 mixes the external audio signal with the built-in microphone signal according to the process shown in the flowchart of FIG. To decide. The mix ratio determination unit 203 sets the mix ratio of the built-in microphone signal to 1 when an external microphone is not connected, and the built-in microphone when an external microphone (wired microphone) that does not use wireless communication is connected. Set the signal mix ratio to zero. On the other hand, the mix ratio determination unit 203 determines the mix ratio based on the silence detection result of the silence detection unit 202 when notified from the wireless microphone connection detection unit 201 that an external microphone using wireless communication is connected. To enable.

  The mixing unit 204 includes an audio signal S2 from the external audio input unit 116 and an audio signal S1 ′ from the internal microphone unit 114 that is temporally synchronized with the audio signal S2 (hereinafter referred to as an internal microphone delay signal S1 ′). Is mixed at the above mix ratio. The mixed audio signal is sent to the recording / playback unit 107 via the bus, and is recorded on the recording medium 108 as a moving image file together with moving image data. The change-over switch 205 is a switch for switching whether to use the audio signal S1 input from the built-in microphone or the built-in microphone delay signal S1 'as the audio signal to be mixed by the mixing unit 204. Note that the audio signal input from the changeover switch 205 to the mixing unit 204 is also simply referred to as a built-in microphone audio signal regardless of the presence or absence of delay.

  FIG. 2 shows a flow of an example of a method for synchronizing the audio signal S2 from the external microphone and the audio signal S1 from the built-in microphone in terms of time. When the audio signal S1 of the built-in microphone unit 114 is faster than the audio signal S2 from the external microphone, the signal of the built-in microphone unit 114 is stored in the memory 105 via the bus 113 for the synchronization delay time, and then the built-in microphone delay signal Obtained from the memory 105 as S1 ′. Thereby, the time synchronization of the audio signal from the external microphone and the audio signal from the built-in microphone is achieved.

  The details of the audio mixing process in the audio processing unit 104 will be described below with reference to FIGS. FIG. 3 is a control flowchart of the audio mix process in the present embodiment. FIG. 4 is a graph showing the characteristics of the mix gain control operation of the mix ratio determination unit 203 in the present embodiment.

Hereinafter, the description will be made mainly with reference to the control flowchart of FIG.
In step S101, the signal processing is started. In step S102, it is confirmed whether or not an external microphone is connected to the external audio input unit 116 via a cable or the like. When the external microphone is not inserted (S102-NO), the changeover switch 205 selects the audio signal S1 (non-delay signal) from the built-in microphone unit 114 in step S115. In step S113, the mix ratio determination unit 203 sets the mix ratio of the built-in microphone signal to 1 (the mix ratio of the external audio signal is 0). That is, the mixing unit 204 outputs the audio signal S1 without delay from the built-in microphone unit 114 to the bus 113 as it is.

  If it is determined in step S102 that an external microphone is connected (S102-YES), the changeover switch 205 selects the built-in microphone delay signal S1 'in step S114. Next, in step S103, the wireless microphone connection detection unit 201 determines whether or not the external microphone connected to the external audio input unit 116 acquires audio via wireless communication. This determination is made based on the external microphone information signal S3 input from the external audio input unit 116 to the wireless microphone connection detection unit 201.

If the connection of the wireless microphone is not detected in step S103 (S103-N)
O) sets the mix ratio of the built-in microphone signal to 0 (the mix ratio of the external audio signal is 1) in step S112. That is, the mixing unit 204 outputs the external audio signal S2 input from the external audio input unit 116 to the bus 113 as it is.

If the connection of the wireless microphone is detected in step S103 (S103-YES), the silence detection unit 202 operates in step S104. Silence detection unit 202 has a level of audio signal S2 from external audio input unit 116 that is below a certain threshold value (threshold value ± J in FIG. 4A) for a period of time t (in FIG. 4C). It is determined that there is no sound during the G period.
When the silence detection unit 202 determines that there is silence in step S105 (S105-YES), the mix ratio determination unit 203 investigates the current mix ratio setting in step S106. In step S106, if the ratio of the built-in microphone signal is not 1 (less than 1) in the current mix ratio setting, the mix ratio determination unit 203 increases the mix ratio of the built-in microphone signal in step S107 (FIG. 4C ) A period). The increase amount of the mix ratio may be a predetermined fixed value, or may be a value determined based on the length of the silent period, the current mix ratio, or the like. When the ratio of the built-in microphone signal is 1, the ratio of the built-in microphone signal cannot be increased any more, so the mix ratio determination unit 203 keeps the ratio of the built-in microphone signal as 1 in step S108 (FIG. 4 (C) E period).
On the other hand, if it is determined in step S105 that there is no silence (S105-NO), the mix ratio determination unit 203 investigates the current mix ratio setting in step S109. In step S109, when the ratio of the built-in microphone signal is larger than 0 in the current mix ratio setting, the mix ratio determination unit 203 lowers the mix ratio of the built-in microphone signal in step S110 (period B in FIG. 4C). ). The reduction amount of the mix ratio may be a fixed value determined in advance, or may be a value determined based on the length of the silent period, the current mix ratio, or the like. If the ratio of the built-in microphone signal is 0, the ratio of the built-in microphone signal cannot be lowered any more, so the mix ratio determining unit 203 keeps the ratio of the built-in microphone signal at 0 in step S111 ( C period in FIG. 4C).

  The audio signal processed by the mixing unit 204 is sent to the recording / reproducing unit 107 via the bus, and is recorded on the recording medium 108 as a moving image file together with the moving image data.

[Advantageous effects of this embodiment]
According to the imaging apparatus 100 according to the present embodiment, when a wireless microphone is used as an external microphone, an audio signal from the built-in microphone can be recorded when wireless communication is disconnected. Therefore, it is possible to avoid silence being recorded even when wireless communication of the wireless microphone is interrupted. Since the ratio of the internal microphone signal is gradually increased according to the continuation of the silent period, it is possible to suppress a sense of incongruity associated with sudden switching of sound.

[Modification]
In the above description, the signal processing is performed on the audio signal after the digital conversion, but the processing may be performed on the analog signal without performing the digital conversion.

  In the above description, the external microphone information signal is output from the external audio input unit 116 to the audio processing unit 104, and it is determined whether or not the external microphone is a wireless microphone based on the external microphone information signal. . However, whether or not the external microphone is a wireless microphone may be input by the user via the operation unit 112, or the user may set the operation unit 112 to enable / disable the mixing process of the external microphone and the built-in microphone based on silence detection. You may input via. Further, the mixing process with the built-in microphone sound may be performed without performing the process of determining whether or not the external microphone is a wireless microphone (step S103 in the flowchart of FIG. 3).

In the above description, it is assumed that a wireless microphone radio receiver is connected to the external audio input unit 116. However, the external audio input unit 116 itself may be a wireless device that acquires audio through wireless communication. For example, the external audio input unit 116 may have a wireless communication function compliant with the Bluetooth standard, and may receive an audio signal transmitted wirelessly from an external microphone. The external audio input unit 116 acquires the audio signal from the external microphone via wireless communication until the external microphone converts the sound into an electric signal until the electric signal is input to the external audio input unit 116. It means that a part of the communication path is wireless communication.

  In the present embodiment, the imaging apparatus (digital video camera) has been described as an example. However, the application of the audio processing apparatus according to the present invention is not limited to the imaging apparatus. The sound processing device according to the present invention has at least the sound input unit 103 (the built-in microphone unit 114 and the external sound input unit 116) and the sound processing unit 104 described above, and collects sound. Other configurations are arbitrary. That is, the voice processing apparatus according to the present invention can be implemented as, for example, a voice recorder, a mobile phone, a smartphone, a personal computer (tablet type, notebook type, desktop type) or the like.

<Second Embodiment>
[Constitution]
The imaging apparatus 100 according to the present embodiment will be described with reference to FIGS. 5, 6, 7, and 8. Constituent elements common to the first embodiment are denoted by the same reference numerals, description thereof is omitted, and differences from the first embodiment are mainly described.

  FIG. 5 is a block diagram illustrating an overall configuration of the imaging apparatus 500 according to the present embodiment. In the present embodiment, a wireless receiver 301 is provided instead of the external audio input unit 116 in the first embodiment. The wireless receiver 301 receives audio data from an external microphone (wireless microphone) through radio waves through wireless communication such as Bluetooth, wireless LAN, and Wireless USB, and receives the received audio data and data representing the received radio wave status as the audio processing unit 104. To pass.

[processing]
In the present embodiment, the configuration of the audio processing unit is different from that of the first embodiment. FIG. 6 shows a control block related to the mix processing of the built-in microphone and the external microphone in the audio processing unit 504 in the present embodiment.

In FIG. 6, a reception BER determination unit 302 is a functional unit that acquires reception bit error rate (BER) information from data of the wireless receiver 301. The reception sensitivity detection unit 303 is a functional unit that acquires reception sensitivity information from the data of the wireless receiver 301. The interference wave detection unit 304 is a functional unit that receives external radio waves that may deteriorate the reception quality of the radio system received by the radio receiver 301 and acquires sensitivity data thereof. The SNR determination unit 305 has a function of calculating an SNR (Signal to Noise Ratio) with the data of the reception sensitivity detection unit 303 as S (signal) and the data of the interference wave detection unit 304 as N (noise). Part.

  The mix ratio K calculation unit 306 is a functional unit that calculates the first provisional mix ratio (referred to as provisional mix ratio K) of the built-in microphone signal based on the data of the reception BER determination unit 302. The provisional mix ratio K calculation unit 306 calculates the provisional mix ratio K so that the provisional mix ratio K increases (increases monotonously in a broad sense) as the reception BER increases. The mix ratio K calculation unit 306 calculates the provisional mix ratio K from the reception BER according to a broad monotone increasing function as in the example of FIG. 8A, for example. That is, if the reception BER is equal to or less than the threshold value R1, the temporary mix ratio K is set to 0. While the reception BER is equal to or greater than the threshold value R1 and equal to or less than the threshold value R2, the temporary mix ratio K is set to a value between 0 and 1. Is equal to or greater than the threshold value R2, the provisional mix ratio K is 1.

  The mix ratio M calculation unit 307 is a functional unit that calculates a second provisional mix ratio (referred to as provisional mix ratio M) of the built-in microphone signal based on the data of the SNR determination unit 305. The mix ratio M calculation unit 307 calculates the temporary mix ratio M such that the higher the SNR, the smaller the temporary mix ratio M (in a broad sense monotone decrease). The mix ratio M calculation unit 307 calculates the temporary mix ratio M according to a broad monotone decreasing function as in the example of FIG. 8B, for example. That is, if the SNR is equal to or less than the threshold value R3, the temporary mix ratio M is set to 1, and the temporary mix ratio M takes a value between 1 and 0 while the SNR is equal to or greater than the threshold value R3 and equal to or less than the threshold value R4. In the above, the provisional mix ratio M is set to zero.

  Note that either or both of the mix ratio K calculation unit 306 and the mix ratio M calculation unit 307 calculate the provisional mix ratio from the received BER or SNR by referring to the lookup table instead of calculating the provisional mix ratio based on the function. May be determined.

  The mix ratio determination unit 308 determines a mix ratio when the mixing unit 204 mixes the external audio signal and the built-in microphone signal. In the present embodiment, the mix ratio determination unit 308 has a larger temporary mix ratio between the provisional mix ratio K obtained by the mix ratio K computation unit 306 and the provisional mix ratio M obtained by the mix ratio M computation unit 307. Select. Since the mix ratio determined by the mix ratio determination unit 308 is the mix ratio of the built-in microphone signal, the mix ratio determination unit 308 selects the higher one of the mix ratio of the built-in microphone signal from the provisional mix ratio K and the provisional mix ratio M. It can also be expressed as.

  Hereinafter, the details of the audio mixing process in the audio processing unit 504 will be described with reference to FIGS. FIG. 7 is a control flowchart of the audio mix processing in the present embodiment. FIG. 8 is a diagram illustrating a calculation example of the provisional mix ratio of the mix ratio K calculation unit 306 and the mix ratio M calculation unit 307 in the present embodiment.

Hereinafter, description will be made mainly with reference to the control flowchart of FIG.
In step S201, this signal processing is started. In step S202, it is determined whether voice wireless communication has been established. If wireless communication is not established (S202-NO), the changeover switch 205 selects the audio signal S1 (non-delay signal) from the built-in microphone unit 114 in step S212. In step S210, mix ratio determination unit 308 sets the ratio of the built-in microphone signal to 1 (the mix ratio of the external audio signal is 0). That is, the mix unit 204 outputs the audio signal S1 of the built-in microphone unit 114 to the bus 113 as it is.

If voice wireless communication is established in step S202 (S202-YES), the changeover switch 205 selects the built-in microphone delay signal S1 ′ in step S211. Next, the process proceeds to step S <b> 203, and the reception BER determination unit 302 acquires reception bit error rate information from the data of the wireless receiver 301. In step S204, the mix ratio K calculation unit 306 calculates a temporary mix ratio K as shown in the graph of FIG. 8A based on the received bit error information. In step S205, the SNR determination unit 305 detects the SNR. Specifically, the SNR value is calculated from the reception sensitivity information (S) acquired by the reception sensitivity detection unit 303 from the data of the wireless receiver 301 and the electric field strength (N) of the external radio wave acquired by the interference wave detection unit 304. . In step S206, the mix ratio M calculation unit 307 calculates the temporary mix ratio M as shown in the graph of FIG. 8B based on the SNR value. In step S207, the mix ratio determination unit 308 determines the magnitude of M and K. If M is large (S207-YES), the process proceeds to step S208, where the mix ratio is determined as M, and the mix ratio is transferred to the mix unit 204. Transmit information. When M is small (S207-NO), the process proceeds to step S209, where the mix ratio is determined as K, and the mix ratio information is transmitted to the mix unit 204. The mixing unit 204 mixes the audio data (external audio) of the radio and the audio data of the built-in microphone signal according to the mix ratio information determined in this way.

[Advantageous effects of this embodiment]
According to the present embodiment, the mix ratio between the external microphone and the built-in microphone can be changed according to the reception status of wireless communication by the external microphone. When the reception status of wireless communication is poor, the wireless communication of audio by the external microphone may be interrupted. According to the present embodiment, by increasing the mix ratio of the internal microphone in such a situation, silence is achieved. Can be prevented from being recorded.

[Modification]
In the above description, the radio receiver 301 outputs the audio signal from the external microphone and the reception status information of the radio signal to the audio processing unit 504. However, the radio receiver 301 receives the audio signal of the external microphone, and the reception status. Different radios may be used. The same effect can be obtained by such a configuration. For example, if the radio that transmits the recorded video file to the outside is provided in the imaging device separately from the radio receiver of the audio signal from the external microphone, the wireless communication reception status is acquired from this radio Also good.
In addition, when different wireless devices are used for voice reception and reception status acquisition in this way, the wireless communication methods of the two wireless devices are preferably the same, but different wireless communication methods may be used. . For example, since both the wireless LAN and the Bluetooth use the 2.4 GHz band, the reception status information may be acquired by the wireless LAN wireless device while performing wireless communication of the audio signal by the Bluetooth wireless device. This is because when the reception status by the wireless LAN is bad, it can be assumed that the reception status of the wireless communication by Bluetooth is also bad.

  In the present embodiment, an external microphone (wireless microphone) receiver is built in the imaging apparatus main body. However, as in the first embodiment, the wireless microphone receiver is an accessory shoe, a USB connector, a voice input terminal, or the like. May be connected to the imaging device. In other words, the wireless device built in the imaging apparatus is used to notify the sound processing unit of the reception status of wireless communication, and may not be used to receive the sound signal. Since the wireless communication quality between the transmitter and the receiver of the wireless microphone is known from the reception status information by the wireless device, the same effect as described above can be obtained.

  The above description is an example of mix ratio determination based on the reception status of wireless communication, and various other methods for determining the mix ratio based on the reception status can be considered. For example, it is conceivable to use an average (such as a simple average or a weighted average) of the temporary mix ratio K and the temporary mix ratio M. Alternatively, the final mix ratio may be directly determined based on a function or look-up table using both BER and SNR as variables. Further, the mix ratio may be determined based on only one of BER and SNR, or the mix ratio may be determined using a parameter representing a reception situation other than BER and SNR. In any case, the mix ratio of the built-in microphone may be increased when the wireless communication reception status deteriorates.

  This embodiment may be implemented in combination with the first embodiment. That is, the mix ratio may be determined based on both the input level of the external microphone and the reception status of wireless communication. Specifically, for example, the mix ratio is determined based on the length of the silence period, the mix ratio is determined based on the reception status information, and the final mix ratio is determined based on these mix ratios. It is possible. Among the two mix ratios, it is conceivable to use a mix ratio with a high mix ratio of the built-in microphone signal, or to use an average (such as a simple average or a weighted average) of these two mix ratios.

  In the present embodiment, the imaging apparatus (digital video camera) has been described as an example. However, the application of the audio processing apparatus according to the present invention is not limited to the imaging apparatus. The sound processing apparatus according to the present invention includes at least the sound input unit 103 (the built-in microphone unit 114 and the wireless receiver 301) and the sound processing unit 504 described above, and any other device that collects sound. The configuration of is arbitrary. That is, the voice processing apparatus according to the present invention can be implemented as, for example, a voice recorder, a mobile phone, a smartphone, a personal computer (tablet type, notebook type, desktop type) or the like.

<Other embodiments>
The present invention can also be implemented by a computer (or a device such as a CPU or MPU) of a system or apparatus that implements the functions of the above-described embodiments by reading and executing a program recorded in a storage device. For example, the present invention can be implemented by a method including steps executed by a computer of a system or apparatus that implements the functions of the above-described embodiments by reading and executing a program recorded in a storage device. . For this purpose, the program is stored in the computer from, for example, various types of recording media that can serve as the storage device (ie, computer-readable recording media that holds data non-temporarily). Provided to. Therefore, the computer (including devices such as CPU and MPU), the method, the program (including program code and program product), and the computer-readable recording medium that holds the program non-temporarily are all present. It is included in the category of the invention.

DESCRIPTION OF SYMBOLS 100 Imaging device 114 Built-in microphone part 116 External audio | voice input part 202 Silence detection part 203 Mix ratio determination part 204 Mix part

Claims (18)

A microphone provided in the apparatus body;
Voice input means capable of accepting voice input via wireless communication;
Silence detection means for detecting that the input level of the voice input means is silence; and
A mix ratio determining means for determining a mix ratio for mixing the sound of the sound input means and the sound of the microphone based on the detection result by the silence detecting means;
Mixing means for mixing the sound of the sound input means and the sound of the microphone at a mix ratio determined by the mix ratio determination means;
An audio processing apparatus comprising: The silence detection means determines that there is no sound when a state where the input level of the voice input means is a predetermined threshold value or less continues for a predetermined period.
The speech processing apparatus according to claim 1. The mix ratio determining means determines the mix ratio according to a length of a period during which the voice input means is silent;
The speech processing apparatus according to claim 1 or 2. The mix ratio determining means increases the mix ratio of the microphone as the period during which the audio input means is silent is long.
The speech processing apparatus according to claim 3. A recording means for recording the sound mixed by the mixing means;
The speech processing apparatus according to any one of claims 1 to 4. The voice input means can be connected to the receiver of an external microphone composed of a microphone main body that acquires and wirelessly transmits voice and a receiver that receives a voice signal transmitted from the microphone main body,
Further comprising detection means for detecting that the receiver is connected to the voice input means;
When the connection of the receiver is detected by the detection unit, the determination of the mix ratio based on the detection result of the silence detection unit is enabled.
The speech processing apparatus according to any one of claims 1 to 5. The voice input means has wireless communication means for receiving a voice signal by wireless communication,
When the reception of the audio signal by the wireless communication is valid, the determination of the mix ratio based on the detection result of the silence detection unit is enabled.
The speech processing apparatus according to any one of claims 1 to 5. A microphone provided in the apparatus body;
Voice input means capable of accepting voice input via wireless communication;
Reception status acquisition means for acquiring the reception status of the wireless communication;
Based on the reception status acquired by the reception status acquisition means, a mix ratio determination means for determining a mix ratio for mixing the voice of the voice input means and the voice of the microphone;
Mixing means for mixing the sound of the sound input means and the sound of the microphone at a mix ratio determined by the mix ratio determination means;
An audio processing apparatus comprising: The mix ratio determining means increases the mix ratio of the microphone as the reception condition is worse.
The speech processing apparatus according to claim 8. The voice input means has wireless communication means for receiving a voice signal by wireless communication,
The reception status acquisition means acquires the reception status of the wireless communication means;
The speech processing apparatus according to claim 8 or 9. The reception status acquisition means acquires a bit error rate,
The mix ratio determining means increases the mix ratio of the microphone as the bit error rate is higher.
The speech processing apparatus according to claim 10. The reception status acquisition means acquires an SNR,
The mix ratio determining means increases the mix ratio of the microphone as the SNR is lower;
The speech processing apparatus according to claim 10 or 11. The reception status acquisition means acquires a bit error rate and an SNR,
The mix ratio determining means calculates a first provisional mix ratio with a higher mix ratio of the microphone as the bit error rate is higher, and calculates a second provisional mix ratio with a higher mix ratio of the microphone as the SNR is lower. , One of the first provisional mix ratio and the second provisional mix ratio, which has a larger mix ratio of the microphone, is determined as the mix ratio;
The speech processing apparatus according to claim 10. A recording means for recording the sound mixed by the mixing means;
The speech processing apparatus according to any one of claims 8 to 13. Imaging means for capturing a moving image;
The voice processing device according to any one of claims 1 to 14,
A recording unit that records the moving image captured by the imaging unit and the audio mixed by the mixing unit of the audio processing apparatus in association with each other;
An imaging apparatus comprising: A method for controlling a voice processing apparatus comprising: a microphone provided in the apparatus main body; and voice input means capable of receiving voice input via wireless communication,
A silence detection step for detecting that the input level of the voice acquired from the voice input means is silent; and
A mix ratio determining step for determining a mix ratio for mixing the sound of the sound input means and the sound of the microphone based on the detection result in the silence detecting step;
A mix step of mixing the sound of the sound input means and the sound of the microphone at the mix ratio determined in the mix ratio determination step;
A method for controlling a speech processing apparatus, comprising: A method for controlling a voice processing apparatus comprising: a microphone provided in the apparatus main body; and voice input means capable of receiving voice input via wireless communication,
A reception status acquisition step of acquiring the reception status of the wireless communication;
Based on the reception status acquired in the reception status acquisition step, a mix ratio determination step for determining a mix ratio for mixing the voice of the voice input means and the voice of the microphone;
A mix step of mixing the sound of the sound input means and the sound of the microphone at the mix ratio determined in the mix ratio determination step;
A method for controlling a speech processing apparatus, comprising:   The program for making a computer perform each step of the method of Claim 16 or 17.

Images