JP2012235379A - Voice multiplexing device, voice hearing device and voice multiplexing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice multiplexing device capable of enhancing the level of easy-to-hear of a marked sound while reducing processing load.SOLUTION: The voice multiplexing device includes a voice input section 110 that inputs a first voice signal and a second voice signal; a first voice multiplexing section 120 that generates a first multiplexed voice signal which is obtained by multiplexing the first voice signal and the second voice signal in a first multiplexing positional relationship; a second voice multiplexing section 130 that generates a second multiplexed voice signal which is obtained by multiplexing the first voice signal and the second voice signal in a second multiplexing positional relationship different from the first multiplexing positional relationship; and a voice output section 140 that outputs the first multiplexed voice signal and the second multiplexed voice signal.

Description

  The present invention relates to an audio multiplexing apparatus, an audio listening apparatus, and an audio multiplexing method that multiplex and output a plurality of audio signals.

  Increasingly, a plurality of sounds are output simultaneously, such as voice communication such as voice chat and telephone conference, and radio broadcasting that multiplexes and distributes a plurality of voice data.

  However, for the listener, when a desired sound (hereinafter referred to as “noticeable sound”) is selected from a plurality of sounds and listened to, sounds other than the notable sound are felt as noise.

  Therefore, for example, Patent Document 1 describes a technique of creating a filter based on a state of each audio signal when there is no sound of interest, and extracting only the sound of interest using the created filter. Further, for example, Patent Document 2 describes a technique for performing audio processing so that an audio signal of a sound other than the target sound (hereinafter, “non-target sound”) can be heard as an environmental sound. In these conventional techniques, it is possible to reproduce a plurality of sounds while reproducing only the sound of interest so that it can be easily heard. As a known technique, there is known a technique in which a plurality of sounds are multiplexed and transmitted / received by object coding that individually encodes sounds, and a user individually controls the sounds.

JP-A-5-87619 JP-A-8-186648

  However, the above-described conventional technique has a problem that a processing load increases due to a system in which a plurality of audio signals are multiplexed and transmitted / received and superimposed and reproduced. That is, in the technique described in Patent Document 1, when the non-target sound is a non-continuous sound such as an utterance voice, it is necessary to repeatedly create a filter according to the change of the non-target sound, which increases the processing load. . In the techniques described in Patent Document 1 and Patent Document 2, when an audio signal is sent from an unspecified number of people in voice chat or the like, the processing load increases according to the number. In the object coding technique, audio signals individually compressed with data are multiplexed and transmitted / received. However, the processing load for individually controlling audio increases depending on the number of audio.

  A technique for superimposing and outputting a plurality of sounds is expected to be applied to various fields. However, if the processing load is high, it is difficult to apply to a small portable terminal such as a mobile phone. Therefore, it is desired that such a technique can make it easy to hear the target sound while suppressing the processing load.

  An object of the present invention is to provide an audio multiplexing device, an audio listening device, and an audio multiplexing method that make it easy to hear a target sound with a reduced processing load.

  An audio multiplexing apparatus according to the present invention includes an audio input unit that inputs a first audio signal and a second audio signal, and a first multiplexing positional relationship between the first audio signal and the second audio signal. What is the first multiplexing positional relationship between the first audio multiplexing unit that generates the first multiplexed audio signal obtained by multiplexing in step 1 and the first audio signal and the second audio signal? A second audio multiplexing unit that generates a second multiplexed audio signal obtained by multiplexing with different second multiplexing positional relationships, and outputs the first multiplexed audio signal and the second multiplexed audio signal And an audio output unit.

  According to another aspect of the present invention, there is provided a sound listening apparatus, a multiple sound receiver that obtains the first multiple sound signal and the second multiple sound signal from the sound multiplexer, the first multiple sound signal, and the second multiple sound signal. And a time adjustment unit for generating a superimposed audio signal obtained by superimposing the multiple audio signal with a predetermined superimposable positional relationship, and based on a user operation, the first multiplexed audio signal and the second A first superimposed positional relationship in which arbitrary positions of the first audio signal included in each of the multiplexed audio signals match, and the first multiplexed audio signal and the second multiplexed audio signal included in each of the first audio signals; An operation unit that switches the predetermined superposition position relationship with a second superposition position relationship in which an arbitrary position of the second sound signal matches, and a sound output unit that outputs the superposition sound signal to a sound output device; Have

  The audio multiplexing method of the present invention includes a step of inputting a first audio signal and a second audio signal, and multiplexing the first audio signal and the second audio signal in a first multiplexing positional relationship. And generating a first multiplexed audio signal, and multiplexing the first audio signal and the second audio signal with a second multiplexing positional relationship different from the first multiplexing positional relationship. And generating a second multiplexed audio signal, and outputting the first multiplexed audio signal and the second multiplexed audio signal.

  According to the present invention, it is possible to make it easy to hear the target sound while suppressing the processing load.

1 is a block diagram showing an example of the configuration of a speech multiplexing apparatus according to Embodiment 1 of the present invention. FIG. 3 is a block diagram showing an example of the configuration of a speech multiplexing device, speech listening device, and speech multiplexing system according to Embodiment 2 of the present invention. The figure which shows typically an example of a structure of the input audio | voice signal in Embodiment 2 of this invention. The figure which shows typically an example of a structure of the 1st and 2nd multiplexed audio | voice signal in Embodiment 2 of this invention. The figure which shows typically the example of a structure of the superimposition audio | voice signal in Embodiment 2 of this invention. The flowchart which shows an example of operation | movement of the audio | voice multiplexing apparatus which concerns on Embodiment 2 of this invention. The flowchart which shows an example of operation | movement of the audio | voice listening apparatus which concerns on Embodiment 2 of this invention. FIG. 3 is a block diagram showing an example of the configuration of a speech multiplexing device, speech listening device, and speech multiplexing system according to Embodiment 3 of the present invention. The figure which shows typically an example of a structure of the 1st and 2nd multiplexed audio | voice signal in Embodiment 3 of this invention. The figure which shows typically the example of a structure of the superimposition audio | voice signal in Embodiment 3 of this invention. FIG. 7 is a block diagram showing an example of the configuration of a speech multiplexing device, speech listening device, and speech multiplexing system according to Embodiment 4 of the present invention. The flowchart which shows an example of operation | movement of the audio | voice multiplexing apparatus which concerns on Embodiment 4 of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  In each embodiment, the multiplexing of a plurality of audio signals and the superimposition of a plurality of multiplexed audio signals are relative relationships of positions on the time axis of the respective audio signals (hereinafter simply referred to as “positions”). To set at least. The relative relationship set in multiplexing is called “multiplexed positional relationship”, and the relative relationship set in superimposing is called “superimposed positional relationship”.

(Embodiment 1)
Embodiment 1 of the present invention is an example of a basic aspect of a speech multiplexing apparatus according to the present invention.

  FIG. 1 is a block diagram showing an example of the configuration of the speech multiplexing apparatus according to the present embodiment.

  In FIG. 1, the speech multiplexing apparatus 100 includes a speech input unit 110, a first speech multiplexing unit 120, a second speech multiplexing unit 130, and a speech output unit 140.

  The voice input unit 110 inputs the first voice signal and the second voice signal.

  The first audio multiplexing unit 120 generates a first multiplexed audio signal obtained by multiplexing the first audio signal and the second audio signal with the first multiplexing positional relationship.

  The second audio multiplexing unit 130 obtains a second audio signal obtained by multiplexing the first audio signal and the second audio signal with a second multiplexing positional relationship different from the first multiplexing positional relationship. A multiple audio signal is generated.

  The audio output unit 140 outputs the first multiplexed audio signal and the second multiplexed audio signal.

  The audio multiplexing apparatus 100 includes, for example, a storage medium such as a central processing unit (CPU) and a random access memory (RAM). In this case, each functional unit described above is realized by the control program being executed by the CPU.

  Such an audio multiplexing apparatus 100 can output two types of multiplexed audio signals having different multiplexing positions of the second audio signal with respect to the first audio signal.

  Such two types of multiplexed audio signals can be superimposed in a state where only the first audio signal or only the second audio signal is selectively matched. The voice of the matched voice signal is clearer and easier to hear than the voice of the mismatched voice signal. That is, the two types of multiplexed audio signals can easily make the first audio signal and the second audio signal easy to hear by simply adjusting the relative positional relationship when superimposed.

  Therefore, the speech multiplexing apparatus 100 can make it easy to hear the target sound with a reduced processing load.

(Embodiment 2)
Embodiment 2 of the present invention is an example of a specific aspect when the present invention is applied to a voice chat system in which an unspecified number of speakers talk about a plurality of topics at the same time.

  First, the configuration of each device and system according to the present embodiment will be described.

  FIG. 2 is a block diagram showing an example of the configuration of the audio multiplexing device and the audio listening device and the configuration of the audio multiplexing system according to the present embodiment.

  In FIG. 2, the audio multiplexing system 200 includes first to fourth audio providing devices 300-1 to 300-4, an audio multiplexing device 100, an audio listening device 400, and an audio output device 500.

  Note that the plurality of audio providing devices 300 and the audio multiplexing device 100, the audio multiplexing device 100 and the audio listening device 400, and the audio listening device 400 and the audio output device 500 are connected so as to be communicable by radio or wire. Shall. The transmission band from the audio multiplexing apparatus 100 to the audio listening apparatus 400 includes a first channel and a second channel that can separate two multiplexed audio signals. The two multiplexed audio signal transmission methods may have separate lines, or may be a time division multiplexing system or a frequency division multiplexing system.

  The first to fourth voice providing apparatuses 300-1 to 300-4 have the same configuration, and will be collectively described as “voice providing apparatus 300” as appropriate.

  Furthermore, the first to fourth voice providing devices 300-1 to 300-4 and the voice listening device 400 can have the same configuration, but here, according to the distinction between the voice supply side and the listening side, These will be described separately.

  The voice providing device 300 is, for example, an information communication terminal carried by a user who performs voice chat. The voice providing apparatus 300 includes a microphone, inputs voice including user's uttered voice, converts the voice into an audio signal that is an electric signal, and transmits the voice signal to the voice multiplexing apparatus 100. In this embodiment, the audio signal is digital.

  The voice multiplexing device 100 is, for example, a voice chat server. The audio multiplexing apparatus 100 receives four types of audio signals (hereinafter referred to as “first to fourth audio signals” in order) sent from the first to fourth audio providing apparatuses 300-1 to 300-4. Receive. The audio multiplexing apparatus 100 multiplexes the received first to fourth audio signals into two channels (two types) of multiplexed audio signals having different multiplexing methods, and transmits the multiplexed audio signals to the audio listening apparatus 400. Hereinafter, a group of the first to fourth audio signals at the time of reception is hereinafter referred to as an “input audio signal”.

  FIG. 3 is a diagram schematically showing an example of the configuration of the input audio signal assumed in the present embodiment.

  As shown in FIG. 3, the input audio signal 610 is assumed to be composed of first to fourth audio signals 611 to 614 here. Assume that the positions of the first to fourth audio signals 611 to 614 at an arbitrary time t0 on the input audio signal 610 are Vta, Vtb, Vtc, and Vtd in this order.

  The audio multiplexing apparatus 100 in FIG. 2 includes an audio input unit 110, a first audio multiplexing unit 120, a second audio multiplexing unit 130, and a multiplexed audio transmission unit 141.

  The voice input unit 110 inputs the first to fourth voice signals.

  Specifically, the voice input unit 110 receives the first to fourth voice signals transmitted from the first to fourth voice providing apparatuses 300-1 to 300-4, and the voice compression unit 111 The amplitudes of the first to fourth audio signals are respectively compressed. Then, the audio input unit 110 converts the compressed first to fourth audio signals (hereinafter simply referred to as “first to fourth audio signals”) into the first audio multiplexing unit 120 and the second audio multiplexing. Output to the unit 130.

  At this time, the audio input unit 110 matches the positions of the second to fourth audio signals received at the same timing as the position with respect to an arbitrary position of the first audio signal. 1st to 4th audio signals are output.

  Note that compression in the present embodiment refers to the number of audio signals 300 according to the number of audio providing apparatuses 300 that simultaneously transmit audio signals (4 in this embodiment, hereinafter referred to as “the number of audio signals”). This includes reducing the amplitude (sound pressure level). For example, compression is performed by dividing the amplitude of each audio signal by the number of audio signals.

  In addition, the compression may include reducing or increasing the amplitude of the audio signal having a low volume so that the maximum value of the amplitude of each audio signal matches a predetermined upper limit value.

  The first audio multiplexing unit 120 generates a first multiplexed audio signal obtained by multiplexing the first to fourth audio signals according to the first multiplexing positional relationship.

  Specifically, the first audio multiplexing unit 120 transmits the input first to fourth audio signals directly as multiplexed transmission targets on the first channel in accordance with the input timing. Output to the unit 141.

  That is, in the above-described first multiplexing positional relationship, the relative positions of the first to fourth audio signals received (input) do not change, and transmission (output) has the same timing as reception (input). It becomes.

  The second audio multiplexing unit 130 multiplexes the second multiplexed audio signal obtained by multiplexing the first to fourth audio signals with a second multiplexing positional relationship different from the first multiplexing positional relationship. Generate.

  That is, in the above-described second multiplexing position relationship, the relative position of the first to the first audio signals received (input) changes, and the timing of transmission (output) is different from that of reception (input).

  Specifically, the second audio multiplexing unit 130 causes the delay processing unit 131 to delay the second to fourth audio signals by different predetermined times. The delay processing unit 131 is, for example, a digital delay that is output after storing an audio signal for an arbitrary time. The second audio multiplexing unit 130 then transmits the first audio signal and the delayed second to fourth audio signals along the delayed timing on the second channel. Is output to the multiplexed audio transmission unit 141.

  In other words, the second multiplexing positional relationship described above is such that the positions of the second to fourth audio signals received (input) at the same timing as the position with respect to an arbitrary position of the first audio signal. The relationship is delayed by a predetermined time corresponding to each.

  FIG. 4 is a diagram schematically showing an example of the configuration of the first and second multiplexed audio signals in the present embodiment, and corresponds to FIG. FIG. 4A shows the configuration of the first multiplexed audio signal. FIG. 4B shows the configuration of the second multiplexed audio signal.

  As shown in FIG. 4A, the relative positional relationship (first multiplexed positional relationship) of the first to fourth audio signals 611 to 614 in the first multiplexed audio signal 620 is shown in FIG. The relative positional relationship in the input audio signal 610 is almost the same.

  That is, the above-described positions Vtb, Vtc, and Vtd of the second to fourth audio signals 612 to 614 coincide with the time t1 corresponding to the position Vta of the first audio signal 311.

  On the other hand, as shown in FIG. 4B, the relative positional relationship (second multiplexed positional relationship) of the first to fourth audio signals 611 to 614 in the second multiplexed audio signal 630 is as shown in FIG. This is different from the relative positional relationship in the input audio signal 610 shown in FIG.

  That is, the above-described positions Vtb, Vtc, and Vtd of the second to fourth audio signals 612 to 614 are delayed with respect to the time t1 corresponding to the position Vta of the first audio signal 311.

  Note that the delay times d1 to d3 of the second to fourth audio signals 612 to 614 are different from each other. The delay time d3 is longer than the delay time d2, and the delay time d2 is longer than the delay time d1. Also, it is assumed that the information indicating the delay times d1 to d3 can be acquired by the audio multiplexing apparatus 100 by being added to the second multiplexed audio signal.

  2 outputs a first multiplexed audio signal and a second multiplexed audio signal.

  Specifically, the multiplex sound transmission unit 141 transmits the input first multiplex sound signal and second multiplex sound signal to the sound listening apparatus 400 using the first channel and the second channel, respectively. To do.

  The voice listening device 400 is, for example, a personal computer (voice chat client) used by a user who performs voice chat.

  The audio listening device 400 includes a multiple audio receiving unit 410, a time adjusting unit 420, an operation unit 430, and an audio output unit 440.

  Multiplex audio receiving section 410 acquires the first multiplexed audio signal and the second multiplexed audio signal from audio multiplexing apparatus 100.

  Specifically, the multiplex audio receiver 410 receives the first multiplex audio signal and the second multiplex audio signal transmitted from the audio multiplexer 100 using the above-described two channels. Multiplex sound receiving section 410 then outputs the received first multiple sound signal and second multiple sound signal to time adjustment section 420.

  The time adjustment unit 420 generates a superimposed audio signal obtained by superimposing the first multiplexed audio signal and the second multiplexed audio signal with a predetermined superposition position relationship that can be adjusted.

  Specifically, the time adjustment unit 420 is controlled by the operation unit 430 to delay one of the first multiplexed audio signal and the second multiplexed audio signal. Thereby, the time adjustment unit 420 adjusts the relative positional relationship between the first multiplexed audio signal and the second multiplexed audio signal. Then, the time adjustment unit 420 generates a superimposed audio signal by superimposing the first multiplexed audio signal and the second multiplexed audio signal with the adjusted relative positional relationship, and outputs the superimposed audio signal to the audio output unit 440.

  Based on the user operation, the operation unit 430 performs at least the above-described predetermined superposition between the first superposition position relationship, the second superposition position relationship, the third superposition position relationship, and the fourth superposition position relationship. Switch the positional relationship.

  Specifically, the operation unit 430 has an operation interface that can obtain input values in the positive direction and the negative direction, such as a dial and a slider. Then, the operation unit 430 causes an arbitrary position of the audio signal designated as a target to make sound easy to hear according to the input value so as to match between the first multiplexed audio signal and the second multiplexed audio signal. Then, the predetermined superposition position relationship is switched.

  The first superimposed positional relationship is a superimposed positional relationship in which arbitrary positions of the first audio signal included in the first multiplexed audio signal and the second multiplexed audio signal match.

  The second superposition position relationship is a superposition position relationship in which arbitrary positions of the second audio signal included in the first multiplexed audio signal and the second multiplexed audio signal match.

  The third superimposed positional relationship is a superimposed positional relationship in which arbitrary positions of the third audio signal included in the first multiplexed audio signal and the second multiplexed audio signal match.

  The fourth superimposed positional relationship is a superimposed positional relationship in which arbitrary positions of the fourth audio signal included in the first multiplexed audio signal and the second multiplexed audio signal match.

  It is assumed that the first to fourth audio signals are assigned audio numbers 1 to 4 in order. Then, operation unit 430 switches the superposition position relationship between the first multiplexed audio signal and the second multiplexed audio signal so as to move the pointer that designates the audio number according to the input value.

  In addition, the first to fourth superposition position relationships are such that, for example, the time adjustment unit 420 acquires and holds information indicating the delay times d1 to d3 from the audio multiplexing apparatus 100, and the delay times d1 to d3 are set. It shall be set based on this.

  FIG. 5 is a diagram schematically showing an example of the configuration of the superimposed audio signal in the present embodiment, and corresponds to FIG. FIG. 5A shows the structure of the superimposed audio signal when audio number 1 is designated (that is, when superimposition is performed in the first superposition position relationship). FIG. 5B shows the configuration of the superimposed audio signal when the audio number 2 is designated (that is, when superimposition is performed in the second superposition position relationship).

  As shown in FIG. 5A, in the superimposed audio signal 640 when the audio number 1 is designated, the position Vta of the first audio signal 611 is the first multiplexed audio signal 620 and the second multiplexed audio signal. 630. The positions Vtb, Vtc, and Vtd of the other second to fourth audio signals 612 to 614 do not match between the first multiplexed audio signal 620 and the second multiplexed audio signal 630.

  The superimposed audio signal 640 obtained by superimposing the same two audio signals in a state where the positions coincide with each other doubles the amplitude and increases the volume. On the other hand, the superimposed audio signal 640 in which the same two audio signals are superimposed in a state where the positions do not coincide with each other does not double the amplitude, and the audio is heard as having reverberation or reverberation, and has a contour. Becomes a blurred sound.

  Therefore, in the audio of the superimposed audio signal 640 as shown in FIG. 5A (hereinafter referred to as “superimposed audio”), only the audio of the first audio signal 611 is clearly heard, and the second to fourth audio signals 612 are clearly heard. Each of the sounds ˜614 will be heard indefinitely.

  As shown in FIG. 5B, in the superimposed audio signal 650 when the audio number 2 is designated, only the position Vtb of the second audio signal 612 is the first multiplexed audio signal 620 and the second multiplexed audio. Matches with signal 630. In such superposed audio, only the audio of the second audio signal 612 can be heard clearly.

  Therefore, the voice listening device 400 can switch the multiplexed positional relationship to switch the voice that can be clearly heard and easily make an arbitrary voice signal easily heard.

  As for the multiplexing position relationship, if the delay time of the non-target voice is too short, the difference in how the target voice and the non-target voice are heard becomes small. Conversely, if the delay time of the non-target speech is too long, the non-target speech in the first multiplexed speech signal 620 and the non-target speech in the second multiplexed speech signal 630 are independent. , It sounds like the same sound was output twice. Therefore, it is desirable that all delay times other than 0 (delay times d1 to d3 in FIG. 4B) fall within a numerical range predetermined by experiments, such as several tens of milliseconds to several hundred milliseconds.

  Furthermore, it is desirable that the first to fourth audio signals are shifted by a predetermined time. That is, it is desirable that the delay time d2 is twice the delay time d1 and the delay time d3 is three times the delay time d1. Thereby, the time adjustment unit 420 can adjust the superposition position relationship in units of the delay time d1, and the processing becomes easy.

  The audio output unit 440 in FIG. 2 outputs the superimposed audio signal to the audio output device 500.

  Specifically, the audio output unit 440 transmits the input superimposed audio signal to the audio output device 500.

  The audio output device 500 is, for example, a headphone that is used by a user connected to a personal computer. The audio output device 500 receives the superimposed audio signal transmitted from the audio listening device 400, converts it into audio, and outputs it.

  The audio providing device 300, the audio multiplexing device 100, and the audio listening device 400 include, for example, a CPU and a storage medium such as a RAM (random access memory). In this case, each functional unit described above is realized by the control program being executed by the CPU.

  Such an audio multiplexing system 200 can output two types of multiplexed audio signals having different multiplexing positions of a plurality of audio signals.

  Such two types of multiplexed audio signals can be superimposed in a state where only one of a plurality of audio signals is selectively matched. The voice of the matched voice signal is clearer and easier to hear than the voice of the mismatched voice signal. That is, these two types of multiplexed audio signals can be made easy to selectively listen to any audio signal simply by adjusting the relative positional relationship during superimposition.

  The audio multiplexing system 200 does not need to create a filter many times as in the technique described in Patent Document 1 or perform audio processing for each audio signal as in the technique described in Patent Document 2. Therefore, the audio multiplexing system 200 can make it easier to hear the sound of interest in a state where the processing load is suppressed as compared with the prior art.

  Also, the audio multiplexing system 200 can adjust the superposition positions of the two types of multiplexed audio signals so that the audio signals of the audio numbers designated by the user match. Thereby, the voice multiplexing system 200 can make it easy to hear only the user's desired voice (noticeable sound).

  Also, the voice multiplexing system 200 can acquire speech voices from a plurality of users, multiplex them, and reproduce them. As a result, the voice multiplexing system 200 can realize voice chat with a large number of people in a state where only the target sound is easily heard.

  In addition, since the audio multiplexing system 200 compresses the audio amplitude when the audio signal is multiplexed, it is possible to prevent the reproduction sound from being distorted due to the amplitude of the multiplexed audio signal being excessively large. .

  This is the end of the description of the configuration of each device and system according to the present embodiment.

  Next, the operation of each device according to the present embodiment will be described.

  FIG. 6 is a flowchart showing an example of the operation of the audio multiplexing apparatus 100.

  First, in step S1010, the voice input unit 110 receives each voice signal (first to fourth voice signals) transmitted from the first to fourth voice providing apparatuses 300-1 to 300-4. For example, the voice input unit 110 receives a voice signal every predetermined period, and proceeds to the next step 1020.

  In step S1020, the audio compression unit 111 compresses the amplitude of each received audio signal (first to fourth audio signals) received.

  Then, in step S1030, first audio multiplexing section 120 generates a first multiplexed audio signal and outputs it as a transmission target on the first channel. That is, the first channel is a channel in which all voices are multiplexed without delay.

  In step S1040, the delay processing unit 131 sets a delay determined for each audio signal for each audio signal. That is, the delay processing unit 131 performs processing (hereinafter referred to as “delay processing”) for appropriately delaying each audio signal with a different delay time.

  In step S1050, the second audio multiplexing unit 130 generates a second multiplexed audio signal from the audio signals after delay processing (first to fourth audio signals), and transmits the second multiplexed audio signal on the second channel. Output as the target of. In other words, the second channel is a channel in which each sound is multiplexed in a state where it is shifted from all other sounds.

  In step S <b> 1060, the multiple sound transmission unit 141 transmits the first and second multiple sound signals to the sound listening device 400.

  In step S1070, the voice input unit 110 determines whether or not there has been a request to end the voice multiplexing process by a user operation or the like.

  If there is no termination request (S1070: NO), the voice input unit 110 returns to step S1010. In addition, when there is an end request (S1070: YES), the voice input unit 110 ends a series of processes.

  With this operation, the audio multiplexing apparatus 100 receives a plurality of audio signals from the audio providing apparatus 300, and transmits two types of multiplexed audio signals having different multiplexing positional relationships to the audio listening apparatus 400. It can be transmitted continuously.

  FIG. 7 is a flowchart showing an example of the operation of the audio listening device 400. Note that the time adjustment unit 420 sets, for example, the first superimposed position relationship as an initial state of the predetermined superimposed position relationship.

  First, in step S2010, the multiplex audio receiving unit 410 receives the first multiplex audio signal and the second multiplex audio signal transmitted from the audio multiplexer 100. For example, the multiplex sound receiving unit 410 receives the first multiplex sound signal and the second multiplex sound signal at predetermined intervals, and proceeds to the next step S2020.

  In step S2020, time adjustment section 420 generates a superimposed audio signal from the first multiplexed audio signal and the second multiplexed audio signal. Then, the audio output unit 440 transmits the superimposed audio signal to the audio output device 500. As described above, the superimposed audio signal is obtained by superimposing the first multiplexed audio signal and the second multiplexed audio signal in the current predetermined overlapping position relationship.

  In step S2030, the multiplexed sound receiving unit 410 determines whether or not there has been a request for termination of the sound multiplexing process by a user operation or the like.

  If there is no termination request (S2030: NO), the multiplexed sound receiving unit 410 proceeds to step S2040. In addition, when there is an end request (S2030: YES), the multiplexed sound receiving unit 410 ends a series of processes.

  In step S2040, the operation unit 430 determines whether there is an input value in the plus direction or the minus direction, that is, whether there is an input of a pointer movement operation that designates a voice number.

  If there is a moving operation (S2040: YES), operation unit 430 proceeds to step S2050. If there is no movement operation (S2040: NO), operation unit 430 returns to step S2010.

  In step S2050, operation unit 430 determines whether or not the pointer movement is in the plus direction.

  That is, the operation unit 430 determines whether or not the pointer has been moved in the direction in which the voice number increases, such as from voice number 1 to voice number 2.

  If the pointer movement is in the plus direction (S2050: YES), operation unit 430 proceeds to step S2060. If the pointer movement is in the minus direction (S2050: NO), operation unit 430 proceeds to step S2070.

  In step S2060, the operation unit 430 switches the predetermined superposition position relationship of the time adjustment unit 420 so as to relatively delay the first multiplexed audio signal (that is, the signal of the first channel) from the current state, The process returns to step S2010.

  That is, the operation unit 430 moves the first multiplexed audio signal and the second multiplexed signal at an arbitrary position of the audio signal corresponding to the one larger audio number, such as from the first audio signal to the second audio signal. Match between audio signals. This corresponds to switching the superimposed audio signal from the state shown in FIG. 5A to the state shown in FIG.

  In step S2070, the operation unit 430 switches the predetermined superposition position relationship of the time adjustment unit 420 so as to relatively delay the second multiplexed audio signal (that is, the signal of the second channel) from the current state, The process returns to step S2010.

  That is, the operation unit 430 moves the first multiplexed audio signal and the second multiplexed signal at an arbitrary position of the audio signal corresponding to the smaller audio number, such as from the second audio signal to the first audio signal. Match between audio signals. This is equivalent to switching the superimposed audio signal from the state shown in FIG. 5B to the state shown in FIG.

  With such an operation, the audio listening device 400 can continuously transmit a superimposed audio signal on which two types of multiplexed audio signals having different multiplexing positional relationships of a plurality of audio signals are superimposed to the audio output device 500. . In addition, the audio listening device 400 can adjust the superposition position relationship so that an audio signal desired by the user can be easily heard.

  Note that the operation unit 430 desirably determines the upper limit and the lower limit of the cumulative value of input values, but is omitted here. The upper limit of the cumulative value of the input value is a number obtained by subtracting 1 from the number of audio signals (4-1 = 3 in the present embodiment). Further, the lower limit of the cumulative value of the input value is 0.

  In addition, the operation unit 430 does not set an upper limit and a lower limit on the cumulative value of the input value, sets the cumulative value to the lower limit (0) when the cumulative value exceeds the upper limit, and sets the cumulative value to the upper limit ( 3). Further, in this case, the operation unit 430 can accept only one of the input value in the plus direction and the input value in the minus direction.

  Further, when the operation unit 430 accepts only an input value in one direction, when the cumulative value reaches the upper limit, the operation unit 430 treats the input value as a negative value, and when the cumulative value reaches the lower limit, the input value is increased in the positive direction. It may be handled as a value of.

  This is the end of the description of the operation of each device according to the present embodiment.

  As described above, the audio multiplexing system 200 according to the present embodiment generates two types of multiplexed audio signals having different multiplexing positions of a plurality of audio signals, and uses these superimposed audio signals as their superimposed position relationships. Adjust and output. As a result, the voice multiplexing system 200 can make it easier to hear the sound of interest while simultaneously outputting a plurality of voices with a reduced processing load as compared with the prior art.

  Note that the audio multiplexing system 200 may generate three or more types of multiplexed audio signals, and adjust these superimposed positional relationships to output them. In this case, the time adjustment unit 420 may adjust the superposition position relationship so that only the audio signal designated as the target to be easily heard matches between all the multiplexed audio signals.

  In addition, when the audio signal is digital, the audio multiplexing system 200 may lower the sampling frequency in units of multiplexed audio signals (channel units). For example, the audio multiplexing apparatus 100 transmits the first multiplexed audio signal as it is as a high-quality audio signal, and transmits the second multiplexed audio signal after lowering its sampling frequency. As a result, the audio multiplexing system 200 can reduce the amount of data to be handled and the processing load without deteriorating the sound quality of the superimposed audio signal.

  Also, when the number of multiplexed audio signals (number of channels) is 2, the audio multiplexing system 200 may assign each multiplexed audio signal to the left and right channels of conventional stereo audio. As a result, the audio multiplexing system 200 can share the communication processing of the multiplexed audio signal with the conventional stereo audio system.

  Also, the audio multiplexing system 200 may assign each audio signal to the left and right channels of stereo audio, or may arrange them in a three-dimensional manner in a virtual acoustic space realized by stereo audio. Thereby, the voice multiplexing system 200 can make it easier to distinguish the target voice.

  Also, the audio multiplexing system 200 allows the number of audio signals to be output to be equal to or less than the number obtained by dividing the upper limit value of the delay time by the lower limit value of the delay time so that the delay time falls within the above-defined numerical range. May be limited. Thereby, the voice multiplexing system 200 can prevent the voice from becoming difficult to hear.

  Note that when there are a plurality of audio signals that are easy to distinguish from each other among the plurality of audio signals, the audio multiplexing system 200 may not shift the delay times of the audio signals.

  For example, when the audio multiplexing system 200a outputs a plurality of audio signals arranged in an arc shape in the virtual sound source space, the delay times of the audio signals that are separated from each other are matched. In addition, for example, the audio multiplexing system 200a matches the delay times for speech signals of speech sounds having greatly different pitches.

  Thereby, the audio multiplexing system 200 can increase the number of audio signals to be simultaneously output while preventing the audio from becoming difficult to hear.

(Embodiment 3)
The third embodiment of the present invention is an example in which non-target speech is canceled by inverting the phase of the second multiplexed speech signal.

  FIG. 8 is a block diagram showing an example of the configuration of the speech multiplexing apparatus, speech listening apparatus, and speech multiplexing system according to the present embodiment, and corresponds to FIG. 2 of the second embodiment. The same parts as those in FIG.

  In FIG. 8, a voice multiplexing system 200a includes a voice multiplexing apparatus 100a instead of the voice multiplexing apparatus 100 of FIG. In addition to the delay processing unit 131, the second audio multiplexing unit 130a of the audio multiplexing device 100a includes a phase inverting unit 132a.

  The phase inverting unit 132a inverts the phases of the first audio signal and the second audio signal included in one of the first multiplexed audio signal and the second multiplexed audio signal, respectively. In the present embodiment, phase inversion section 132a inverts all the phases of the first to fourth audio signals of the second multiplexed audio signal. The phase inversion unit 132a may perform the phase inversion before the delay process or after the delay process.

  In the present embodiment, the delay processing unit 131 divides the audio signals included in the second multiplexed audio signal into two groups, and delays all the audio signals of one group with the same delay time. And

  Specifically, the delay processing unit 131 uses the first and third audio signals as non-delay groups, the second and fourth audio signals as delay groups, and the second and fourth audio signals as the first. The delay time is assumed to be delayed.

  FIG. 9 is a diagram schematically showing an example of the configuration of the first and second multiplexed audio signals in the present embodiment, and corresponds to FIG. 4 of the second embodiment.

  The relative positional relationship (first multiplexed positional relationship) of the first to fourth audio signals 611 to 614 in the first multiplexed audio signal 620 shown in FIG. 9A is the same as in the second embodiment. This is the same as the relative positional relationship in the input audio signal 610 shown in FIG.

  On the other hand, the second multiplexed audio signal 630 of the present embodiment shown in FIG. 9B is a first to fourth inverted audio signal obtained by inverting the phases of the first to fourth audio signals 611 to 614, respectively. 611 ′ to 614 ′. Unlike the second embodiment, the relative positional relationship (second multiplexed positional relationship) between the first to fourth inverted audio signals 611 ′ to 614 ′ is the second and fourth delay groups. Only the inverted audio signals 612 ′ and 614 ′ are delayed by the first delay time d1.

  In other words, the above-described position Vtc of the third inverted audio signal 613 ′ matches the position Vta of the first inverted audio signal 611 ′. The above-described positions Vtb and Vtd of the second and fourth inverted audio signals 612 ′ and 614 ′ are respectively delayed by the first delay time d1 with respect to the time t1 corresponding to the position Vta.

  The audio listening device 400 receives the first multiplexed audio signal and the second multiplexed audio signal and superimposes them to generate a superimposed audio signal. At this time, the time adjustment unit 420 of the audio listening device 400 receives the control from the operation unit 430, and performs the above-described predetermined overlapping position between the first overlapping position relationship and the second overlapping position relationship. Switch relationships.

  In this embodiment, voice number 1 is assigned to the delay group (second and fourth voice signals), and voice number 2 is assigned to the non-delay group (first and third voice signals). It shall be.

  FIG. 10 is a diagram schematically showing an example of the configuration of the superimposed audio signal in the present embodiment, and corresponds to FIG.

  As shown in FIG. 10A, in the superimposed audio signal 660 when the audio number 1 (delay group) is designated, the position Vta of the first audio signal 611 and the first inverted audio obtained by inverting the phase thereof. It coincides with the position Vta of the signal 611 ′. Similarly, the position Vtc of the third audio signal 613 matches the position Vta of the third inverted audio signal 613 ′.

  When an audio signal having a reversed phase is superimposed on the audio signal, the audio signal is canceled. Therefore, in the superimposed sound of the superimposed sound signal 660 when the sound number 1 is designated, the sound of the first sound signal 611 and the sound of the third sound signal 613 (the sound of the non-delay group) cannot be heard.

  Further, in the superimposed audio signal 660 when the audio number 1 is designated, the position Vtb of the second audio signal 612 and the position Vtb of the second inverted audio signal 612 ′ are shifted by the first delay time d1. . Similarly, the position Vtd of the fourth audio signal 614 and the position Vtd of the fourth inverted audio signal 614 ′ are shifted by the first delay time d1. Therefore, the sound of the second sound signal 612 and the sound of the fourth sound signal 614 (the sound of the delay group) can be heard although the outline is somewhat blurred.

  On the other hand, as shown in FIG. 10B, the coincidence / non-coincidence of the positions Vta, Vtb, Vtc, Vtd in the superimposed audio signal 670 when the audio number 2 (non-delay group) is designated is shown in FIG. The reverse pattern is the case of the superimposed audio signal 660 shown in FIG. Therefore, in the superimposed voice of the superimposed voice signal 670 when the voice number 2 is designated, the voice of the delay group can hardly be heard and only the voice of the non-delayed group can be heard.

  Therefore, the audio listening device 400 can switch the audio thinning method by switching the multiplexing positional relationship, and can selectively hear only the audio signals of any group of the two groups. Can do.

  Such an audio multiplexing system 200a can invert the phase of one of two types of multiplexed audio signals having different multiplexing positions of a plurality of audio signals and adjust the superposition position relationship for output. . Thereby, the speech multiplexing system 200a can make the non-notable sound inaudible and make it relatively easy to hear the notable sound.

  In addition, when there are two audio signals that are particularly difficult to distinguish, the audio multiplexing system 200a desirably performs grouping so that the audio signals belong to different groups.

  For example, when the audio multiplexing system 200a outputs a plurality of audio signals arranged in an arc shape in the virtual sound source space, grouping is performed so that the groups to which the audio signals belong are alternately different in the order of arrangement. I do. Further, for example, the speech multiplexing system 200a performs grouping so that speech sounds having close pitches belong to different groups.

  Thereby, the audio multiplexing system 200a can increase the number of audio signals to be simultaneously output while preventing the audio from becoming difficult to hear.

(Embodiment 4)
Embodiment 4 of the present invention is an example of a specific mode when the present invention is applied to a portable player that stores and reproduces a large number of audio signals (audio data).

  First, the configuration of each device and system according to the present embodiment will be described.

  FIG. 11 is a block diagram showing an example of the configuration of the speech multiplexing apparatus, speech listening apparatus, and speech multiplexing system according to the present embodiment, and corresponds to FIG. 2 of the second embodiment. The same parts as those in FIG. 2 are denoted by the same reference numerals, and description thereof will be omitted.

  In FIG. 11, the audio multiplexing system 200b includes an audio multiplexing device 100b and an audio output device 500.

  The audio multiplexing apparatus 100b includes an audio input unit 110b instead of the audio input unit 110 of the second embodiment, and further includes a time adjustment unit 420, an operation unit 430, and an audio output unit 440. In the present embodiment, the audio multiplexing device 100b is, for example, a portable player.

  The voice input unit 110b holds a database that stores a large number of voice signals, and acquires a plurality of voice signals from the database. The voice input unit 110b includes a voice compression unit 111, a voice holding unit 112b, and a voice search unit 113b.

  The voice holding unit 112b is the database described above. Each audio signal stored in the audio holding unit 112b is added with meta information of the audio signal.

  Various kinds of information can be applied as the meta information.

  When the database is a collection of audio data of a large number of music pieces, the meta information can include, for example, an artist name and a genre. Further, when the database is a collection of audio data of a large number of lectures, the meta information can include, for example, a date, a speaker name, and a lecture theme. Further, the meta information may include a genre in which the lecture theme is classified.

  Note that the voice input unit 110b itself may perform voice recognition processing on each voice data, for example, and give the recognition result to each voice signal as meta information.

  For example, the voice search unit 113b receives a voice signal condition from the user, and searches the voice holding unit 112b for a voice signal to which meta information satisfying the input condition is added. Then, the voice search unit 113b outputs the searched voice signal to the voice compression unit 111.

  In the following description, it is assumed that a plurality of audio signals are always searched. When only one audio signal is searched, for example, the audio compression unit 111 in the subsequent stage may output the audio signal directly to the audio output unit 440.

  In the present embodiment, first audio multiplexing section 120 outputs the first multiplexed audio signal to audio output section 440. Second audio multiplexing unit 130 outputs the second multiplexed audio signal to time adjustment unit 420.

  In the present embodiment, the first audio multiplexing unit 120 reproduces, for example, by aligning the start positions of all audio signals.

  In this case, the first multiplexing positional relationship described above is a relationship in which the starting positions of all the other audio signals (second audio signals) coincide with the starting position of the first audio signal. The second multiplexing position relationship described above is a relationship in which the start positions of all other sound signals (second sound signals) are delayed by a predetermined time with respect to the start position of the first sound signal.

  Such an audio multiplexing system 200b can select a plurality of audio signals from a large number of audio signals to be held, and can simultaneously output the audio of the selected audio signals. Also, the audio multiplexing system 200b generates two types of multiplexed audio signals having different multiplexing positions of a plurality of audio signals, and adjusts and outputs the superimposed positional relationship between these two types of multiplexed audio signals. Therefore, the audio multiplexing system 200b can make it easy to hear an arbitrary audio signal.

  This is the end of the description of the configuration of each device and system according to the present embodiment.

  Next, the operation of speech multiplexing apparatus 100b according to the present embodiment will be described.

  FIG. 12 is a flowchart showing an example of the operation of speech multiplexing apparatus 100b, and corresponds to FIGS. 6 and 7 of the second embodiment. The same parts as those in FIGS. 6 and 7 are denoted by the same step numbers, and description thereof will be omitted. Note that the audio multiplexing apparatus 100b executes, for example, the following process shown in FIG. 12 each time an instruction to start searching for an audio signal is given.

  First, in step S1011b, the voice search unit 113b searches for a voice signal in the voice holding unit 112b.

  In step S1012b, the voice search unit 113b acquires a plurality of searched voice signals.

  Then, in steps S1020 to S1050, as in the second embodiment, the audio multiplexing apparatus 100b performs the second multiplexing from the plurality of audio signals, the first multiplexed audio signal, and the multiplexing position relationship different from this. And an audio signal.

  And in step S2020-2070, the audio | voice multiplexing apparatus 100b produces | generates a superimposition audio | voice signal, adjusting the superimposition positional relationship according to user operation similarly to the audio | voice listening apparatus 400 of Embodiment 2. Output to the audio output device 500.

  For example, the time adjustment unit 420 performs processing on the second multiplexed audio signal at predetermined intervals. When this period is very short, the audio multiplexing apparatus 100b can switch the target to make the sound easy to hear during the reproduction of the audio signal.

  With such an operation, the audio multiplexing apparatus 100b can output the audio of a plurality of audio signals in a state in which the specific audio is easily heard, and can switch the target to be easily heard during the output.

  This is the end of the description of the operation of speech multiplexing apparatus 100b according to the present embodiment.

  As described above, the speech multiplexing system 200b according to the present embodiment allows the user to simultaneously confirm a plurality of speeches even when the number of speech signals is large, such as when there are many search results. The desired voice can be easily found.

  In addition, since the audio multiplexing system 200b integrates the audio multiplexing device 100 and the audio listening device 400 of the first embodiment, it is possible to eliminate the need for a communication circuit or individual housing between them. . That is, the audio multiplexing system 200b can simplify the entire system.

  Note that the audio multiplexing apparatus 100b may include the phase inversion unit 132a as in the second embodiment. In this case, the same effect as in the second embodiment can be obtained.

  Of the above-described embodiments, Embodiment 2 and Embodiment 3 have described examples in which the audio providing apparatus 300 is provided as an apparatus for inputting an audio signal. In the fourth embodiment, the example in which the audio output device 500 is provided as an apparatus for converting the superimposed audio signal into sound has been described. However, the application of the present invention is not limited to these.

  For example, the audio multiplexing apparatus according to the present invention can be a headset having an audio input function such as a microphone, an audio listening apparatus function, and an audio output function such as a speaker.

  In addition, the method of switching the superposition position relationship between the first multiplex speech signal and the second multiplex speech signal is a method of switching the designated speech number according to the order of the numbers in the above embodiments. It is not limited to.

  The voice listening device according to the present invention accepts designation of a voice number by, for example, inputting a numerical value or pressing a key switch, and switches the superposition position relationship so that the voice signal of the designated voice number can be easily heard. Good.

  Also, the voice listening device, for example, moves the pointer on the virtual axis on which each voice number is arranged, and when the pointer position matches the position of any voice number, the voice signal of that voice number can be easily heard. Alternatively, the superposition position relationship may be switched.

  In addition, the voice listening device may switch the superposition position relationship by accepting, from the user, an operation of sliding the time axis of the second multiplexed voice signal with respect to the time axis of the first multiplexed voice signal, for example. Good.

  Further, the present invention can be applied to various systems and devices other than the above-described voice chat system and portable player. For example, the present invention may be applied to a radio receiver that can simultaneously receive a plurality of radio broadcasts and select a desired radio broadcast sound.

  An audio multiplexing apparatus, an audio listening apparatus, and an audio multiplexing method according to the present invention are an audio multiplexing apparatus, an audio listening apparatus, and an audio multiplexing method that make it easy to hear an attention sound while suppressing a processing load. Useful as.

100, 100a, 100b Audio multiplexer 110, 110b Audio input unit 111 Audio compression unit 112b Audio holding unit 113b Audio search unit 120 First audio multiplexing unit 130, 130a Second audio multiplexing unit 131 Delay processing unit 132a Phase inversion unit 140, 440 Audio output unit 141 Multiplex audio transmission unit 200, 200a, 200b Audio multiplexing system 300 Audio providing device 400 Audio listening device 410 Multiplex audio reception unit 420 Time adjustment unit 430 Operation unit 500 Audio output device

Claims (9)

An audio input unit for inputting the first audio signal and the second audio signal;
A first audio multiplexing unit that generates a first multiplexed audio signal obtained by multiplexing the first audio signal and the second audio signal in a first multiplexing positional relationship;
A second multiplexed audio signal is generated which is obtained by multiplexing the first audio signal and the second audio signal with a second multiplexing positional relationship different from the first multiplexing positional relationship. Two audio multiplexing units;
An audio output unit that outputs the first multiplexed audio signal and the second multiplexed audio signal;
Audio multiplexer. The second audio multiplexing unit includes:
A delay processing unit for delaying the second audio signal;
The speech multiplexing apparatus according to claim 1. The first multiplexing positional relationship is a relationship in which an arbitrary position of the first audio signal matches the position of the second audio signal input at the same timing as the position,
The second multiplexed positional relationship is a relationship in which the position of the second audio signal input at the same timing as the position is delayed by a predetermined time with respect to an arbitrary position of the first audio signal. ,
The speech multiplexing apparatus according to claim 1. The first multiplexing positional relationship is a relationship in which the start position of the first audio signal and the start position of the second audio signal match.
The second multiplexed positional relationship is a relationship in which the start position of the second audio signal is delayed by a predetermined time with respect to the start position of the first audio signal.
The speech multiplexing apparatus according to claim 1. A phase inverting unit for inverting the phases of the first audio signal and the second audio signal, respectively, included in one of the first multiplexed audio signal and the second multiplexed audio signal;
The speech multiplexing apparatus according to claim 1. The audio output unit
The first multiplexed audio signal and the second multiplexed audio are output to an audio listening device that superimposes and outputs the first multiplexed audio signal and the second multiplexed audio signal in a predetermined superimposable positional relationship. Send signal,
The speech multiplexing apparatus according to claim 1. A time adjustment unit that generates a superimposed audio signal obtained by superimposing the first multiplexed audio signal and the second multiplexed audio signal in a predetermined superposition position that can be adjusted;
Based on a user operation, at least a first superimposed positional relationship in which an arbitrary position of the first audio signal matches between the first multiplexed audio signal and the second multiplexed audio signal; An operation of switching the predetermined superposition position relationship between a second superposition position relationship in which an arbitrary position of the second sound signal is coincident between the first multiple sound signal and the second multiple sound signal And further comprising
The audio output unit
Outputting the superimposed audio signal to an audio output device;
The speech multiplexing apparatus according to claim 1. A multiplex sound receiving unit that acquires the first multiplex sound signal and the second multiplex sound signal from the sound multiplex device according to claim 1;
A time adjustment unit that generates a superimposed audio signal obtained by superimposing the first multiplexed audio signal and the second multiplexed audio signal in a predetermined superposition position that can be adjusted;
Based on a user operation, a first superimposed positional relationship in which arbitrary positions of the first audio signal included in each of the first multiplexed audio signal and the second multiplexed audio signal match, and the first Switching the predetermined superposition position relationship between the second superposition position relationship in which the arbitrary positions of the second sound signal included in each of the multiple sound signals of the second and the second multiple sound signals are the same And
An audio output unit that outputs the superimposed audio signal to an audio output device;
Voice listening device. Inputting a first audio signal and a second audio signal;
Multiplexing the first audio signal and the second audio signal in a first multiplexing position relationship to generate a first multiplexed audio signal;
Multiplexing the first audio signal and the second audio signal in a second multiplexing position relationship different from the first multiplexing position relationship to generate a second multiplexed sound signal;
Outputting the first multiplexed audio signal and the second multiplexed audio signal,
Audio multiplexing method.

Images